※ 引述《LCL2 (新年快熱~)》之銘言：
: ※ 引述《ydaniel00 (Daniel)》之銘言：
: : Loading the covtype file takes anywhere from 5 to 10 minutes
: : on a P4 2.8G depending on CPU usage. However, the fact that
: : covtype matrix needs 1 GB of RAM may be a potential
: : problem.
: I recommend using gc() function before and after file loading to
: release some system memory from R.
: For covtype matrix, we have 55*581012 entries. Assume each
: entries takes 8 bytes (type=double), overall memory = 255645280 bytes
: = 243.8 Mb. With original loaded file size 86Mb, at least 330 Mb is
: necessary.
: In my experience, it's usually more, but about 400Mb is the upper
: bound for most evaluation times (Sometimes R may wierdly rush to 600Mb, but
: not often, don't know why, it should take time to understand its
: underlying.)
: I'm interesting about whether someone uses machine with less than
: 500 Mb(or even 256) mem for this job running, maybe the Hard Disk
: will dancing around all the time.
I spend about 5.5 mins to read the covtype data, but my RAM is only 256MB.
My CPU is AMD 1.7GHz.
Because we know the attributes of the covtype, so we can customized the
the reading file procedure for the covtype file.
As the professor said, there are 40 binary attributes and only one will
exists, so we can use only one column to record this.
So the total 55 attributes become 13.