PDA

View Full Version : read a huge file over 100GB



jesse_mark
23rd April 2013, 22:34
I want to read a huge binary file and parse it. the file can be over 100GB.


any idea how can i do that ??
thanks

wysota
24th April 2013, 00:04
Piece by piece :)

Santosh Reddy
24th April 2013, 00:04
Just like any other binary file, only take care to read only the amount of data which after parsing and decoding can fit in the system's available RAM

ChrisW67
24th April 2013, 00:11
Reminds me of the last time I tried to eat an elephant... :)

The feasibility/difficulty of parsing a binary file piecemeal is strongly dependent on the internal structures of the file, something we do not have information on.