Re: reading file with multi-thread
Excellent explanation, thanks!Originally Posted by wysota
Because disk access is not multithreaded. There is a single (or 'single') header in the disk that sequentially reads data based on pending requests. If you read two or more files that are in different sectors of the disk the header has to jump back and forth between the sectors to do the reading or optimizes reading in series (reading a few sectors from one file while other files are getting starved) which in both cases slows the whole process down. The situation might be worsened by caching issues ("cache flickering" or whatever that's called in English) where you constantly overwrite cached data that you are going to need in a moment with data that you need currently.
If you were reading those files from different physical devices, multithreading would have made things faster. You can optimize reading with reading the data in large chunks and caching it in memory (i.e. by using mmap()) or by reading from a ramdisk (try comparing compilation of a large project from disk and from ramdisk). There are also prefetching calls in your operating system you might use to ask the system to prefetch and store a file in its system buffers for you. Using a different operating system with a different disk driver might also help. It's not a discovery that Linux reads FAT disks faster than Windows does
By the way - you can see this "feature" in action when using a system that is constantly swapping memory to/from disk. One thing is that data transfer rate is decreased because more data needs to be read/written and another is the header spends most of its time moving, not accessing data. You can hear that the disk becomes much louder when the machine is swapping and doing something else in the same time. The same issue applies to fragmented file systems - the header jumps back and forth and the cached data is useless. That's why adding memory to your computer makes a larger difference than replacing the disk with a faster one or replacing the CPU with a faster one. Of course once your computer's memory requirements get saturated, the difference is much smaller although still significant because decent operating systems buffer data in memory that's not allocated for other processes. That's one of the reasons why Vista's memory usage seems so high.
For files in different physical devices, are you sure multi-thread will be faster? My observation is that it will be even slower due to bandwidth limitation.
If those disks belong to different computers, then sure it will be faster, but we won't need to discuss this scenario
We have a cluster with more than 2000 computers nodes with one giant disk server, if the 2000 nodes read/write at the same time, you may just have to kill the job...
I just get to my office, so here is the actual number in my test:
File size 1098 x 1168 x 1001 x 335 (float points), each read/write a block 1098 x 1168 points:
If only one process read/write, it take 15333 second (4.15'33"), speed = 107 MB/second
If two processes read/write two files of same size:
process 1 take 52300 seconds (14.31'40"), speed = 31.37 MB / second
process 2 take 55254 seconds (15.20"54"), speed = 29.69 MB / second
Reply With Quote
Bookmarks