read file from end to beginning..

[soul_rebel]

its really a rather simple thing, i want to preview a textfile by showing its last 5 lines...
is there an easy way i can read the last 5 lines from a file without parsing it entirely? or maybe even clearer: how can i find out how many lines the file has, how can i jump to certain line?

thanks!

[helloandrew]

In *nix systems there's the 'tail' command. Find out it's source code and explore it...

[michel]

its really a rather simple thing, i want to preview a textfile by showing its last 5 lines...
is there an easy way i can read the last 5 lines from a file without parsing it entirely? or maybe even clearer: how can i find out how many lines the file has, how can i jump to certain line?

thanks!

I don't know if Qt has a similar function in its QFile class, but in raw file handling you could probably move the cursor of the file to the end with lseek(QFile::handle(), 0, SEEK_END).
Then just keep using lseek and work backwards locating the newlines. Hold on, I'll just check the docs quickly. Yeah, QFile inherits one called seek() that does this. Just it in conjunction with QFile::size() to send the cursor to the end of the file. It's going to be a pain in the ass for you to scan those lines backwards though Have fun. If you really want to do it, I would guess the best bet would be to read the entire file into a string if it's not insanely large, then use

int QString::lastIndexOf ( QChar ch, int from = -1, Qt::CaseSensitivity cs = Qt::CaseSensitive ) const

int QString::lastIndexOf ( QChar ch, int from = -1, Qt::CaseSensitivity cs = Qt::CaseSensitive ) const

To copy to clipboard, switch view to plain text mode 

with '\n' as your QChar and an integer to hold the offset. Make a loop counting down from 4 to 0, and create a QStringList with the last 5 lines using filebuffer.at(offset+1). Remember, if you make the loop count upwards instead of downards, all your strings will be in reverse order.

But no, you can't find out how many lines a file has without reading it. I mean, how else is it going to know how many newlines your file has if it can't scan the entire file and count them all for you? Counting the number of lines is no different from counting the number of times your file contains the letter 'a' or a comma. You can't magically tell without reading the entire file and doing some matching. Unless your text file is like 400000000000000000000000000000000 lines long or it has to be loaded a billion times sequentially, you'll save yourself about half an hour of unnecessary work by just doing it from front to back.

[soul_rebel]

i know tail but it is rather complicated to understand undocumented unix code, also i do not want to invoke a pipe or fork a shell to execute the command....
after all i just want to do simple file i/o !!

[wysota]

A simple, but not the quickest solution is something like this:

QString lines[5];
int index = 0;
QFile file("xxx");
file.open(QIODevice::ReadOnly);
while(!file.atEnd()){
  lines[index] = file.readLine();
  index = (index+1) %5
}
for(int i=0;i<5;i++) std::cout<< lines[index++ % 5];

QString lines[5];
int index = 0;
QFile file("xxx");
file.open(QIODevice::ReadOnly);
while(!file.atEnd()){
  lines[index] = file.readLine();
  index = (index+1) %5
}
for(int i=0;i<5;i++) std::cout<< lines[index++ % 5];

To copy to clipboard, switch view to plain text mode 

[soul_rebel]

Yeah, QFile inherits one called seek()

in qt3 it doesnt...

but i could get the last position with size and move backwards with at() until i reach the 5th-last line and then simply readline...
i'll try that now...

@wysota: since your example parses the fil from the beginning, it could be rather slow in my app (the file could well be a couple of megs)

[soul_rebel]

ok this does it:

QString UpdateAvSrc::getLastLinesFromFile(QString pathToFile, int lines)
{
	QFile file(pathToFile);
 
	if (!file.exists())
		return "file doesn't exist...";
 
	file.open(IO_ReadOnly);
 
	file.at(file.size()-1);
 
	int count = 0;
 
	while ( (count <=lines) && (file.at() > 0) )
	{
		QChar ch = file.getch();
		file.at(file.at()-2); /// minus 2 because getch moves one forward
 
		if (ch == '\n')
			count++;
 
	}
 
	QString r = file.readAll();
 
	file.close();
 
	return r;
}

QString UpdateAvSrc::getLastLinesFromFile(QString pathToFile, int lines)
{
	QFile file(pathToFile);
	
	if (!file.exists())
		return "file doesn't exist...";

	file.open(IO_ReadOnly);
	
	file.at(file.size()-1);
	
	int count = 0;
	
	while ( (count <=lines) && (file.at() > 0) )
	{
		QChar ch = file.getch();
		file.at(file.at()-2); /// minus 2 because getch moves one forward
		
		if (ch == '\n')
			count++;
		
	}

	QString r = file.readAll();
	
	file.close();
	
	return r;
}

To copy to clipboard, switch view to plain text mode 

[wysota]

@wysota: since your example parses the fil from the beginning, it could be rather slow in my app (the file could well be a couple of megs)

Yes, I know. I said it's simple, not that it's fastest. But reading char by char and making a seek every read is not efficient too. The fastest thing would be probably to make a kind of binary search -- split the file in half, take the second half, split it in half, take the second half, etc. until you have a single character. If it's a newline, mark the newline as found and go back one step, take the preceding character, check if it's newline, etc. If you do a smart check of each group, finding a n-th newline from the end of the file shouldn't be hard. Of course, in the worst case you have to scan the whole file...

[soul_rebel]

if i only want the last three or five lines of a logfile that may have up to 40000 lines it is faster to read char by char from the end than continously splitting the thing in halves, is it not?
thanks anyway!

[wysota]

if i only want the last three or five lines of a logfile that may have up to 40000 lines it is faster to read char by char from the end than continously splitting the thing in halves, is it not?
thanks anyway!

Provided that you know that it contains 40000 lines, than yes. But what if it contains three or four (huge) lines?

Besides, it is never worth reading char by char. Every device reads (at least) a complete block at once, so you can safely go 1024 by 1024 bytes at a time. It'll be much faster this way. And if you notice, that's like splitting the file in "halves" up to a 1k slice -- the algorithm is always the same, no matter if you start from beginning, end or from the middle. The whole idea is to have a way to check if a slice contains a new line without scanning each byte, to gain some computing time. If it doesn't, it's not worth checking it. For example on Windows you'd have to check only each second byte, as the newline consists of two characters. Another way would be to use 32b calculations to check whether a block may contain a newline.

One way or the other, if you really value your time -- just cut&paste the code from "tail"

[zxzxy1988]

mark it... I was confused by the question several days ago

[ChrisW67]

Did you really need to wake up a six year old thread to add a nonsensical comment?