strtok and '\r\n'

[mickey]

char delims = ',';
	char* result = NULL;
	result = strtok( _buffer, delims );
	vector<double> v;
		while (result != NULL) { //and !endOfLine
			result = strtok( NULL, delims );
			v.push_back( atof(result) );
		}
         InsertInVectorOfVector(v)

char delims = ',';
	char* result = NULL;
	result = strtok( _buffer, delims );
	vector<double> v;
		while (result != NULL) { //and !endOfLine
			result = strtok( NULL, delims );
			v.push_back( atof(result) );
		}
         InsertInVectorOfVector(v)

To copy to clipboard, switch view to plain text mode 

Hello,
I have to read the _buffer into a a vector; problem is that I need also to check when I step over the '\r\n', because in this case I need to insert 'v' in a vector of vector; Is there a way to do that control using strtok() ?

Thanks,

[lvi]

You could use a copy of buffer and use strtok(_buffer, '\r') on that and compare the pointers. But if _buffer is very large, that's not really an option.

My suggestion would be to use QByteArray and its members QByteArray::indexOf() and QByteArray::left() and optionally QByteArray::simplified() to get the data from your buffer.

[mickey]

You could use a copy of buffer and use strtok(_buffer, '\r') on that and compare the pointers. But if _buffer is very large, that's not really an option.
My suggestion would be to use QByteArray and its members QByteArray::indexOf() and QByteArray::left() and optionally QByteArray::simplified() to get the data from your buffer.

I can't use qt library; only the c++ standard library

[lvi]

In that case...
I would first tokenize the buffer by strtok(_buffer, '\n') and then tokenize by strtok(_buffer, ','):

char* line = strtok(_buffer, '\n');
while (line) {
    char* c = strtok(_buffer, ',');
    while (c) {
        double number = atof(c);
        c = strtok(NULL, ',');
    }
    line = strtok(NULL, '\n');
}

char* line = strtok(_buffer, '\n');
while (line) {
    char* c = strtok(_buffer, ',');
    while (c) {
        double number = atof(c);
        c = strtok(NULL, ',');
    }
    line = strtok(NULL, '\n');
}

To copy to clipboard, switch view to plain text mode 

Or you could use std::string::first_index_of() and std::string::substr() to achieve the same:

int lineStart = buffer.first_index_of('\n');
int lineEnd = buffer.first_index_of('\n', lineStart + 1);
std::string line = buffer.substr(lineStart, lineEnd - lineStart);

int lineStart = buffer.first_index_of('\n');
int lineEnd = buffer.first_index_of('\n', lineStart + 1);
std::string line = buffer.substr(lineStart, lineEnd - lineStart);

To copy to clipboard, switch view to plain text mode 

In some loop... str::first_index_of returns std::npos if the delimeter is not found.

Since your lines are terminated by \r\n, you need to use do something smart with the pointers/indices (e.g. increment by 1 appropriately).

Use code at own risk ;-)

[mickey]

Hello,
IMHO there's an error in the first version of your code ( the second strtok); this is my version of code (probably is what you wanted); anyway it doens't work because at //*, line takes "nullptr" and this first loop end...any hints?

vector<double> v;
	vector< vector<double> > doubleVector;
	char* line;
	line = strtok(_buffer, "\r\n");
	while ( line != NULL ) {
		cout << line << endl;
		char* value = strtok(line, ",");
			while ( value != NULL) {
				cout << value << endl;
				v.push_back( strtod(value, NULL) );
				value = strtok(NULL, ",");			
			}		
		doubleVector.push_back(v);
		v.clear();
		line = strtok(NULL, "\r\n");	//*	<------ problem
	}

vector<double> v;
	vector< vector<double> > doubleVector;
	char* line;
	line = strtok(_buffer, "\r\n");
	while ( line != NULL ) {
		cout << line << endl;
		char* value = strtok(line, ",");
			while ( value != NULL) {
				cout << value << endl;
				v.push_back( strtod(value, NULL) );
				value = strtok(NULL, ",");			
			}		
		doubleVector.push_back(v);
		v.clear();
		line = strtok(NULL, "\r\n");	//*	<------ problem
	}

To copy to clipboard, switch view to plain text mode 

[lvi]

You are right about the second strtok. But my code is flawed anyway

My best guess would be that on line #15 the buffer has no newlines left:

// buffer: 0,1,2,3,4\r\n5,6,7,8,9\r\n
line = strtok(buffer, "\r\n");   // buffer: 0,1,2,3,4\0\n5,6,7,8,9\r\n
value = strtok(line, ',');       // buffer: 0\01,2,3,4\0\n5,6,7,8,9\r\n
while ( value != NULL ) {
    value = strtok(NULL, ',');
}
// at this point, buffer: 0\01\02\03\04\0\n5,6,7,8,9\r\n

// buffer: 0,1,2,3,4\r\n5,6,7,8,9\r\n
line = strtok(buffer, "\r\n");   // buffer: 0,1,2,3,4\0\n5,6,7,8,9\r\n
value = strtok(line, ',');       // buffer: 0\01,2,3,4\0\n5,6,7,8,9\r\n
while ( value != NULL ) {
    value = strtok(NULL, ',');
}
// at this point, buffer: 0\01\02\03\04\0\n5,6,7,8,9\r\n

To copy to clipboard, switch view to plain text mode 

You'd need to fiddle around with some pointers to get past the \0. But then again you need to be careful not to get a segfault at the end of your buffer or something.
I suggest you use std::string, it will save you tons of trouble

[mickey]

hello,
I'm trying to work with this code but it doens't work with ','; any helps, please?

istringstream is("100,200,300", istringstream::in);
	int num;
	vector<int> vec;
	is.exceptions(~ios::goodbit);
	try {
		while ( is.good() ) {
			is >> num;
			vec.push_back(num);
			if (is.peek() == ',') 
				is.ignore(1);				
		}
	}
	catch(ios_base::failure failure) {
		assert (!is.good());
		if (!is.eof()) {
			is.clear();  
		}
	}

istringstream is("100,200,300", istringstream::in);
	int num;
	vector<int> vec;
	is.exceptions(~ios::goodbit);
	try {
		while ( is.good() ) {
			is >> num;
			vec.push_back(num);
			if (is.peek() == ',') 
				is.ignore(1);				
		}
	}
	catch(ios_base::failure failure) {
		assert (!is.good());
		if (!is.eof()) {
			is.clear();  
		}
	}

To copy to clipboard, switch view to plain text mode 

It works fine only with a space between the numbers (but even in this case, at the end of scanning, an exception is thrown