strtok and '\r\n'

[mickey]

char delims = ',';
char* result = NULL;
result = strtok( _buffer, delims );
vector<double> v;
while (result != NULL) { //and !endOfLine
result = strtok( NULL, delims );
v.push_back( atof(result) );
}
InsertInVectorOfVector(v)

Hello,
I have to read the _buffer into a a vector; problem is that I need also to check when I step over the '\r\n', because in this case I need to insert 'v' in a vector of vector; Is there a way to do that control using strtok() ?

Thanks,

[lvi]

You could use a copy of buffer and use strtok(_buffer, '\r') on that and compare the pointers. But if _buffer is very large, that's not really an option.

My suggestion would be to use QByteArray and its members QByteArray::indexOf() and QByteArray::left() and optionally QByteArray::simplified() to get the data from your buffer.

[mickey]

You could use a copy of buffer and use strtok(_buffer, '\r') on that and compare the pointers. But if _buffer is very large, that's not really an option.
My suggestion would be to use QByteArray and its members QByteArray::indexOf() and QByteArray::left() and optionally QByteArray::simplified() to get the data from your buffer.

I can't use qt library; only the c++ standard library

[lvi]

In that case...
I would first tokenize the buffer by strtok(_buffer, '\n') and then tokenize by strtok(_buffer, ','):

char* line = strtok(_buffer, '\n');
while (line) {
char* c = strtok(_buffer, ',');
while (c) {
double number = atof(c);
c = strtok(NULL, ',');
}
line = strtok(NULL, '\n');
}


Or you could use std::string::first_index_of() and std::string::substr() to achieve the same:

int lineStart = buffer.first_index_of('\n');
int lineEnd = buffer.first_index_of('\n', lineStart + 1);
std::string line = buffer.substr(lineStart, lineEnd - lineStart);

In some loop... str::first_index_of returns std::npos if the delimeter is not found.

Since your lines are terminated by \r\n, you need to use do something smart with the pointers/indices (e.g. increment by 1 appropriately).

Use code at own risk ;-)

[mickey]

Hello,
IMHO there's an error in the first version of your code ( the second strtok); this is my version of code (probably is what you wanted); anyway it doens't work because at //*, line takes "nullptr" and this first loop end...any hints?

vector<double> v;
vector< vector<double> > doubleVector;
char* line;
line = strtok(_buffer, "\r\n");
while ( line != NULL ) {
cout << line << endl;
char* value = strtok(line, ",");
while ( value != NULL) {
cout << value << endl;
v.push_back( strtod(value, NULL) );
value = strtok(NULL, ",");
}
doubleVector.push_back(v);
v.clear();
line = strtok(NULL, "\r\n"); //* <------ problem
}

[lvi]

You are right about the second strtok. But my code is flawed anyway ;)

My best guess would be that on line #15 the buffer has no newlines left:


// buffer: 0,1,2,3,4\r\n5,6,7,8,9\r\n
line = strtok(buffer, "\r\n"); // buffer: 0,1,2,3,4\0\n5,6,7,8,9\r\n
value = strtok(line, ','); // buffer: 0\01,2,3,4\0\n5,6,7,8,9\r\n
while ( value != NULL ) {
value = strtok(NULL, ',');
}
// at this point, buffer: 0\01\02\03\04\0\n5,6,7,8,9\r\n

You'd need to fiddle around with some pointers to get past the \0. But then again you need to be careful not to get a segfault at the end of your buffer or something.
I suggest you use std::string, it will save you tons of trouble :)

[mickey]

hello,
I'm trying to work with this code but it doens't work with ','; any helps, please?

istringstream is("100,200,300", istringstream::in);
int num;
vector<int> vec;
is.exceptions(~ios::goodbit);
try {
while ( is.good() ) {
is >> num;
vec.push_back(num);
if (is.peek() == ',')
is.ignore(1);
}
}
catch(ios_base::failure failure) {
assert (!is.good());
if (!is.eof()) {
is.clear();
}
}

It works fine only with a space between the numbers (but even in this case, at the end of scanning, an exception is thrown