problem with storing greek chars to a buffer (os linux)

[nass]

hello everyone and happy new year.
i am not sure how to tackle this problem or where is originates from so i am writing here in hope that if you can not help you can at least point me in a direction.

I am writing some little program using only standard c++ library and i am opening a file that contains strings. there are numbers , english and greek letters among the characters. the reading is done fine and the data are then stored onto a buffer (without undergoing any processing) as a few int values ( the lengths of the strings), followed by a long c string - the concatenation of all the strings of the file.
the receive application can then use the lengths of the strings to separate the strings from the string again.

so examining the buffer contents in the shared memory i found that the numbers have their correct corresponding values (0x30 for char '0' etcetc.) and the lengths of the strings in the c string are correct too. but the greek characters and system chars found among them (like spacebar) where wrong - did not have their expected (extended) ascii values. the funny thing is if i printf the strings of the file, they appear correctly on console!!

i have set LC_ALL environment variable in my linux machine to en_US.UTF-8, just in case this is an important detail. and the file was written from vim (not internally from the program). is it possible that the file is written in utf-16 format? utf-8? could it be something wrong in the code?(see below)
thank you in advance for your help


CODE:
once i have the independent strings with 'loadInfoConf()' i serialise them and send them to the shMem using:

string detailsStr;	details.writeStructToStream(GUILastMessage.oNL,GUILastMessage.oTL,GUILastMessage.sNL,GUILastMessage.sLL,
   GUILastMessage.iDL,GUILastMessage.mDL,detailsStr);
strcpy(&GUILastMessage.oAndS_str,detailsStr.c_str());

string detailsStr;	details.writeStructToStream(GUILastMessage.oNL,GUILastMessage.oTL,GUILastMessage.sNL,GUILastMessage.sLL,
   GUILastMessage.iDL,GUILastMessage.mDL,detailsStr);
strcpy(&GUILastMessage.oAndS_str,detailsStr.c_str());

To copy to clipboard, switch view to plain text mode 

---------------------------------------------------------------

void InfoClass::loadInfoConf()
{
	string curLine="",sumLine="", lines[6];
	int i=0;
 
	ifstream infoConfFile(INFOCONF_FILENAME);
	if (infoConfFile.is_open())
	{
		while (!infoConfFile.eof())
		{
			getline(infoConfFile,curLine);
			if ((curLine!="EOV"))
			{
				if (curLine!="")
					sumLine=sumLine+curLine+"\n";
			}
			else
			{
				lines[i]=sumLine;
				sumLine="";
				i++;
			}
		}
		infoConfFile.close();
 
	//remove the last char which is always \n due to the way 
	//the file open parser (just above) works
	lines[0].erase( (lines[0].length()-1) ,1);
	lines[1].erase( (lines[1].length()-1) ,1);
	lines[2].erase( (lines[2].length()-1) ,1);
	lines[3].erase( (lines[3].length()-1) ,1);
	lines[4].erase( (lines[4].length()-1) ,1);
	lines[5].erase( (lines[5].length()-1) ,1);
 
	info.operatorTel = lines[0];
	info.operatorName = lines[1];
	info.stationNum = lines[2];
	info.stationLoc = lines[3];
	info.instDate = lines[4];
	info.maintDate = lines[5];
 
	}
	else
		cout<<"could not open info file\n";
}

void InfoClass::loadInfoConf()
{
	string curLine="",sumLine="", lines[6];
	int i=0;

	ifstream infoConfFile(INFOCONF_FILENAME);
	if (infoConfFile.is_open())
	{
		while (!infoConfFile.eof())
		{
			getline(infoConfFile,curLine);
			if ((curLine!="EOV"))
			{
				if (curLine!="")
					sumLine=sumLine+curLine+"\n";
			}
			else
			{
				lines[i]=sumLine;
				sumLine="";
				i++;
			}
		}
		infoConfFile.close();
	
	//remove the last char which is always \n due to the way 
	//the file open parser (just above) works
	lines[0].erase( (lines[0].length()-1) ,1);
	lines[1].erase( (lines[1].length()-1) ,1);
	lines[2].erase( (lines[2].length()-1) ,1);
	lines[3].erase( (lines[3].length()-1) ,1);
	lines[4].erase( (lines[4].length()-1) ,1);
	lines[5].erase( (lines[5].length()-1) ,1);

	info.operatorTel = lines[0];
	info.operatorName = lines[1];
	info.stationNum = lines[2];
	info.stationLoc = lines[3];
	info.instDate = lines[4];
	info.maintDate = lines[5];

	}
	else
		cout<<"could not open info file\n";
}

To copy to clipboard, switch view to plain text mode 

------------------------------------------------------------------------------------

void InfoClass::writeStructToStream (int &on_len, int &ot_len, int &sn_len, int &sl_len,
      int &id_len, int &md_len, string &buffer)
{
	on_len=info.operatorName.length();
	ot_len=info.operatorTel.length();
	sn_len=info.stationNum.length();
	sl_len=info.stationLoc.length();
	id_len=info.instDate.length();
	md_len=info.maintDate.length();
 
	//int totLen=on_len+ot_len+sn_len+sl_len+id_len+md_len/*+1*/;
	string tmpbuf="";
	tmpbuf += info.operatorTel;
	tmpbuf += info.operatorName;
	tmpbuf += info.stationNum;
	tmpbuf += info.stationLoc;
	tmpbuf += info.instDate;
	tmpbuf += info.maintDate;
 
	buffer=tmpbuf;
}

void InfoClass::writeStructToStream (int &on_len, int &ot_len, int &sn_len, int &sl_len,
      int &id_len, int &md_len, string &buffer)
{
	on_len=info.operatorName.length();
	ot_len=info.operatorTel.length();
	sn_len=info.stationNum.length();
	sl_len=info.stationLoc.length();
	id_len=info.instDate.length();
	md_len=info.maintDate.length();
	
	//int totLen=on_len+ot_len+sn_len+sl_len+id_len+md_len/*+1*/;
	string tmpbuf="";
	tmpbuf += info.operatorTel;
	tmpbuf += info.operatorName;
	tmpbuf += info.stationNum;
	tmpbuf += info.stationLoc;
	tmpbuf += info.instDate;
	tmpbuf += info.maintDate;

	buffer=tmpbuf;
}

To copy to clipboard, switch view to plain text mode 

-------------------------------------------------------------------------

[wysota]

Maybe the string contains some \0 characters because of using unicode? Strcpy would then only copy the part of the string until the first occurence of the null character. You shouldn't use strcpy for extended strings. Either use memcpy or encode the string somehow to contain only valid values.

[nass]

i did change that but nothing changed.
english characters and numbers work fine.
if i add greek the application can not handle it.
i just tried the opposite too. from the other application ( a GUI) i type in english chars, and this application (the controller) reads them fine and saves them fine. if i type greek from the GUI, the RIGHT ascii characters codes appear on the sharedMem but the controller can not read them (a cout at runtime outputs little squares on the console) and in the file what is saved is 'ÈáÃ*Üóçò' (if it is readable - that does not look lie greek chars to my end).

i tried in main.cpp to do a setlocale(LC_ALL,"el_GR.ISO-8859-7"); but no luck either.

[wysota]

How do you save that buffer to a shared mem segment? How do you extract the data from the string? Do you use c_str()? What happens if you write that data to a file instead of storing it in the shared segment?