send binary file over serial port with QSerialDevice

[ilpaso]

Hi all,
I'm using the class QSerialDevice in order to send datas over the serial port.
I compiled this class and it works. I write a word and it sends over the serial port.

Now I need to transfer a binary file.
Inside the QSerialDevice class there are these functions:
qint64 write(const char *data, qint64 maxSize);
qint64 write(const char *data);
qint64 write(const QByteArray &byteArray);

Please can you help me to send whole binary file and send the datas using one of these functions and Qfile?
I think it is easy but I'm not expert with data streams.

Thank you in advance

ilpaso

[high_flyer]

Please can you help me to send whole binary file and send the datas using one of these functions and Qfile?
Sure, if you explain what problem you have.
And if the problem is - that you have no idea how this is done, then first read:
QIODevice
and
QDataStream and QTextStream.

[ilpaso]

Hi high_flyer,
thank you for your replay.

I open the file and I convert the file into Qbytearray variable with the function "QByteArray QIODevice::readAll()".
After this I use the function "qint64 write(const QByteArray &byteArray)" to send the datas.

I don't know if this is the right way (or the better way). It seems do not work.

Thank you

ilpaso

[high_flyer]

It seems do not work.
QSerialDevice is not a Qt class, but a custom class made by someone.
Usually these class will come with some documentation on how to use them - did you check that?
Usually they also include some examples, you might want to have a look at those too.
I have never used QSerialDevice, so I can't tell if its any good.
I have however used QextSerialPort, and it works quite well.

The problem you have can be all sorts of things - from failing in QSerialDevice to bad serial configuration on your part (baud rate stop bits etc).
You deliver hardly any data as to the checks you made to all the various things that can go wrong, so its really hard to tell.

[ilpaso]

Ok I understand the problem to answer to a not specific question.
QSerialDevice has working examples but I can only send text. So the boud rate and all the serial port settings are ok.
In the examples there is nothing for an entire binary file.

There is not a complete documentation but I read this class derived from QIODevice class.

I don't know if this is enough in order to help me.

Thank you

ilpaso

[high_flyer]

There is no large diference between sending textual or binary data.
Read about QDataStream.

[ilpaso]

thanks high_flyer for your answer,
I read about QDataStream. As you can understand I'm a newbie in Qt.
It is easy read a file with QDataStream but I'm not able to send the stream to the serial port.
QSerialDevice inherits from QIODevice and send over serial port should not be difficoult but I need some clues in order to read the binary file (don't modify it) and send it over the port.
QextSerialPort is more famous than QSerialDevice but there is the same problem.

[marcvanriet]

I'm not able to send the stream to the serial port.

Do you mean nothing is transmitted over the serial port ? Or only part of the datastream ? Or is your problem on the receiving side ? What do you have on the receiving side ? Another computer or some sort of electronics device ? If you connected it to another com-port, you can check with any 'hyperterminal' application if something is received. An oscilloscope may help to check what is transmitted also.

If you create a small QByteArray with readable text in it, and then use "qint64 write(const QByteArray &byteArray)", does it get sent or not ?

Best regards,
Marc

[high_flyer]

QSerialDevice inherits from QIODevice and send over serial port should not be difficoult but I need some clues in order to read the binary file
If you tell us where exactly your difficulty is, we could suggest solutions.
The best way would be to show us what it is you did, what your intent was, and what is the problem you have with the code.

[ilpaso]

Hi all,
thanks for the help. I explain you my intent and the problems.
The intent is to connect my Qt application to a serial printer with its own protocoll. In order to do this I've to send 2 files: the first is a binary file and the second is a text file with some commands inside. I've some commands in order to check the connection and the number of bytes I sent.
The connection is a Rs232 with a XON/XOFF control.
I tried 2 classes: QextSerialPort and QSerialDevice. My choice was on QSerialDevice but the 2 classes are similar.
I creare a QByteArray reading the Qfile "textfile.bin" and I send it with "qint64 write(const QByteArray &byteArray)" function. It works!
The problem is to send the "imageBinary.bin" with the same procedure. My Qt application returns "32Kbyte sent!" but the device receives only 7Kbyte. I'm not able to see what there is inside the file with a text editor because this is binary file.
I'm sure the printer is ok because with its windows software it works.

I suppose the problem is in the conversion from the binary file to QbyteArray. So I read something about QDataStream and QTextStream. I suppose QDataStream reads raw datas without a conversion.
Another hypothesis could be that the QSerialDevice class has bug but I don't think.

Maybe this is enough in order to help me.
Thank you very much for your help and sorry for my Qt incompetence and my English.

ilpaso

[squidge]

Have you configured the serial port to work with XON/XOFF flow control?

Do you stop sending data when XOFF is signalled to you?, and resume when XON is signalled?

I assume that your application has a layer between the serial port driver (QSerialDevice) and your application to manage the flow control between your application and the printer ? Otherwise the QSerialPort will transmit all 32KB in one go, and the printer will ignore most of it as it was not ready.

[high_flyer]

QSerialDevice has working examples but I can only send text. So the boud rate and all the serial port settings are ok.
The serial settings (such as baud rate, flow control etc) are specific to the connection you are targeting - in your case the printer.
So if you have correct settings for the examples, it doesn't mean they are correct for your device.
Do you know what your serial settings should be?

[ilpaso]

Hi squidge,
This night I'll read all the source code of the QSerialDevice in order to understand where is the problem. Often I think that a "ready to use" class is ready to use but often is not.
I set the XON/XOFF and I was sure that the control was implemented inside the QSerialDevice class.

bye
ilpaso

[squidge]

It may well be done there or in the OS itself.

Regardless however, the serial device must be told that this is what you want.

[ilpaso]

Regardless however, the serial device must be told that this is what you want.
In the device technical specifications they wrote "XON/XOF" flow control.

[squidge]

Does it also tell you the other parameters, such as baud rate, data bits, stop bits ?

You might also find it useful to post your code that doesn't work.

[marcvanriet]

People, ilapso was able to send a text file and get readable output on the printer. So baudrate and such are OK. It's not a trivial issue.

Ilapso,

Don't be too hard on yourself. Serial communications is always a pain.

The XON/XOFF is rarely used and probably not in the samples. When and how do you set this ? XON/XOFF is implemented in the operating system, and QSerialDevice sets the corresponding flags when opening the serial port.

Can you set the printer to some other kind of handshaking ? E.g. RTS/CTS.

A quick-and-dirty fix may be to send the image data in smaller portions and wait a bit in between, so that the printer is always able to process the data.

As a test, you could set handshaking off, transmit the binary image, and see if the printer sends and XOFF back. You could also do the XON/XOFF handshaking yourself in software, sending smaller portions of data and checking if XON/XOFF is received.

Best regards,
Marc

[ilpaso]

thank you marcvanriet,
you have understood all my problems!
I do not understand why a class for serial communication is not ready to manage the xon/xoff flow control (or other flow control) in a easy way and so I think I'm not able to use it.
I'll try to send the file in smaller partitions. This is a good idea!

This is my code:
I set the Xon/Xoff:


if (!port->setFlowControl(AbstractSerial::FlowControlXonXoff) ) {
qDebug() << "Set flow " << AbstractSerial::FlowControlXonXoff << " error.";
return;
}

.....
inside the function "writeBinaryFile(QString fileName)" I send the datas


{
QFile file(fileName);
if (!file.open(QIODevice::ReadOnly)){
qDebug() << "Open file fault";
start(false);
port->close();
return;
}
QByteArray data = file.readAll();
quint64 r = port->write(data);
port->waitForBytesWritten(responseTimeout);
port->waitForReadyRead(responseTimeout);
cout<<r<<endl;
if (r == quint64(data.size())) qDebug() << "Writed text file.";
else qDebug() << "Bytes writed small: " << r;
}


when I send a small text file all is ok. When I send a bigger binary file (32KByte) printer returns only 7Kbyte read.

regards
ilpaso

[marcvanriet]

Seems OK at first sight.

By the way, it doesn't matter if you first open the port and then do port->setFlowControl or not. If you do the settings first and then open the port, QSerialDevice will make the settings on the newly opened port.

You mentioned that your application says that 32KB is sent, and that the printer only receives 7 KB, right ? So your application says "Writed text file" but the printer hasn't received it ?

You could take two serial ports and see what is received on the other side using e.g. teraterm. You could also take 2 other serial ports and make them listen on the transmit and receive line of the serial port of your printer to find out what is going on.

QSerial device uses overlapped reads and writes it seems, which makes everything more complex. QSerialDevice handles the complexity for you however, so it shouldn't make a difference. QExtSerialport uses blocking reads and writes. Maybe you could try QExtSerialPort anyway.

I once used a very basic serial class from Thierry Schneider. It is very basic, so not much can go wrong. You find it in attachment. You just have to set dcb.fOutX and dcb.fInX to 1 to make it use Xon/Xoff.

Hope this helps,
Best regards,
Marc

[ilpaso]

You mentioned that your application says that 32KB is sent, and that the printer only receives 7 KB, right ? So your application says "Writed text file" but the printer hasn't received it ?


This is my last test:
I set the Flow control OFF.
I created a signal/slot system with SIGNAL:readyRead() and SLOT:"readTheResponse".
This showed a Xoff char as input and after a while an Xon char.
If I set the XON/XOFF flow control the QSerialDevice hid the XON/XOFF chars in the input data but I don't know how it manages this event.




I once used a very basic serial class from Thierry Schneider. It is very basic, so not much can go wrong. You find it in attachment. You just have to set dcb.fOutX and dcb.fInX to 1 to make it use Xon/Xoff.

Thank you very much. I'll try it if I don't solve with QSerialDevice

Regards
ilpaso

[squidge]

So what is the output from your program, "Writed text file" ?

[ilpaso]

So what is the output from your program, "Writed text file" ?

Sorry there is an error:


if (r == quint64(data.size())) qDebug() << "Writed binary file.";


this means:
"r" is the number of sent bytes
if r is equal to the file size the program prints "Writed binary file."

The error is: the program sends all the bytes but if there is a XOFF the device doesn't read all the bytes.
Now I've to find this XOFF answer, wait the XON signal and restart to write.
I thought the QSerialDevice manage this XON/XOFF but I don't know if it works.:confused:

[kuzulis]

2 ilpaso,

1.
try in the file nativeserialengine_win.cpp in the method:
"bool NativeSerialEnginePrivate:: nativeSetFlowControl (AbstractSerial:: Flow flow)" in the selection
of "case AbstractSerial:: FlowControlXonXoff:" add the following code (example):


...
this->dcb.XonChar = 0x11;
this->dcb.XoffChar = 0x13;
this->dcb.XonLim = 128;
this->dcb.XoffLim = 128;
...

Try to change or XonLim XoffLim (see MSDN for these parameters).
But I'm not sure that this will help. I have not tested mode Xon / Xoff.

2.
Try to open a port in Unbuffered! Maybe this will help.

3. Install it on your computer program such as: port sniffer, such as Free Serial Port Monitor.
This will help to analyze the traffic.
And look what happens, because I do not have your printer and I can not help you. Define the problem, and I'll try to fix it. :)

4. Another idea: in some way (I do not know how) the software simulated your printer. Ie so that this "virtual" printer with a buffer overflow, reported the suspension of transmission.

[ilpaso]

Thank you kuzulis for your help. You are the QSerialDevice developer and you know your class.

I'm working on a linux distro and I think the file to read is nativenerialnngine_unix



I have not tested mode Xon / Xoff.

Ok. This is an important info to know! I'll try your code but an easy solution could be to implement the XON/XOFF manually.




3. Install it on your computer program such as: port sniffer, such as Free Serial Port Monitor.
This will help to analyze the traffic.
And look what happens, because I do not have your printer and I can not help you. Define the problem, and I'll try to fix it. :)

Thankyou very much. I'll install a port sniffer.
Regards
ilpaso

[kuzulis]

I'm working on a linux distro and I think the file to read is nativenerialnngine_unix
Yes.
Then look at the documentation: Serial Programming Guide for POSIX Operating Systems
There is information on xon/xoff.
Look also to parameters: termios.c_cc, maybe you want to add (define) in nativenerialnngine_unix.cpp options: VSTART, VSTOP (ie, check them out and maybe give them a value of 0x31, 011).


Thankyou very much. I'll install a port sniffer.
I do not know sniffers for Linux. Try using strace. But I'm not sure.

[marcvanriet]

Hi,


if r is equal to the file size the program prints "Writed binary file."
The error is: the program sends all the bytes but if there is a XOFF the device doesn't read all the bytes.

Just to make sure I understand you right : you do get the message "Writed binary file.", right ? This may be significant because QSerialDevice doesn't send all data at once, but in chuncks of 512 or so bytes. It might be that the serial port driver returns false on a write() when an XOff has been received. Unlikely though.


I thought the QSerialDevice manage this XON/XOFF but I don't know if it works.

The operating system should take care of this. QSerialDevice doesn't need to do this.

It is very easy to implement the Xon/Xoff handshaking yourself :

// until all data sent :
// send a chunck of data
// if something is received from the printer :
// if Xoff was received :
// wait until Xon received


Best regards,
Marc

[squidge]

What is likely is that if you are sending data that is larger than the serial port buffers. What does QSerialDevice typically set the serial buffers to?

Also, try sending in smaller chunks, say 128 bytes or even 64 or 32 bytes (you can always increase the number if it starts to work) and possibly put in a small delay between each chunk, say 100ms or so.

[kuzulis]

Also, try sending in smaller chunks, say 128 bytes or even 64 or 32 bytes (you can always increase the number if it starts to work) and possibly put in a small delay between each chunk, say 100ms or so.
Yes, you can try to reduce the size of the pieces, as The default library sends data chunks of 512 bytes:


#define WRITE_CHUNKSIZE Q_INT64_C(512)

[ilpaso]

Yes, you can try to reduce the size of the pieces, as The default library sends data chunks of 512 bytes:


#define WRITE_CHUNKSIZE Q_INT64_C(512)


Tomorrow I'll try to implement the Xon/Xoff handshaking myself and find the right buffer.
Thank you

ilpaso

[ilpaso]

Hi all,
I implemented myself the flow control and the transfer works with a buffer size of 256bytes and 512bytes.
I tried to change the "WRITE_CHUNKSIZE" variable to 256bytes and send datas without my flow control. The transfer doesn't work.

[marcvanriet]

Great.

Maybe kuzulis could integrate it in QSerialDevice. Xon/Xoff handshaking in software instead of in the driver of the OS.

Regards,
Marc