QSerialPort read delay

[M4chin3]

Hi,

i'm using QSerialPort to read a constant datastream from my comport.
Therefore i'm using a seperate thread that calls itself with a QTimer within a 1ms interval with signal and slots.

Basically like this:

connect(ReceiveTimer, SIGNAL(timeout()), this, SLOT(SlotTimer()));
 
void SlotTimer(void){
 QByteArray ReceivedDat = SerialPort->readAll();
qDebug() << "Received bytes: " << ReceivedDat.size() << " Timestamp: " << QDateTime::currentMSecsSinceEpoch();
}

connect(ReceiveTimer, SIGNAL(timeout()), this, SLOT(SlotTimer()));

void SlotTimer(void){
 QByteArray ReceivedDat = SerialPort->readAll();
qDebug() << "Received bytes: " << ReceivedDat.size() << " Timestamp: " << QDateTime::currentMSecsSinceEpoch();
}

To copy to clipboard, switch view to plain text mode 

So far this works fine for me. The outputs of qDebug() show me that the thread is called every millisecond like it should be.
The problem is that ReceivedDat does not hold data at some calls of the slot depending on the baudrate.

I use two baudrates 38400 and 57600 so therefore within 1ms about 3-4 Bytes and 5-6 Bytes should be stored in the buffer.
When running my program i got new data in an interval of 16ms (38400) and around 800!ms (57600).

Is there anything i'm missing?


Win7 Prof 64-bit
Qt 5.2.0 (MSVC 2010, 32 bit)
Revision 27d10d8dcd

[Lesiok]

Why You don't use readyRead() signal ?

[kuzulis]

@M4chin3,

why you are surprised? It is normal because asynchronous I/O. Besides these intervals depend on load of Qt-event loop of thread (main GUI thread or other thread) where is your QSerialPort instance.

[M4chin3]

@Lesiok

You are right i could and i tried but it does not change the update interval. 800ms is way too much for my application.

@kuzulis

I'm surprised because my QSerialPort instance is in the separte thread with the QTimer. Also the load of the program/whole pc is < 5% and therefore i dont understand delays in the size of ~800ms.
Next point is why should this time increase from 16ms to 800ms only from changing the baudrate from 38400 to 57600?

edit: btw the QThread does nothing more than creating the Timer interval and reading the data from the serialport.

[Lesiok]

Windows is not RT system.

[M4chin3]

Windows is not RT system.

Absolutely!
But i think with 2x 2,6GHz it should be possible to receive data from a comport under 800ms (0,8s).

Maybe it is not quite clear i try it with an example:

void SlotTimer(void){ // called with a 1ms interval
QByteArray ReceivedDat = SerialPort->readAll();
if(0 < ReceivedDat.size()){
qDebug() << "Received bytes: " << ReceivedDat.size() << " Timestamp: " << QDateTime::currentMSecsSinceEpoch();
}else{
qDebug() << "No data." <<  " Timestamp: " << QDateTime::currentMSecsSinceEpoch();
}
}

void SlotTimer(void){ // called with a 1ms interval
QByteArray ReceivedDat = SerialPort->readAll();
if(0 < ReceivedDat.size()){
qDebug() << "Received bytes: " << ReceivedDat.size() << " Timestamp: " << QDateTime::currentMSecsSinceEpoch();
}else{
qDebug() << "No data." <<  " Timestamp: " << QDateTime::currentMSecsSinceEpoch();
}
}

To copy to clipboard, switch view to plain text mode 

Using a baudrate of 38400 results in this output pattern (16ms between every chunk of data):
"Received bytes: 62 Timestamp: 1389632940000"
"No data. Timestamp: 1389632940001"
... ~13 times more
"No data. Timestamp: 13896329400015"
"Received bytes: 61 Timestamp: 1389632940016"
"No data. Timestamp: 1389632940017"
... ~13 times more
"No data. Timestamp: 1389632940031""
"Received bytes: 62 Timestamp: 1389632940032"
... and so on for minutes

Using a baudrate of 57600 results in this output pattern (800ms between every chunk of data):
"Received bytes: 4608 Timestamp: 1389632940000"
"No data. Timestamp: 1389632940001"
... ~800 times more
"No data. Timestamp: 1389632940799"
"Received bytes: 4608 Timestamp: 1389632940800"
"No data. Timestamp: 1389632940801"
... ~800 times more
"No data. Timestamp: 1389632941599"
"Received bytes: 4608 Timestamp: 1389632941600"
... and so on for minutes

So i'm talking not about an sporadic error.

[kuzulis]

Ok, now please:

1. What type of serial device you use?
2. Please re-implement to readyRead() usage and show your timestamps, e.g.:

connect(SerialPort, SIGNAL(readyRead()), this, SLOT(SlotReadyRead()));
 
    void SlotReadyRead() {
        QByteArray ReceivedDat = SerialPort->readAll();
        if(0 < ReceivedDat.size()){
            qDebug() << "Received bytes: " << ReceivedDat.size() << " Timestamp: " << QDateTime::currentMSecsSinceEpoch();
        }else{
            qDebug() << "No data." << " Timestamp: " << QDateTime::currentMSecsSinceEpoch();
        }
    }

connect(SerialPort, SIGNAL(readyRead()), this, SLOT(SlotReadyRead()));

    void SlotReadyRead() {
        QByteArray ReceivedDat = SerialPort->readAll();
        if(0 < ReceivedDat.size()){
            qDebug() << "Received bytes: " << ReceivedDat.size() << " Timestamp: " << QDateTime::currentMSecsSinceEpoch();
        }else{
            qDebug() << "No data." << " Timestamp: " << QDateTime::currentMSecsSinceEpoch();
        }
    }

To copy to clipboard, switch view to plain text mode 

[M4chin3]

@kuzulis

1. I'm using a virtual comport device with a controller from FTDI (FT 232 RL)
2. I reimplemented it with readyRead() usage and it seems to work. I will take a closer look tomorrow and reply in detail. (@Lesiok: sorry i misread your reply, i thought you ment readAll and readData)

Thanks in advance.

[ChrisW67]

I'm surprised because my QSerialPort instance is in the separte thread with the QTimer.

You do not need a separate thread to read from a serial port. Unless the thread has another reason for existing it only complicates what should be a straightforward problem.

[kuzulis]

@M4chin3,

I reimplemented it with readyRead() usage and it seems to work.

Hmm.. It is very interest and strange.. Can you please also, in additional, provide a your project's source code?

@ChrisW67,

if to be honest, in Windows use of a additional thread is desirable, because still there is this bug with a freezing: https://bugreports.qt-project.org/browse/QTBUG-34946

[M4chin3]

@ChrisW67:

I know it would work without a separte thread but i need it for another reason as you already expected. :-)

@kuzulis:

I tested the new code (with readyRead()) in detail today and unfortunately it does not work. Still the same problem with the delay.

What i did yesterday when it "worked":
- I hooked up 2 FTDI (FT 232 RL) together.
- Number1 was sending periodically via Terminal
- Number2 was receiving with my test program.

This setup seemed to work. I received data under 100ms delay. (~30ms or whatever)

What i did today:
- Hooked up one FTDI (FT 232 RL) to my target board with a microcontroller
- microcontroller was sending all the time
- FTDI was receiving with my test program.

This seemed not to work so i investigated a little more and hooked up a scope to the communications. It turned out that the Terminal program i used yesterday was sending the bytestream every 10ms. Between this bytestreams was a silence time without any bytes on the line. In my situation with the microcontroller there is no such silence time. The controller is sending all the time. So i modified my microcontroller program to build in such a silence time and suddently my PC program could read without delays. (but this is no solution for me :-/)

I found out that the communication is working without delays at higher baudrates than 38400 if there is a silence time (time without data been sent) of about 1.7ms or more.
So somehow it seems that either the FTDI chip or Windows does some kind of buffering if the communication speed is above 38400.

I added 3 screenshots:
Baud38400_nodelay.jpg: This shows the communication between µC and PC. The µC is sending almost all the time, silence time (=between the two cursors) is about 125µs. The PC program has no delay while receiving.
Baud38400_nodelay.jpg

Baud57600_nodelay.jpg: Silence time is about 1705µs. The PC program has no delay while receiving.
Baud57600_nodelay.jpg

Baud57600_delay.jpg: Silence time is about 1375µs. The PC program has delay while receiving. (about 800ms)
Baud57600_delay.jpg

I zipped my test project and attachted it to this post.
TestProject.zip

[kuzulis]

@M4chin3,

I'm checked your example with use:

* Win8
* Qt 4.8.3
* QtSerialPort: 2a83fbd6c70032d236bbdf0d87aaf51908cd5afd
* Serial ports: com0com null-modem - http://sourceforge.net/projects/com0com/

and got following results:

Received bytes: 11 Timestamp: 1389963333319
Received bytes: 11 Timestamp: 1389963333320
Received bytes: 12 Timestamp: 1389963333321
Received bytes: 12 Timestamp: 1389963333322
Received bytes: 11 Timestamp: 1389963333323
Received bytes: 12 Timestamp: 1389963333324
Received bytes: 11 Timestamp: 1389963333325
Received bytes: 12 Timestamp: 1389963333326
Received bytes: 34 Timestamp: 1389963333329
Received bytes: 35 Timestamp: 1389963333332
Received bytes: 11 Timestamp: 1389963333333

and etc.

Of course, need to check on real device.. A some later I try to check on on-board serial port (in motherboard) and on USB/Serial PL2303..

Probably such behavior depends on device driver and etc.

So, can you please download a latest sources from Git and try do tests?

[kuzulis]

UPD:

env:

* qt 5.2.0/MinGW
* win 8.1x64

Device: on-board serial port
Speed: 38400
Stream: without delays
Result:

Received bytes: 8 Timestamp: 1390039802077
Received bytes: 8 Timestamp: 1390039802079
Received bytes: 8 Timestamp: 1390039802081
Received bytes: 8 Timestamp: 1390039802084
Received bytes: 8 Timestamp: 1390039802086
Received bytes: 8 Timestamp: 1390039802088
Received bytes: 8 Timestamp: 1390039802090
Received bytes: 8 Timestamp: 1390039802092
Received bytes: 8 Timestamp: 1390039802094

Device: on-board serial port
Speed: 115200
Stream: without delays
Result:

Received bytes: 8 Timestamp: 1390040133628
Received bytes: 8 Timestamp: 1390040133629
Received bytes: 8 Timestamp: 1390040133630
Received bytes: 8 Timestamp: 1390040133630
Received bytes: 8 Timestamp: 1390040133631
Received bytes: 8 Timestamp: 1390040133632
Received bytes: 8 Timestamp: 1390040133632
Received bytes: 8 Timestamp: 1390040133633
Received bytes: 8 Timestamp: 1390040133634
Received bytes: 8 Timestamp: 1390040133634
Received bytes: 8 Timestamp: 1390040133635
Received bytes: 8 Timestamp: 1390040133636
Received bytes: 8 Timestamp: 1390040133637

Device: USB/Serial PL2303 serial port
Speed: 38400
Stream: without delays
Result:

Received bytes: 2 Timestamp: 1390040288136
Received bytes: 2 Timestamp: 1390040288137
Received bytes: 2 Timestamp: 1390040288137
Received bytes: 1 Timestamp: 1390040288138
Received bytes: 7 Timestamp: 1390040288139
Received bytes: 1 Timestamp: 1390040288140
Received bytes: 5 Timestamp: 1390040288141
Received bytes: 2 Timestamp: 1390040288142
Received bytes: 5 Timestamp: 1390040288143
Received bytes: 2 Timestamp: 1390040288143
Received bytes: 5 Timestamp: 1390040288145

Device: USB/Serial PL2303 serial port
Speed: 115200
Stream: without delays
Result:

Received bytes: 6 Timestamp: 1390040455936
Received bytes: 15 Timestamp: 1390040455938
Received bytes: 8 Timestamp: 1390040455939
Received bytes: 23 Timestamp: 1390040455940
Received bytes: 9 Timestamp: 1390040455941
Received bytes: 20 Timestamp: 1390040455943
Received bytes: 7 Timestamp: 1390040455944
Received bytes: 22 Timestamp: 1390040455946
Received bytes: 12 Timestamp: 1390040455946
Received bytes: 17 Timestamp: 1390040455948
Received bytes: 7 Timestamp: 1390040455949
Received bytes: 15 Timestamp: 1390040455950

So, I can not reproduce delays with ~800msec on my side...

[M4chin3]

@ kuzulis:
Thank you. I tried to get com0com installed by the IT department here on this machine. No progress there...

I will post an update as soon as i can but from my point of view now this could take a while.

---

Added after 1 23 minutes:

@ kuzulis:
Since you tested it with a onboard serial port and a PL2303 converted it seemed to me that my FT232RL must be the problem i found the following in a FTDI driver document:

Data is received from USB to the PC by a polling method. The driver will request a certain amount
of data from the USB scheduler. This is done in multiples of 64 bytes. The 'bulk packet size' on
USB is a maximum of 64 bytes. The host controller will read data from the device until either:
a) a packet shorter than 64 bytes is received or
b) the requested data length is reached
The device driver will request packet sizes between 64 Bytes and 4 Kbytes. The size of the packet
will affect the performance and is dependent on the data rate. For very high speed, the largest
packet size is needed. For 'real-time' applications that are transferring audio data at 115200 Baud
for example, the smallest packet possible is desirable, otherwise the device will be holding up 4k of
data at a time. This can give the effect of 'jerky' data transfer if the USB request size is too large
and the data rate too low (relatively).

and

A worst case condition could occur when 62 bytes of data are received in 16 milliseconds. This
would not cause a timeout, but would send the 64 bytes (2 status + 62 user data bytes) back to
USB every 16 milliseconds. When the USBD system driver receives the 64 bytes it would hold on
to them and request another 'IN' transaction. This would be completed another 16 milliseconds
later and so on until USBD gets all of the 4K of data required. The overall time would be (4096 /
64) * 16 milliseconds = 1.024 seconds between data packets being received by the application. In
order to stop the data arriving in 4K packets, it should be requested in smaller amounts. A short
packet (< 64 bytes) will of course cause the data to pass from USBD back to the FTDI driver for
use by the application.

This seems to be my problem. The driver receives more than 62/64 byte within 16 ms (62byte*10bit/byte/16ms = 38750/s; there is the baudrate higher than 38400) and therefore waits for more data until the 4kB Buffer is full.

I know want to test this with a smaller buffer (can be configured via windows device manager) but since i got no administrator privileges, i'm waiting again ^_^.

I will post, as soon as there are news.

[kuzulis]

So, you can try resize the driver's buffer queue through platform-specific Win32 API: SetupComm(),
where HANDLE of device you can take from the QSerialPort::handle() method. Of course, device should be already opened. But I'm not sure that it helps...