serial.readAll() strange problem reading form FTD232

[linoprit]

Hi,
several years ago I wrote a piece of code that reads data from a serial device. Code and application worked fine until today.
When I start up the application, it gets some data but then stops reading.
I have a "reconnect" button that closes and opens the port again. After reconnecting, still there is no data. But now when you move the mouse, data is read. As long as the mouse is moved, data comes in. Stop mouse moving, stops data reading.
Strange.

My system is Ubuntu 14.04,
QT Version is (QT-Creator says this): Based on Qt 5.5.1 (GCC 4.9.1 20140922 (Red Hat 4.9.1-10), 64 bit)

Here is how I read the data, the function is called by a timer every 20ms:

void serial_service::serial_receive(uint8_t channel)
{
  if (channel < NUM_OF_CHANNELS)
    {      
      QByteArray data = service_data[channel].serial.readAll();
 
      // we throw away data, if data.count() > SERIAL_BUFFER_SIZE
      foreach (const char rec_char, data)
        {
          // use ringbuffer for code compatibility with the microcontroller-application
          ringBufferWrite(&service_data[channel].receive_buffer, (uint8_t) rec_char);          
        }
    }
}

void serial_service::serial_receive(uint8_t channel)
{
  if (channel < NUM_OF_CHANNELS)
    {      
      QByteArray data = service_data[channel].serial.readAll();

      // we throw away data, if data.count() > SERIAL_BUFFER_SIZE
      foreach (const char rec_char, data)
        {
          // use ringbuffer for code compatibility with the microcontroller-application
          ringBufferWrite(&service_data[channel].receive_buffer, (uint8_t) rec_char);          
        }
    }
}

To copy to clipboard, switch view to plain text mode 

Here are my serial settings:

void open_serial_port(uint8_t channel, QString serial_device_name)
{
  service_data[channel].receive_buffer.readIndex  = 0; // todo: put ringbuffer into a class
  service_data[channel].receive_buffer.writeIndex = 0;
 
  if (serial_device_name.isEmpty())
    return;
 
  if (channel < NUM_OF_CHANNELS)
    {
      if (service_data[channel].serial.isOpen())
        service_data[channel].serial.close();
 
      service_data[channel].serial.setPortName(serial_device_name);
      service_data[channel].serial.open(QIODevice::ReadWrite);
 
      service_data[channel].serial.setBaudRate(QSerialPort::Baud9600);    
 
      service_data[channel].serial.setDataErrorPolicy(QSerialPort::SkipPolicy);
 
      service_data[channel].serial.setFlowControl(QSerialPort::NoFlowControl);  
 
      service_data[channel].serial.setDataBits(QSerialPort::Data8);
      service_data[channel].serial.setParity(QSerialPort::NoParity);
 
      service_data[channel].serial.setStopBits(QSerialPort::OneStop);     
 
      //qDebug() << "serial_service::open_port: " << serial_device_name;
    }
}

void open_serial_port(uint8_t channel, QString serial_device_name)
{
  service_data[channel].receive_buffer.readIndex  = 0; // todo: put ringbuffer into a class
  service_data[channel].receive_buffer.writeIndex = 0;

  if (serial_device_name.isEmpty())
    return;

  if (channel < NUM_OF_CHANNELS)
    {
      if (service_data[channel].serial.isOpen())
        service_data[channel].serial.close();

      service_data[channel].serial.setPortName(serial_device_name);
      service_data[channel].serial.open(QIODevice::ReadWrite);

      service_data[channel].serial.setBaudRate(QSerialPort::Baud9600);    

      service_data[channel].serial.setDataErrorPolicy(QSerialPort::SkipPolicy);

      service_data[channel].serial.setFlowControl(QSerialPort::NoFlowControl);  

      service_data[channel].serial.setDataBits(QSerialPort::Data8);
      service_data[channel].serial.setParity(QSerialPort::NoParity);

      service_data[channel].serial.setStopBits(QSerialPort::OneStop);     

      //qDebug() << "serial_service::open_port: " << serial_device_name;
    }
}

To copy to clipboard, switch view to plain text mode 

The problem is really urgent. It's a database application that reads lap-times from a stopwatch (arduino inside). We use it for Kart-Racing and next week we have a race...
Thank you for any help!

by the way: Code and schematics are open source and can be found at www.typsiland.de

[ChrisW67]

The timer will be dependent on the event loop running. Are you blocking it somehwere? Where is your code executing if it is not transiting serial_receive?
Why do you use a timer to call serial_receive() rather than just reacting when a serial port tells you that it has data?

[linoprit]

Hi Chris,
thanks for your answer.
The software worked fine for the last two years. I use it almost every weekend for logging the laptimes in training.
I think that something has changed in the Qt-Libs. My OS is Ubuntu 14.04, there must have been some update during the last four weeks.

You are right, the thing with the timer is a little strange. The reason behind is that I use parts of the code as well in the microcontroller.

What I didn't mention was that one CPU thread was at 100% when the serial read was active. Commenting the readAll line out resulted in normal CPU behavior.

My solution was to read again the good old "Serial programming HOWTO" by Gerry Frerking and Peter Baumann. Then I wrote a new Class that does the serial setup and reading.

I don't know if it is of interest for viewers, but here is the code. There's potential for optimizations, but things are running now:
(by the way I found a short discussion at Stack Overflow, describing the same issue on a Windows machine: http://stackoverflow.com/questions/2...rking-in-5-3-1)

/*
 * Thanks to Gary Frerking (gary@frerking.org) and Peter Bauman
 * for writing the "Serial Programming Howto"
 *
 */
 
 
#include "serial_low_level.h"
 
bool wait_flag = true;
void signal_handler_IO (int status);
 
void signal_handler_IO (int status)
{
    Q_UNUSED (status);
    //printf("received SIGIO signal.\n");
    wait_flag = false;
}
 
 
serial_low_level::serial_low_level()
{
    file_desc = -1;
}
 
 
int serial_low_level::initialize_serial_port(QString devicename)
{
    open_serial_port(devicename);
    if (file_desc < 0)
        return -1;
 
    install_serial_handler();
 
    //qDebug() << "serial_low_level::initialize_serial_port: file_desc = " <<  file_desc;
 
    return file_desc;
}
 
 
void serial_low_level::open_serial_port(QString devicename)
{
    // convert string to const char*
    QByteArray tmp_devname = devicename.toLatin1();
    const char *devname_as_char = tmp_devname.data();
 
    //open the device in non-blocking way (read will return immediately)
    file_desc = open(devname_as_char, O_RDWR | O_NOCTTY | O_NONBLOCK);
}
 
 
void serial_low_level::install_serial_handler(void)
{
    //install the serial handler before making the device asynchronous
    saio.sa_handler = &signal_handler_IO;
    sigemptyset(&saio.sa_mask);   //saio.sa_mask = 0;
    saio.sa_flags = 0;
    saio.sa_restorer = NULL;
    sigaction(SIGIO,&saio,NULL);
 
    // allow the process to receive SIGIO
    fcntl(file_desc, F_SETOWN, getpid());
 
    // make the file descriptor asynchronous
    fcntl(file_desc, F_SETFL, FASYNC);
 
    // set new port settings for canonical input processing
    newtio.c_cflag      = B9600 | CRTSCTS | CS8 | CLOCAL | CREAD;
    newtio.c_iflag      = IGNPAR;
    newtio.c_oflag      = 0;
    newtio.c_lflag      = 0;
    newtio.c_cc[VMIN]   = 1;
    newtio.c_cc[VTIME]  = 0;
    tcflush(file_desc, TCIFLUSH);
    tcsetattr(file_desc,TCSANOW,&newtio);
}
 
 
QByteArray serial_low_level::read_channel(void)
{
    char tmp_buffer[255];
    int  byte_count;
    QByteArray result;
 
    if (wait_flag == true)  //if input is available
        return result;
 
    byte_count = read(file_desc,tmp_buffer,255);
    if (byte_count <= 0)
        return result;
 
    // headnut. For some reason get/setRawData doesn't work
    for (int i=0; i<byte_count; i++)
        result.append(tmp_buffer[i]);
 
    wait_flag = true;      /* wait for new input */
 
    //qDebug() << "serial_low_level::read_channel: byte_count = " << result.count() << " " << byte_count;
 
    return result;
}
 
 
void serial_low_level::write_channel(char* data, int length)
{
    int result = write(file_desc, data, length);
    Q_UNUSED(result);
}
 
 
void serial_low_level::close_serial_port(void)
{
    close(file_desc);
    file_desc = -1;
}
 
 
bool serial_low_level::isOpen(void)
{
    if (file_desc == -1)
        return false;
    else
        return true;
}

/*
 * Thanks to Gary Frerking (gary@frerking.org) and Peter Bauman
 * for writing the "Serial Programming Howto"
 *
 */


#include "serial_low_level.h"

bool wait_flag = true;
void signal_handler_IO (int status);

void signal_handler_IO (int status)
{
    Q_UNUSED (status);
    //printf("received SIGIO signal.\n");
    wait_flag = false;
}


serial_low_level::serial_low_level()
{
    file_desc = -1;
}


int serial_low_level::initialize_serial_port(QString devicename)
{
    open_serial_port(devicename);
    if (file_desc < 0)
        return -1;

    install_serial_handler();

    //qDebug() << "serial_low_level::initialize_serial_port: file_desc = " <<  file_desc;

    return file_desc;
}


void serial_low_level::open_serial_port(QString devicename)
{
    // convert string to const char*
    QByteArray tmp_devname = devicename.toLatin1();
    const char *devname_as_char = tmp_devname.data();

    //open the device in non-blocking way (read will return immediately)
    file_desc = open(devname_as_char, O_RDWR | O_NOCTTY | O_NONBLOCK);
}


void serial_low_level::install_serial_handler(void)
{
    //install the serial handler before making the device asynchronous
    saio.sa_handler = &signal_handler_IO;
    sigemptyset(&saio.sa_mask);   //saio.sa_mask = 0;
    saio.sa_flags = 0;
    saio.sa_restorer = NULL;
    sigaction(SIGIO,&saio,NULL);

    // allow the process to receive SIGIO
    fcntl(file_desc, F_SETOWN, getpid());

    // make the file descriptor asynchronous
    fcntl(file_desc, F_SETFL, FASYNC);

    // set new port settings for canonical input processing
    newtio.c_cflag      = B9600 | CRTSCTS | CS8 | CLOCAL | CREAD;
    newtio.c_iflag      = IGNPAR;
    newtio.c_oflag      = 0;
    newtio.c_lflag      = 0;
    newtio.c_cc[VMIN]   = 1;
    newtio.c_cc[VTIME]  = 0;
    tcflush(file_desc, TCIFLUSH);
    tcsetattr(file_desc,TCSANOW,&newtio);
}


QByteArray serial_low_level::read_channel(void)
{
    char tmp_buffer[255];
    int  byte_count;
    QByteArray result;

    if (wait_flag == true)  //if input is available
        return result;

    byte_count = read(file_desc,tmp_buffer,255);
    if (byte_count <= 0)
        return result;

    // headnut. For some reason get/setRawData doesn't work
    for (int i=0; i<byte_count; i++)
        result.append(tmp_buffer[i]);

    wait_flag = true;      /* wait for new input */

    //qDebug() << "serial_low_level::read_channel: byte_count = " << result.count() << " " << byte_count;

    return result;
}


void serial_low_level::write_channel(char* data, int length)
{
    int result = write(file_desc, data, length);
    Q_UNUSED(result);
}


void serial_low_level::close_serial_port(void)
{
    close(file_desc);
    file_desc = -1;
}


bool serial_low_level::isOpen(void)
{
    if (file_desc == -1)
        return false;
    else
        return true;
}

To copy to clipboard, switch view to plain text mode 

The Header:

#ifndef SERIAL_LOW_LEVEL_H
#define SERIAL_LOW_LEVEL_H
 
#include <QString>
#include <QByteArray>
#include <qdebug.h>
 
#include <termios.h>
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdlib.h>
#include <sys/signal.h>
#include <sys/types.h>
 
 
QStringList get_serial_ports(void);
 
 
class serial_low_level
{
public:
    serial_low_level();
    int initialize_serial_port(QString devicename);
    QByteArray read_channel(void);
    void write_channel(char* data, int length);
    void close_serial_port(void);
    bool isOpen(void);
 
 
private:
    void open_serial_port(QString devicename);
    void install_serial_handler(void);
 
 
    int file_desc;
    struct termios newtio;
    struct sigaction saio;
 
};
 
#endif // SERIAL_LOW_LEVEL_H

#ifndef SERIAL_LOW_LEVEL_H
#define SERIAL_LOW_LEVEL_H

#include <QString>
#include <QByteArray>
#include <qdebug.h>

#include <termios.h>
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdlib.h>
#include <sys/signal.h>
#include <sys/types.h>


QStringList get_serial_ports(void);


class serial_low_level
{
public:
    serial_low_level();
    int initialize_serial_port(QString devicename);
    QByteArray read_channel(void);
    void write_channel(char* data, int length);
    void close_serial_port(void);
    bool isOpen(void);


private:
    void open_serial_port(QString devicename);
    void install_serial_handler(void);


    int file_desc;
    struct termios newtio;
    struct sigaction saio;

};

#endif // SERIAL_LOW_LEVEL_H

To copy to clipboard, switch view to plain text mode 

[kuzulis]

Probably it is Ubuntu's bug. Please look on this: https://bugreports.qt.io/browse/QTBUG-48304

[linoprit]

Thanks.
Yes, the kernel I use is one of the affected.
Just reboot the machine with an older kernel? I'll do testing tomorrow and report the results.

[kuzulis]

> Just reboot the machine with an older kernel?

Yes

[linoprit]

Hi kuzulis,
you are right!
As mentioned above, I rebooted with kernel 3.13.0-62-generic and the serial-readAll works. Back to Kernel 3.13.0-65-generic the problem still exists (surprise, surprise,...).

So all this has nothing to do with qt. It's really a kernel bug.
(Never had such a nasty bug affecting me since the last 12 years...)

Once again - Thank you very much!