Need Qt SerialPort class example using signals and slots WITHIN A THREAD

[RolandHughes]

Answer to this question should be very interesting.

Why? Because there are many devices. You don't burn a thread using blocking waitFor calls when you have many, potentially many dozens, of devices communicating via serial, tcp/ip, digital, and other means. The GUI simply provides a control and reporting interface. Each device needs to be service by non-blocking threads so all devices can be service without data loss. To do this you need signals and slots to occur WITHIN THE THREAD, not the GUI thread. Later, once the entire application runs at low volumes with dynamic OS scheduling, one can add the thread scheduling specific to the CPU vendor so the GUI can run on one CPU and the devices can be evenly allocated across other CPUs. First, you have to get each device to live within a thread. University research centers, labs, and many other organizations have been redeveloping this basic application design for decades. It isn't new. It has been done in Assembly, Fortran, C, and a host of other languages over the years. This particular instance chose to use Qt for a front end so an attempt is being made to use Qt for all of it instead of a hodge-podge.

Beginner:
>>P.S.: this is standard way to use QObject derived objects with signals and slots from other thread.

Thanks for the code, but it has little to do with the issue. I'm very aware of how to do threads and get signals across them. The problem is that signals are not working WITHIN a thread for the Serial stuff...unless that "thread" happens to be the main GUI thread.

Kuzulis:

>>This is not quite true. This class is an addon, but is not part of Qt! And most likely will not be as part of Qt, because it is the development of
custom developers, who do not have the time and resources to prepare and test the ad

Then qtproject should change the wording at the top of this page:
http://qt-project.org/wiki/QtSerialPort

"The QtSerialPort module is an add-on for the Qt5 library, providing a single interface for both hardware and virtual serial ports.
Note: Also added support and for library Qt4."

>>Where cheating? In BlockingMaster/BlockingSlave? O_o
Cheating, not testing, whichever. Because there is no example using signals and slots within threads, it generally means there also is no unit test for the same. Burning a thread with blocking IO may be fine student type work which only has to run for 5 minutes while the instructor looks at it, but it isn't the kind of programming which should be taught, especially for the library which created the signals and slots concept.

>>In addition to the Terminal example, there are other examples of the use of signals / slots without using Threads.
Look here: https://codereview.qt-project.org/#change,41981

Once again, not what I'm looking for. Signals and slots always work well within the GUI thread. Where testing and development have habbitually been an issue is making signals and slots live within a thread which is not the GUI thread. The classes for handling TCP/IP do this splendidly. Other classes from other developers for device interface also work perfectly in their own little thread. Currently, the serial port works well outbound only. The readyRead() signal gets lost. I asked for someone to provide a working example because if there was an actual unit test for this the code would be readily available. Instead, I got a bunch of flak and attempts to cut corners.

wysota:

Given a signal is getting lost, QSignalSpy is the next step when one is on their own debugging a signal problem. Hopefully it will reveal a method of jimmying the connect()/signal map to get around this bug/issue.

kuzulis:
>>Therefore, your conclusions are absolutely baseless and unconstructive.

I assume you were looking in the mirror when you wrote that.


Ok. So, obviously, there is no unit test code out there launching a ReadWrite serial port in its own thread which can be verified by connecting with Putty or some other terminal emulator. In short, I'm on my own tracking this bug down and fixing it.

[Lesiok]

RolandHughes, just use QExtSerialPort. It is working in threads.

[wysota]

Why? Because there are many devices.

So?

You don't burn a thread using blocking waitFor calls when you have many, potentially many dozens, of devices communicating via serial, tcp/ip, digital, and other means.

You don't burn a thread when using signals and slots. Nobody says you should use waitFor*.

Each device needs to be service by non-blocking threads so all devices can be service without data loss.

What data loss? What are you talking about? Non-blocking threads? So if you want to handle say... 20 devices, you'd need 20 "non-blocking threads", thus you'd need at least 21 concurrent threads (assuming there are no other processes running in the system) thus at least 21 cores or processing units in your machine. Otherwise those threads will block because they have to share a limited number of processors.

To do this you need signals and slots to occur WITHIN THE THREAD, not the GUI thread.

No. Especially if, as you said yourself, GUI provides only an interface for controlling and reporting results. Unless you are running a really old machine (like 10 years old), a single thread will happily handle the UI and serial input.

Given a signal is getting lost

Signals can't be "getting lost". They are not socks falling under your washing machine. Qt's signal handling code is deterministic.

[RolandHughes]

RolandHughes, just use QExtSerialPort. It is working in threads.

That was my initial thought because I have used it before successfully, despite some quirks. That was before the requestor saw the wiki page which makes it sound like this new library is supposed to be part of Qt at some point. That, and there are persistent stories that QExtSerialPort isn't under development anymore. I may be forced to go back to that.

Thanks

---

Added after 21 minutes:

wysota,

Please stop replying to this thread. I do not know if it is a language barrier or what, but so far you haven't understood a single thing I've said. I thank you for your time.

You don't burn a thread when using signals and slots. Nobody says you should use waitFor*.

everyone who suggested the examples and unit tests were good "suggested" this since it is the only thing they used when not running in the GUI thread.

What data loss? What are you talking about? Non-blocking threads? So if you want to handle say... 20 devices, you'd need 20 "non-blocking threads", thus you'd need at least 21 concurrent threads (assuming there are no other processes running in the system) thus at least 21 cores or processing units in your machine. Otherwise those threads will block because they have to share a limited number of processors.

No. Aparently you have never done data acquisition. Your assumption is completely invalid. We do have multiple cores, but you do not need one per thread, especially for serial. I've run a 16 serial ports flat out (115200 baud at each port) on a sub Ghz processor without ever losing anything.

No. Especially if, as you said yourself, GUI provides only an interface for controlling and reporting results. Unless you are running a really old machine (like 10 years old), a single thread will happily handle the UI and serial input.

NOT EVEN CLOSE!

Tests left running over night or over the weekend generate millions of data points. Someone kicking off the GUI "report" to plot and graph outliers, or some other such "report" will bind that processor for minutes, if not hours. Don't compare this report to something you've done in school. Think SETI or some other serious research application where data points from a few minutes to a few days can completely fill a 1TB drive when stored in binary format. This is not two people chatting via some cheesey terminal.

Signals can't be "getting lost". They are not socks falling under your washing machine. Qt's signal handling code is deterministic.

Signals are dynamically mapped. Signals can easily get lost any time the entry in the signal map is missing or pointing to a null or pointing to an incorrect thread, hence why I was looking at the signal spy stuff as the next step.

[Lesiok]

QExtSerialPort IS developed. Here is QextSerialPort 1.2 RC

[kuzulis]

@RolandHughes

Mr. b-s-a showed for you how to use QtSerialPort object in non-gui threads, this is the basic concept when
need start Qt-event looping by call exec() method. Without Qt-event loop signals will not work.

Your statement that "lost signals" are totally groundless, you did not provide any simple code to reproduce this "problem".
All that you said - it's just empty words. No one will offer a turnkey solution for you, you must implement it yourself.

QExtSerialPort IS developed. Here is QextSerialPort 1.2 RC

QExtSerialPort does not provide fully asynchronous I/O on Windows, so you can lost productivity.
But in any case, if you consider yourself an expert, you can implement everything in WinAPI / POSIX themselves.


I repeat:
QtSerialPort - an Open Source project, if you do not like - you can fix it yourself or suggest a some solution,
or on end, do not use it, find another project.

[sarcasm]
After all, you're an "expert", but all around - "students"...bla-bla-bla
[/sarcasm]

[RolandHughes]

kuzulis,

Your statement that "lost signals" are totally groundless, you did not provide any simple code to reproduce this "problem".
All that you said - it's just empty words. No one will offer a turnkey solution for you, you must implement it yourself.

Please learn just a tiny bit about a library before inserting your foot so far into your mouth. Signals are lost in most Qt applications and this is be design. When you use an object which emits signals do you assign a slot to catch each and every signal? No. You assign a slot to catch only the signal you are interested in. ALL THE OTHER SIGNALS EMITTED BY THE OBJECT ARE LOST. They have no entry in the dynamically created table matching signals to slots within and across threads. They are emitted and lost.

When new "projects" get added, sometimes they don't do things correctly when creating entries for non-GUI threads. When this happens, the signals emitted which do not have routing entries in the table are LOST. Rather than tossing errors and making developers catch each and every signal from each and every object, the Qt architecture allows developers to catch only those signals and it quietly LOSES the rest.

QExtSerialPort does not provide fully asynchronous I/O on Windows, so you can lost productivity.
But in any case, if you consider yourself an expert, you can implement everything in WinAPI / POSIX themselves.

Please re-read first message. This shop doesn't speak Windows. Development is occuring on Ubuntu Linux.

Students must learn to pay attention when gathering requirements so they don't develop something completely useless.

Thank you for your time.

[wysota]

everyone who suggested the examples and unit tests were good "suggested" this since it is the only thing they used when not running in the GUI thread.

I don't know who is "everyone" and what they "suggested", I'm not telling you to look at examples but to use QIODevice API like it is supposed to be used. You can search the Internet for similar discussions on doing networking with threads. "Everyone" "suggests" this is supposed to be done in threads because that's how they have been doing it using other technologies that didn't provide non-blocking API. It is not supposed to be done in threads.

Aparently you have never done data acquisition. Your assumption is completely invalid. We do have multiple cores, but you do not need one per thread, especially for serial. I've run a 16 serial ports flat out (115200 baud at each port) on a sub Ghz processor without ever losing anything.

Which is exactly my point. You don't need "non-blocking threads" (whatever you mean by that) to handle serial ports simply because the operating system will buffer data for you. If you are running 115200 baud transmission, it doesn't mean you need to read 115200 bits per second. It means you need to read 115200 per second on average. This is only 14kB per port, I'm sure single thread can handle reading 16 of these per second and still be able to do some fancy stuff in your UI.

Someone kicking off the GUI "report" to plot and graph outliers, or some other such "report" will bind that processor for minutes, if not hours.

Did it come to your mind to do this "minutes if not hours" lasting processing in a thread (or even in a separate process)? Are you really blocking your UI for "minutes if not hours" and still think that's ok? Are you sure it is me who never did any data aquisition and processing?

Don't compare this report to something you've done in school

We never did that in school because by the time I was finishing school we were running a single 56k modem shared by around 20 machines.

Think SETI or some other serious research application where data points from a few minutes to a few days can completely fill a 1TB drive when stored in binary format.

Oh jeez, and you want to process it in the GUI thread? That's so professional...

Signals are dynamically mapped. Signals can easily get lost any time the entry in the signal map is missing or pointing to a null or pointing to an incorrect thread, hence why I was looking at the signal spy stuff as the next step.

You have to be kidding. If you are worried about it and are unable to check if that is really the case then do proper synchronization in your code.

Stop trolling and claiming to have all the answers and start reading on proper usage of components at hand. Your post on disabled copy constructor in a QObject subclass clearly shows how well you understand Qt code.

---

Added after 6 minutes:

Please learn just a tiny bit about a library before inserting your foot so far into your mouth.

I think Kuzulis knows best of us how the QtSerialPort library works.

Signals are lost in most Qt applications and this is be design. When you use an object which emits signals do you assign a slot to catch each and every signal? No. You assign a slot to catch only the signal you are interested in. ALL THE OTHER SIGNALS EMITTED BY THE OBJECT ARE LOST.

So that's what you mean by a "lost" signal... I would rather call that "ignored".

They have no entry in the dynamically created table matching signals to slots within and across threads. They are emitted and lost.

Some of them might even not be emitted at all

When new "projects" get added, sometimes they don't do things correctly when creating entries for non-GUI threads. When this happens, the signals emitted which do not have routing entries in the table are LOST.

Are you talking about your own projects or projects in general?

[prof.ebral]

I'm still not 100% certain what you are looking for. From the initial post you want the QtSerialPort to process data while in a thread, and then the main program to handle that data while in a thread? Is that correct? If that is what you are looking for you might consider an MVC model where you model is your QtSerialPort classes and QThread classes, your View handles UI updates, and your Controller handles the signal connections. Your controller would accept emitted signals and start new threads to handle data processing.

As far as I understand data sent by signals is sent to the main thread. You are asking for this, "Looking actual compiling example of SerialPort class using signals and slots within a thread which is NOT the GUI thread."

[wysota]

As far as I understand data sent by signals is sent to the main thread.

There are two things here. "Data" (as in arguments to signals) are not sent to any thread, in the meaning that thread affinity of them does not change in any way. If you pass a pointer to a QObject that lives in thread A to thread B, then accessing that object there might cause data corruption or worse. If you pass some non-QObject by pointer then it doesn't have any thread affinity and it is its thread-safety that decides whether this is safe or not. if you pass a non-QObject by value then it is copied and in regular cases will have no notion of its other clone being accessed from the original thread. However another topic is if we interpret what you said as "signals are sent to the main thread" in which case this is not true -- signals are sent in the thread they are emitted in (it's just a plain function call) but connecting to them may cause an event to be posted to an object living in a different thread (it doesn't have to be the "main" thread though).

[prof.ebral]

However another topic is if we interpret what you said as "signals are sent to the main thread" in which case this is not true -- signals are sent in the thread they are emitted in (it's just a plain function call) but connecting to them may cause an event to be posted to an object living in a different thread (it doesn't have to be the "main" thread though).

Hmmm, I thought the main thread received the signal as a handler and then executed the method defined by the slot.

[wysota]

Hmmm, I thought the main thread received the signal as a handler and then executed the method defined by the slot.

No, it depends on the thread affinity of the object that the slot belongs to. Here is a simple example:

#include <QtCore>
 
class Object : public QObject {
  Q_OBJECT
public:
  Object(QObject *parent = 0) : QObject(parent) {
 
  }
public slots:
  void reportThread() {
    qDebug() << Q_FUNC_INFO << QThread::currentThread();
  }
};
 
#include "main.moc"
 
int main(int argc, char **argv) {
  QCoreApplication app(argc, argv);
 
  QVector<QThread*> threads;
  QVector<Object*> objects;
  for(int i=0;i<4;++i) {
    QThread *thr = new QThread;
    threads << thr;
 
    Object *o = new Object;
    o->moveToThread(thr);
    objects << o;
 
    thr->start();
 
  }
  // last object in main thread
  objects << new Object;
  qDebug() << "GUI thread:" << objects.last()->thread();  
  QTimer t;
  t.start(2000);
  foreach(Object *o, objects)
    QObject::connect(&t, SIGNAL(timeout()), o, SLOT(reportThread()));  
  return app.exec();
}

#include <QtCore>

class Object : public QObject {
  Q_OBJECT
public:
  Object(QObject *parent = 0) : QObject(parent) {
  
  }
public slots:
  void reportThread() {
    qDebug() << Q_FUNC_INFO << QThread::currentThread();
  }
};

#include "main.moc"

int main(int argc, char **argv) {
  QCoreApplication app(argc, argv);
  
  QVector<QThread*> threads;
  QVector<Object*> objects;
  for(int i=0;i<4;++i) {
    QThread *thr = new QThread;
    threads << thr;
    
    Object *o = new Object;
    o->moveToThread(thr);
    objects << o;
    
    thr->start();
    
  }
  // last object in main thread
  objects << new Object;
  qDebug() << "GUI thread:" << objects.last()->thread();  
  QTimer t;
  t.start(2000);
  foreach(Object *o, objects)
    QObject::connect(&t, SIGNAL(timeout()), o, SLOT(reportThread()));  
  return app.exec();
}

To copy to clipboard, switch view to plain text mode 

You can also move the timer to one of the threads and see if it changes anything.

Oh, one more thing -- you can run this example under a debugger, set a breakpoint in the slot and look at the backtrace. You'll see how it differs for the last added object (the one that lives in the main thread) compared to other objects.

[prof.ebral]

Ok, thanks wysota. I use PyQt and when I look at the current thread inside a method that has been called by a signal it is the main thread. I've yet to see an instance when it is isn't in PyQt.

[RolandHughes]

I don't know who is "everyone" and what they "suggested", I'm not telling you to look at examples but to use QIODevice API like it is supposed to be used. You can search the Internet for similar discussions on doing networking with threads. "Everyone" "suggests" this is supposed to be done in threads because that's how they have been doing it using other technologies that didn't provide non-blocking API. It is not supposed to be done in threads.

Yes it is. Please do not try to view this application in terms of something running on a machine where the user can also surf the Web and check email.

Which is exactly my point. You don't need "non-blocking threads" (whatever you mean by that) to handle serial ports simply because the operating system will buffer data for you. If you are running 115200 baud transmission, it doesn't mean you need to read 115200 bits per second. It means you need to read 115200 per second on average. This is only 14kB per port, I'm sure single thread can handle reading 16 of these per second and still be able to do some fancy stuff in your UI.

That is very poor design and not allowed in a scientific environment were loss of a single data point scraps the entire test run.

Did it come to your mind to do this "minutes if not hours" lasting processing in a thread (or even in a separate process)? Are you really blocking your UI for "minutes if not hours" and still think that's ok? Are you sure it is me who never did any data aquisition and processing?

Oh jeez, and you want to process it in the GUI thread? That's so professional...

It is a design requirement. Do not confuse this with some application on a machine which has any other purpose or ability in life. The really large data sets can and do get off-loaded to serious computers for detailed analysis. The initial analysis to weed out non-interesting data sets occurs on the collection machine. What you consider a waste saves weeks and in some cases months of serious computing time.

You have to be kidding. If you are worried about it and are unable to check if that is really the case then do proper synchronization in your code.

Stop trolling and claiming to have all the answers and start reading on proper usage of components at hand. Your post on disabled copy constructor in a QObject subclass clearly shows how well you understand Qt code.

I'm not trolling or claiming to have all of the answers. I came in asking if there was unit test code proving this works. I got a bunch of responses which sounded like a Bing commercial.

My next debugging course was to use the signal tool, but any class which does not have a copy constructor cannot be used with it...a point you have repeatedly failed to recognize.

Please don't view this as a user application. This is a single purpose system running in an environment were no data point can be lost. Every device must be service by a non-blocking thread and those threads must be distributed across processors. It is not an end user application nor a hokey little smart phone app. It gathers all data points and the GUI only has two purposes. 1) Configure everything for a test run 2) run the reports/graphs used to identify interesting data sets. That's it.

Some of them might even not be emitted at all

Hence the wish to test with the signal tool.

Are you talking about your own projects or projects in general?

Projects which get released for use with Qt. I find they only provide unit test code/examples for GUI use, and nothing for use within non-GUI threads.

At any rate. This message thread can now end. I'm on my own fixing it or I have to switch to qextserial.

---

Added after 7 minutes:

As far as I understand data sent by signals is sent to the main thread. You are asking for this, "Looking actual compiling example of SerialPort class using signals and slots within a thread which is NOT the GUI thread."

http://doc.qt.digia.com/qt/threads-q...across-threads

Queued Connection The slot is invoked when control returns to the event loop of the receiver's thread. The slot is executed in the receiver's thread.