PDA

View Full Version : QTcpSocket problem



brucie25
26th July 2016, 12:46
Hi,

When I use a QTcpSocket in a desktop Qt 5 app built and run on osx I seem to have a problem which makes my new app unusable if I can't fix it.

My Qt app creates a socket connection to my server app on an Android device and this all works as it should but then when I want to write data to the socket sometimes (mostly sends instantly) Qt won't send it straight away but the delay is sometimes minutes, sometimes it's been nearly 5 minutes before the server receives the data on the already opened socket.

I have spent a couple of days researching and nowhere do I find this kind of delay to be something we have to put up with if using TCP (I have read about and disabled Nagle algorithm).

When the problem occurs I don't see the data in Wireshark when I send it (triggered from a button press on my Qt app) for the minutes until it arrives on my android device so that's make me sure it's related to my Qt side.

I am sending very small strings when I write to the socket which are less than 20 characters long are always terminated with "\n".

I have tried using waitForBytesWritten() and various socket options such as the ones below with no success in making my QTcpSocket send the data I write to it within a few seconds, can anyone explain why the data would just be sat somewhere for minutes before being sent sometimes?

socket->setSocketOption(QAbstractSocket::SendBufferSizeSoc ketOption,5);
socket->setSocketOption(QAbstractSocket::LowDelayOption,1) ;
socket->setSocketOption(QAbstractSocket::KeepAliveOption,1 );

Thank you.

anda_skoa
26th July 2016, 13:09
This is very uncommon, even a plain TCP socket (without any specially set options) should not do that.

I assume you are not blocking the thread that runs the event loop used by the QTcpSocket?

Does it work if you run the server on the same machine?

Cheers,
_

brucie25
26th July 2016, 14:59
Thanks for replying, I'll do more tests now I have it confirmed it's uncommon- it's difficult because mostly it will send instantly as mentioned but definatley sometimes I press a button on my app which should just write to an opened socket and there is no activity for minutes in wireshark.

I'm not sure about blocking event loop thread - I am of the understanding I don't need to use threads with Qt for such activity so whenever a button is pressed in my app my qml will call c++ function and if the socket is null in my c++ class I create and connect it and send data in my connected slot or if it's not null I just write the data. Can you see a problem doing it like that?

Thanks again

Added after 1 27 minutes:

The problem still exists even when the server is local.

I am not using QThreads in my app - can someone tell me should I be using them - could that be the problem?

There is definitely nothing being sent for a few minutes sometimes from Qt according to Wireshark - I find it hard to believe myself.

Added after 7 minutes:

Some more information that might help somebody to help me - just waited about 3-4 mins for my data to be sent from Qt - I pressed the qml button 4 times in this time and there is no Wireshark activity (there should be 4 writes to server) but then after 4 minutes I see one single packet being sent in Wireshark which contains all strings I am sending seperatley so they are getting bunched up and getting sent sometimes after 4 minutes (random amount).

Please help!

anda_skoa
26th July 2016, 15:42
I'm not sure about blocking event loop thread - I am of the understanding I don't need to use threads with Qt for such activity so whenever a button is pressed in my app my qml will call c++ function and if the socket is null in my c++ class I create and connect it and send data in my connected slot or if it's not null I just write the data. Can you see a problem doing it like that?

This sounds ok.

Blocking the main thread only occurs if you do something like sleep() or a loop that doesn't return for a while or a function call that takes long to execute.



I am not using QThreads in my app - can someone tell me should I be using them - could that be the problem?

Using threads should not be necessary unless you have long running or blocking operations.

Can you try calling the socket's flush() method after writing to it?

Cheers,
_

brucie25
26th July 2016, 17:37
Hi, I do use socket->flush() straight after each write already unfortunately.

Shouldn't SendBufferSizeSocketOption tell Qt to send the data instantly if I set it to a very low value do you know?

Added after 1 7 minutes:

I have now tried using the blocking approach with QThread and waitForBytesWritten but still get the same result.

Could this be related to my laptop being low on resources or something like that?

brucie25
26th July 2016, 20:04
This problem still exits when using UDP instead of TCP.

It never seems to happen when the window is shown - I can send a command one after another many times (press a button) and the data is instantly received but when I switch to some other app on my laptop and then go back to the qt window and press a button this is usually when the long delay occurs until data is received.

Why would there be a delay until my app can start sending data again when coming back into foreground and can this be avoided? I am using osx btw.

anda_skoa
26th July 2016, 21:49
It sounds like there is some buffering inside the operating system that causes this.

Once the application has transferred its data to the OS buffer, it is out of its control.

You could try running this on another operating system to see if that makes a difference.

Cheers,
_

brucie25
26th July 2016, 21:57
Hi,

Thanks.

I have just run an app called PacketSender which might be built using Qt and when I press send in this app to send a udp packet I can see the same behaviour - wireshark will usually show packets being sent instantly but every now and again there is a long delay until the packet passes through wireshark when I send a packet (press send).

I have put so much effort into getting my Qt ui sorted I never thought I wouldn't be able to send data.

anda_skoa
27th July 2016, 00:19
I have also never seen such weird operating system behavior before.

Cheers,
_

jefftee
27th July 2016, 04:58
Are you using IP addresses or DNS host names? Reason I ask is if you have a wonky DNS server, that could be contributing to a delay in the initial connection. I'm guessing that since the Qt Socket calls are asynchronous, could it be that it's your initial connection that is delayed?

How about posting a representative sample of your code that can be used to replicate the problem?

brucie25
27th July 2016, 13:10
Hi, Thanks for replying.

I am hardcoding a local ip address into my code for now "192.168.0.8" so I don't think DNS is the problem.

Below is releveant code. I create a qml document with buttons and I setContextProperty() (my CommandClient class) on this doc and when I press one of the buttons I call a function in my CommandClient class which then emits a signal to my worker class I created earlier which then fires a UDP packet off. Sometimes this won't appear in Wireshark for minutes.

Just to mention again this happens with both TCP and UDP and with and without Threads.



//I create my thread in CommandClient constructor and emit a signal sendCommandSignal when qml calls CommandClient function
QThread* qt = new QThread();
mst = new MySocketThread();
mst->ipAddress = "192.168.0.8";
mst->port = 7755;
mst->moveToThread(qt);
connect(qt, SIGNAL(started()), mst, SLOT(doWork()));
connect(qt, SIGNAL(finished()), mst, SLOT(deleteLater()));
connect(this, SIGNAL(sendCommandSignal()), mst, SLOT(sendCommandSlot()));
qt->start();


void MySocketThread::doWork()
{

socket = new QUdpSocket(this);
socket->setSocketOption(QAbstractSocket::LowDelayOption,1) ;
}

void MySocketThread::sendCommandSlot()
{


qDebug() << "socket is not null "<<toSend;
QString txPacket;//, txDatagram;
txPacket= "";
QByteArray txDatagram;// = "";

txPacket.append(toSend);
txPacket.append(" ");
txDatagram = txPacket.toUtf8(); //convert string to datagram
socket->writeDatagram(txDatagram.data(), txDatagram.size(), QHostAddress("192.168.0.8"), 1212);
socket->flush();// tried with and without flush and waitForBytesWritten()
qDebug() << "bytes written-"<<socket->waitForBytesWritten(30000);
toSend = "";




}

anda_skoa
27th July 2016, 14:02
Just to mention again this happens with both TCP and UDP and with and without Threads.

DNS resolve delay would not have affected an already established TCP connection anyway.

Since the delay also happens after waitForBytesWritten() it looks to be outside of the application's scope, i.e. somewhere in the operating system.

You can try to add the Unbuffered flag to the OpenMode argument or connectToHost() for the TCP connection, but that should have the same effect as waiting for all bytes being written.

Once the data has been written to the underlying socket descriptor, it is "sent" as far as the application can tell:
https://code.woboq.org/qt5/qtbase/src/network/socket/qnativesocketengine_unix.cpp.html#_ZN26QNativeSock etEnginePrivate11nativeWriteEPKcx

Cheers,
_

brucie25
27th July 2016, 15:06
After 2 long days I believe it is either a bug in osx or my laptop has got some kind of virus/spyware which is causing lost packets/network congestion.

I have managed to get hold of a windows laptop and it seems more responsive after running my app on it so it must be one of the two reasons above.

I'll report back in the very unlikely event that running in windows also has the same problem.

Thank you.

brucie25
28th July 2016, 14:35
I went out and got another macbook pro yesterday to test on a newly installed os and this problem still occurs.

I am 99% sure it is something to do with window focus because it will work fine until I go to another app and come back to my app and try to send data again.

I think this thread below is the same problem and therefore I think it must be a bug.

http://stackoverflow.com/questions/31148664/qt-udp-sockets-hang-up-when-my-application-is-minimized-only-on-os-x

Any suggestions?

anda_skoa
28th July 2016, 15:13
Maybe some change in macOS requires that an app somehow "protects" against being paused like that, e.g. some "feature" coming in from iOS.

The implementation of sockets on macOS is the same as on any other Unix as far as I can tell, but maybe it would now be required to be treated differently.

What I find a bit surprising is that no one has posted a link to a Qt issue ticket on stackoverflow, given that the questions is over a year old now.

I searched for a couple of combinations of keyboards related to this and couldn't find any on https://bugreports.qt.io/ either

Cheers,
_

brucie25
28th July 2016, 16:53
Thanks.

Just one bit of information which I think is significant but don't understand what it's telling me yet..

If I press the button - qt sends the data but I don't see the packet in Wireshark. I have noticed though that in activity monitor network section that the packets sent for my app increases each time I press a button so that to me is some kind of acknowledgement from the os that it is getting the data to send.

anda_skoa
28th July 2016, 17:35
It almost looks like the OS is waiting for some OS side buffer to fill up to some level before it actually sends it onto the wire.

Cheers,
_

brucie25
29th July 2016, 23:32
It turns out this was caused by the android server going to sleep and therefore just stopping any more udp communication between client and server. If I keep the android screen awake then this problem goes away. I didn't even consider it for a while because I thought udp was connectionless and therefore couldn't be affected by the server. Thank you for your help.

anda_skoa
30th July 2016, 13:07
See, this is why I suggested, in comment #2, to test both ends running on the same machine :)

Cheers,
_