Results 1 to 7 of 7

Thread: Multiple QNetworkRequests (post) with QNetworkNetworkAccessManager in a loop

  1. #1
    Join Date
    Mar 2015
    Posts
    4
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Multiple QNetworkRequests (post) with QNetworkNetworkAccessManager in a loop

    Hello all,

    This is my first post on here. QtCentre and forum are a great source of information.

    I have read through the forum and I found somewhat related posts, but not quite on the money, so I thought I'd give this a go, hoping I haven't missed any existing information on the subject.
    As I state in the description, I’m having a problem with sending multiple network requests via the QnetworkAccessManager(QNAM) post method. The windows application runs as a QApplication.

    The entire application consists of a client and a server, communicating through Http requests made by the client. The server only responds to requests and doesn't take any 'initiative'.

    Qt Code:
    1. Sync::Sync(ConnectionInfo info) //info contains ip address and user auth
    2. {
    3. p = new SyncPrivate;
    4. p->mManager = new QNetworkAccessManager(this);
    5. connect(p->mManager, SIGNAL(authenticationRequired(QNetworkReply*,QAuthenticator*)),
    6. SLOT(authenticate(QNetworkReply*, QAuthenticator*)));
    7. p->mIpAddress = info.remoteIP();
    8. switch(info.userLevel()) {
    9. //UserAuth info here
    10. }
    11. p->mPassword = info.password();
    12. }
    13.  
    14. void ControllerListEntry::on_actionStop_Monitoring_All_Nodes_triggered()
    15. {
    16. if (mSync != 0)
    17. return;
    18.  
    19. mSync = new Sync(mController->connectionInfo());
    20. connect(mSync, SIGNAL(finished()), this, SLOT(stopMonitoringAllNodesFinished()));
    21. foreach(Node *mNode, mController->getNodes()){ //Looping through all nodes
    22. if(!mSync->setMonitoring(mNode, false))
    23. break;
    24. }
    25. }
    26.  
    27.  
    28. void ControllerListEntry::on_actionStart_Monitoring_All_Nodes_triggered()
    29. {
    30. if (mSync != 0)
    31. return;
    32.  
    33. mSync = new Sync(mController->connectionInfo());
    34. connect(mSync, SIGNAL(finished()), this, SLOT(startMonitoringAllNodesFinished()));
    35. foreach(Node *mNode, mController->getNodes()){ //Looping through all nodes
    36. if(!mSync->setMonitoring(mNode, true))
    37. break;
    38. }
    39. }
    40.  
    41.  
    42. //For each iteration of the previous loop, the following method is called:
    43. bool Sync::setMonitoring(Node n, bool enabled)
    44. {
    45. QNetworkRequest req(QUrl("http://" + p->mIpAddress + "/rpc/set-monitoring"));
    46. req.setAttribute(QNetworkRequest::HttpPipeliningAllowedAttribute, QVariant(true));
    47. QByteArray postReq = "nodename=" + encode(n->getName());
    48. postReq += "&monitoring=" + QByteArray(enabled ? "true" : "false");
    49. p->mBusy = true;
    50. p->mReply = p->mManager->post(req, postReq);
    51.  
    52. connect(p->mReply, SIGNAL(finished()), SLOT(replyFinished()));
    53.  
    54. return true;
    55. }
    56.  
    57. //For completeness sake
    58. void Sync::replyFinished()
    59. {
    60. p->mReply->deleteLater();
    61. p->mBusy = false;
    62.  
    63. // Check for errors.
    64. if (p->mReply->error() != QNetworkReply::NoError) {
    65. emit error(p->mReply->errorString()); //will call slot showing error message
    66. return;
    67. }
    68.  
    69. emit finished(); //ControllerListEntry will do some ui stuff
    70. }
    To copy to clipboard, switch view to plain text mode 

    The problem is that requests are sent for only part of the Nodes in the Controller when above a certain number of Nodes. This certain number is somewhere around 8, this sound a bit vague and it is, but that's the nature of this problem. Sometimes 6 requests are sent, sometimes up to 11. Always more than 6 though.

    I'm aware of the following line from the Qt Documentation for QNetworkAccessManager:
    Note: QNetworkAccessManagerqueues the requests it receives. The number of requests executed in parallel is dependent on the protocol. Currently, for the HTTP protocol on desktop platforms, 6 requests are executed in parallel for one host/port combination.

    That makes me expect that all the requests will be sent in blocks of 6, until no requests are left. Am I wrong in this?
    Allowing HTTP Pipelining or disabling it doesn't seem to make any difference in the result.

    Sometimes I get the following in the application output: QCoreApplication::postEvent: Unexpected null receiver. Would this happen when not receiving a network reply?

    I can confirm:
    • The setMonitoring code finished every time for every Node.
    • Only for part of the (1 - very small amount) Nodes a reply is caught by the replyFinished Slot
    â—‹ Using Wireshark:
    § Http Replies are actually seen on the network
    § Not always an Http reply for every request
    • It is always the requests for the last n Nodes that don't get sent. No "gaps" in between.
    • Qt version: Qt 4.7.4.

    For 18 Nodes this is what happens on the network on Http level between client and server.
    WiresharkSetMon.jpg

    I hope someone can provide some ideas on the matter.

    Many thanks in advance,


    Tim

  2. #2
    Join Date
    Mar 2015
    Posts
    4
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Multiple QNetworkRequests (post) with QNetworkNetworkAccessManager in a loop

    Hi all,

    I see I could have clarified the structure of the method calls.

    The following happens:
    * The user clicks a button in the UI, which connects to the on_actionStart_Monitoring_All_Nodes_triggered() and it's "Stop" counterpart.
    * Either of these methods loops through all the Node elements in the system and calls the Sync::setMonitoring() method for each of these.
    * In the setMonitoring() method, the request is built and posted to the server.

    For clarification on how the QNetworkAccessManager is initiated I added the constructor for Sync. I also added the slot that the finished() signal from the QNetworkReply connects to, for completeness sake.

    I hope this makes it a bit easier to read.

    It's sometimes hard to write up these parts out of their context. For me at least .

    Looking forward to your thoughts.

    Tim
    Last edited by TimWoo; 20th October 2015 at 09:00. Reason: updated contents

  3. #3
    Join Date
    Oct 2009
    Posts
    483
    Thanked 97 Times in 94 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Multiple QNetworkRequests (post) with QNetworkNetworkAccessManager in a loop

    I do not understand all the details of your code, but something caught my attention. The line
    Qt Code:
    1. p->mReply = p->mManager->post(req, postReq);
    To copy to clipboard, switch view to plain text mode 
    overwrites the SyncPrivate's mReply member every time you send a request. Since all the requests made in the loop seem to go through the same Sync instance, I suppose that after the loop, mReply points to the last reply. Then, in Sync::replyFinished(), you always deal with that last reply, even if the slot is probably called for several of them. Here is a suggestion: instead of connecting to the QNetworkReply's finished() signal, consider connecting to QNetworkAccessManager::finished(QNetworkReply *) instead; that way, you get a pointer to the reply in the slot, right where you need it.

  4. #4
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,729
    Thanks
    13
    Thanked 1,610 Times in 1,537 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: Multiple QNetworkRequests (post) with QNetworkNetworkAccessManager in a loop

    The problem is that requests are sent for only part of the Nodes in the Controller when above a certain number of Nodes. This certain number is somewhere around 8, this sound a bit vague and it is, but that's the nature of this problem. Sometimes 6 requests are sent, sometimes up to 11. Always more than 6 though.
    As part of being a well-behaved netizen QNetworkAccessManager limits the number of simultaneous connections to a single server address. I am fairly sure this limit is 6 connections in the Qt5 source. Are all your connections to the same host?

  5. #5
    Join Date
    Mar 2015
    Posts
    4
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Multiple QNetworkRequests (post) with QNetworkNetworkAccessManager in a loop

    Quote Originally Posted by ChrisW67 View Post
    As part of being a well-behaved netizen QNetworkAccessManager limits the number of simultaneous connections to a single server address. I am fairly sure this limit is 6 connections in the Qt5 source. Are all your connections to the same host?
    Thanks for responding.

    Only 1 host is concerned in this case. Is a new connection really created for every request in such a "batch" of 6? I don't think it explains why sometimes more than 6 requests (up to 11 observed so far) can be monitored on the network (and always the first n, no gaps in the sequence), do you have an explanation for this?

    Please note that my code is written in Qt 4.7. Thought I'd mention it since you talk about Qt5.

    Quote Originally Posted by yeye_olive View Post
    I do not understand all the details of your code, but something caught my attention. The line
    Qt Code:
    1. p->mReply = p->mManager->post(req, postReq);
    To copy to clipboard, switch view to plain text mode 
    overwrites the SyncPrivate's mReply member every time you send a request. Since all the requests made in the loop seem to go through the same Sync instance, I suppose that after the loop, mReply points to the last reply. Then, in Sync::replyFinished(), you always deal with that last reply, even if the slot is probably called for several of them. Here is a suggestion: instead of connecting to the QNetworkReply's finished() signal, consider connecting to QNetworkAccessManager::finished(QNetworkReply *) instead; that way, you get a pointer to the reply in the slot, right where you need it.
    Good point I guess. That would explain why the application only reports replies for some of the requests. Do you think this would influence the behaviour of sending the requests?

  6. #6
    Join Date
    Oct 2009
    Posts
    483
    Thanked 97 Times in 94 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Multiple QNetworkRequests (post) with QNetworkNetworkAccessManager in a loop

    Quote Originally Posted by TimWoo View Post
    Good point I guess. That would explain why the application only reports replies for some of the requests. Do you think this would influence the behaviour of sending the requests?
    I do not know the implementation details of QNetworkAccessManager, but it is entirely possible that the manager stops sending requests while too many QNetworkReply are around, waiting for the user to read their contents and then delete them.

  7. #7
    Join Date
    Mar 2015
    Posts
    4
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Multiple QNetworkRequests (post) with QNetworkNetworkAccessManager in a loop

    @ChrisW67
    Do you have any further insights?

    @yeye_olive
    I'm not familiar with the QNetworkAccessManager implementation myself, maybe someone else on the forum is? If so, I would very much welcome the input .

    Further:
    I cannot find any information about the maximum number of requests offered to QNetworkAccessManager simultaneously in the Qt 4.8 Documentation or anywhere else for that matter. I'm under the impression that it shouldn't be a problem for QNetworkAccessManager, but I could very well be wrong offcourse. It's all quite complex to me, as I'm still not quite familiar with asynchronous message handling.
    I'm currently tempted to prototype a workaround to see if offering 6 requests to the QNAM at the same time solves the problem. I would then wait sending the next 6 requests once the previous 6 have been processed.

    If anyone can shed some more light on the subject I would be very grateful.

    Best Regards,

    Tim

Similar Threads

  1. How to use for loop to check multiple number of objects
    By gunturrohith in forum Qt for Embedded and Mobile
    Replies: 5
    Last Post: 9th September 2015, 09:01
  2. add multiple QCoombox using loop
    By uz_qt in forum Qt Programming
    Replies: 8
    Last Post: 27th September 2013, 11:24
  3. Replies: 1
    Last Post: 18th April 2012, 07:47
  4. Replies: 4
    Last Post: 6th August 2011, 01:40
  5. Replies: 6
    Last Post: 25th May 2010, 10:42

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.