Results 1 to 9 of 9

Thread: QFtp::Get, yet one more zero-length files thread

  1. #1
    Join Date
    Aug 2013
    Posts
    32
    Thanks
    2
    Thanked 1 Time in 1 Post
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default QFtp::Get, yet one more zero-length files thread

    Hello.

    I am trying to get a bunch of files from an ftp repo.

    Qt Code:
    1. ///
    2. /// FTP stuff
    3. ///
    4.  
    5. ftp = new QFtp;
    6. ftp->connectToHost("moo.cow");
    7. ftp->setTransferMode(QFtp::Passive);
    8. ftp->login("foo", "bar");
    9. ftp->cd(QString("/"));
    10. lista_archivos_ftp = new QStringList;
    11. ftp->list("/*.jpg");
    12.  
    13. //ftp.cd("img_art");
    14.  
    15. connect(ftp, SIGNAL(dataTransferProgress(qint64, qint64)),
    16. this, SLOT(my_gestor_progreso(qint64, qint64)));
    17.  
    18. connect(ftp, SIGNAL(commandStarted(int)),
    19. this, SLOT(my_empieza_comando_ftp(int)));
    20.  
    21. connect(ftp, SIGNAL(commandFinished(int,bool)),
    22. this, SLOT(my_acaba_comando_ftp(int,bool)));
    23.  
    24. connect(ftp, SIGNAL(done(bool)),
    25. this, SLOT(my_hecho_ftp(bool)));
    26.  
    27. connect(ftp, SIGNAL(stateChanged(int)),
    28. this, SLOT(my_estado_cambiado_ftp(int)));
    29.  
    30. connect(ftp, SIGNAL(listInfo(QUrlInfo)),
    31. this, SLOT(my_list_list_ftp(QUrlInfo)));
    To copy to clipboard, switch view to plain text mode 

    Besides the usual stuff, I have the following relevant methods:

    Qt Code:
    1. void kooker::my_acaba_comando_ftp(int comando, bool error)
    2. {
    3. if(error)
    4. {
    5. // booo
    6. }
    7. else
    8. {
    9. if(ftp->currentCommand() == QFtp::List)
    10. on_btn_descargar_clicked();
    11. }
    12. }
    13.  
    14. void kooker::my_list_list_ftp(QUrlInfo url)
    15. {
    16. my_print_log(QString(Q_FUNC_INFO).append(url.name()));
    17. lista_archivos_ftp->append(url.name());
    18. }
    19.  
    20. void kooker::on_btn_descargar_clicked()
    21. {
    22. qDebug() << Q_FUNC_INFO;
    23. QEventLoop loop;
    24. connect(ftp, SIGNAL(commandFinished(int,bool)),
    25. &loop, SLOT(quit()));
    26.  
    27. QString item;
    28. foreach(item, *lista_archivos_ftp)
    29. {
    30. QString img_local_path = QString(photoDir->path().append("/%1").arg(item));
    31. QFile *file = new QFile(img_local_path);
    32.  
    33. if(!file->open(QIODevice::WriteOnly))
    34. {
    35. delete file;
    36. return;
    37. }
    38.  
    39. QString ftp_file = QString("/%1").arg(item);
    40. ftp->get(ftp_file, file);
    41. loop.exec();
    42.  
    43. file->close();
    44. }
    45. }
    To copy to clipboard, switch view to plain text mode 

    Note that, despite its name, on_btn_descargar_clicked() is called when the ftp list command completes, and not via a button. The funniest part is that, sometimes, I get a single zero bytes jpg file, it seems to hang in QFtp::get forever since the loop doesn't quit(). But some other times, I get all the files, also with zero length.

    So, please, can you tell me if you see anything wrong with the code I wrote? I am having a really difficult time with this one. I've been struggling with this for days. Oh, and before you ask, I have also tried the loop-less (AKA standard) version, like this, with the same results:

    Qt Code:
    1. void kooker::on_btn_descargar_clicked()
    2. {
    3. qDebug() << Q_FUNC_INFO;
    4. // QEventLoop loop;
    5. // connect(ftp, SIGNAL(commandFinished(int,bool)),
    6. // &loop, SLOT(quit()));
    7.  
    8. QString item;
    9. foreach(item, *lista_archivos_ftp)
    10. {
    11. QString img_local_path = QString(photoDir->path().append("/%1").arg(item));
    12. ftp_local_file = new QFile(img_local_path);
    13.  
    14. if(!ftp_local_file->open(QIODevice::WriteOnly))
    15. {
    16. delete ftp_local_file;
    17. return;
    18. }
    19.  
    20. QString ftp_file = QString("/%1").arg(item);
    21. ftp->get(ftp_file, ftp_local_file);
    22. // loop.exec();
    23.  
    24. // file->close();
    25. }
    26. }
    27.  
    28. void kooker::my_acaba_comando_ftp(int comando, bool error)
    29. {
    30. if(error)
    31. {
    32. // booooooo
    33. }
    34. else
    35. {
    36. if(ftp->currentCommand() == QFtp::List)
    37. on_btn_descargar_clicked();
    38.  
    39. if(ftp->currentCommand() == QFtp::Get || ftp->currentCommand() == QFtp::Put)
    40. ftp_local_file->close();
    41. }
    42. }
    To copy to clipboard, switch view to plain text mode 

    In this case, I seem to always get the whole bunch of files, never a single one, but also with zero length.


    Any tip is welcome, I am a bit stuck with this one

  2. #2
    Join Date
    Sep 2009
    Location
    Wroclaw, Poland
    Posts
    1,394
    Thanked 342 Times in 324 Posts
    Qt products
    Qt4 Qt5
    Platforms
    MacOS X Unix/X11 Windows Android

    Default Re: QFtp::Get, yet one more zero-length files thread

    Can you download the files with Qt FTP example application ?

  3. #3
    Join Date
    Aug 2013
    Posts
    32
    Thanks
    2
    Thanked 1 Time in 1 Post
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: QFtp::Get, yet one more zero-length files thread

    I can download them even with my QEventLoop based code in another tool I made the past week. But now it doesn't work in this new program. I have no idea why. Either I am missing something that's really simple or there's some obscure bug in QFtp.

    In the case of the QEventLoop version, the commandFinished is never emitted, why?

    In the case of the regular version, I do what any other example in the net does (included the one you posted, which I have reviewer a dozen times by now): open the file and get it when currentCommand() == QFtp::List, and close() it when currentCommand() == QFtp::Get. Right?

    The slot tied fo transferProgress() is telling me that the data is being downloaded, and a new one I added to the readyRead() signal also shows the data is being downloaded... I really don't know what to think of this.

    Qt Code:
    1. void kooker::my_readyRead()
    2. {
    3. QByteArray array = ftp->readAll();
    4. qDebug() << Q_FUNC_INFO << QString("GOT: '%1'").arg(QString(array));
    5. }
    6.  
    7.  
    8. // produces qDebug() output below....
    9. =========================================
    10.  
    11. void kooker::my_commandStarted(int) "Launched command 19: '8'"
    12. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 0 bytes out of 4029."
    13. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 1448 bytes out of 4029."
    14. void kooker::my_readyRead() "GOT: 'ÿØÿà '"
    15. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 2896 bytes out of 4029."
    16. void kooker::my_readyRead() "GOT: '‚¤r÷{ã?•vv*ÄQÏ‚9GÝqØú{ŠR„fIÄöŠât¶¦“;W
    17. 2I'õ5…áÍ{Ã*ñ}žà …¸‚3þx®…‘dR®¡”õdÃ¥qÃ¥vféÝhd¿ˆ¬c”DDÆBâ=¡:1 cëÍ=µ¸A¨2Î͵U¾PzóŸN=t°·vRÂ¥v›u㜏ӊbé9–ÖH›o˜¢'"
    18. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 2920 bytes out of 4029."
    19. void kooker::my_readyRead() "GOT: ')®2¤S¨ë@ ˜žD=f±Õ͎¢$?w'"
    20. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 4029 bytes out of 4029."
    21. void kooker::my_readyRead() "GOT: '8oÂ¥o^!Gá5›©[_1yUÁô"K©©Ô³õ*bÁÃ*Â¥'"
    22. void kooker::my_commandStarted(int) "Launched command 20: '8'"
    To copy to clipboard, switch view to plain text mode 

    A funny thing is that 1 second after I run this code, +100 empty files are created or updated, and their modification time is set to the current time. But, ftp goes on (at least that's what qDebug() is telling me) for almost one minute, fetching files with are thrown to a QFile that's been close a year ago, hence, ending nowhere. I know what the problem is. I have no idea *where* it lies, or how to fix it by any means.

  4. #4
    Join Date
    Sep 2009
    Location
    Wroclaw, Poland
    Posts
    1,394
    Thanked 342 Times in 324 Posts
    Qt products
    Qt4 Qt5
    Platforms
    MacOS X Unix/X11 Windows Android

    Default Re: QFtp::Get, yet one more zero-length files thread

    Of course you are aware that ftp->get() is asynchronous, right ? So if you have one "global" pointer to QFile and you assign it to each get() command in a loop, without waiting for previous get() to finish, things are gonna go wrong. In your "ftp_command_finished()" slot you call close() only on the last QFile object, so yes, you can see bunch of files with 0 length created, and never really closed.
    Better, at least more clear, approach would be to create a queue of pending downloads, and start next pending job only when previous is done.
    I can download them even with my QEventLoop based code in another tool I made the past week
    So refactor that code and reuse the needed components

  5. #5
    Join Date
    Aug 2013
    Posts
    32
    Thanks
    2
    Thanked 1 Time in 1 Post
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: QFtp::Get, yet one more zero-length files thread

    Yes, I am aware...

    I am also sure I missing something obvious, but can't imagine what. First thing first, the slot tied to commandFinished() is run *each time* a command finishes, right? And not only on the last Get(). Either that or I am completely wrong about the whole QFtp thing, which wouldn't surprise me either

    So, I am closing each file, and I am doing it in the same way that example (and the rest of examples on the net) does (or do). Is there something else I need to do to "wait" for Get() to finish? I have no idea how to do that, if there's something to do at all. No example does that. They simple call get, and wait for ftp->currentCommand() == QFtp::Get in the slot tied to the signal commandFinished(). Is there anything else that needs to be done in the while?

    Secondly, if your assumption was right, the last file would not be empty, which it is.

    Thirdly, I am using a QStringList to "queue" the items. When QFtp::List completes, a slot is called, and that slop is supposed to loop through the list, call Get() on each element with a given file name based on the ftp element name. That file which is global is then closed (supposedly) when QFtp::Get() emits commandFinished(), and so some more qDebug() speech shows:

    Qt Code:
    1. void kooker::my_commandStarted(int) "Launched command 8: '3'"
    2. void kooker::my_stateChanged(int) "FTP STATUS changed: 'QFtp::HostLookup'"
    3. void kooker::my_stateChanged(int) "FTP STATUS changed: 'QFtp::Connecting'"
    4. void kooker::my_stateChanged(int) "FTP STATUS changed: 'QFtp::Connected'"
    5. void kooker::my_commandStarted(int) "Launched command 9: '1'"
    6. void kooker::my_commandStarted(int) "Launched command 10: '4'"
    7. void kooker::my_stateChanged(int) "FTP STATUS changed: 'QFtp::LoggingIn'"
    8. void kooker::my_commandStarted(int) "Launched command 11: '7'"
    9. void kooker::my_commandStarted(int) "Launched command 12: '6'"
    10. void kooker::fetchFtpList() Entering...
    11. void kooker::my_commandStarted(int) "Launched command 13: '8'"
    12. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 0 bytes out of 4266."
    13. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 1448 bytes out of 4266."
    14. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 2896 bytes out of 4266."
    15. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 2920 bytes out of 4266."
    16. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 4266 bytes out of 4266."
    17. void kooker::my_commandFinished(int, bool) "closing file associated with command 13, 0 bytes"
    18. void kooker::my_commandStarted(int) "Launched command 14: '8'"
    19. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 0 bytes out of 2342."
    20. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 1448 bytes out of 2342."
    21. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 2342 bytes out of 2342."
    22. void kooker::my_commandFinished(int, bool) "closing file associated with command 14, 0 bytes"
    23. void kooker::my_commandStarted(int) "Launched command 15: '8'"
    24. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 0 bytes out of 3099."
    25. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 1448 bytes out of 3099."
    26. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 2896 bytes out of 3099."
    27. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 2920 bytes out of 3099."
    28. void kooker::my_dataTransferProgress(qint64, qint64) "Downloaded 3099 bytes out of 3099."
    29. void kooker::my_commandFinished(int, bool) "closing file associated with command 15, 0 bytes"
    30. void kooker::my_commandStarted(int) "Launched command 16: '8'"
    To copy to clipboard, switch view to plain text mode 

    Thanks for your suggestions, I will review this a bit more and see if I can find what's wrong.

  6. #6
    Join Date
    Sep 2009
    Location
    Wroclaw, Poland
    Posts
    1,394
    Thanked 342 Times in 324 Posts
    Qt products
    Qt4 Qt5
    Platforms
    MacOS X Unix/X11 Windows Android

    Default Re: QFtp::Get, yet one more zero-length files thread

    So, I am closing each file
    No, you don't You have a "global" file pointer which you overwrite on each loop iteration, so example of control flow could look like this:

    1.
    [ftp_local_file = 0, loop iteration = 0]
    ftp_local_file = new QFile(); // lets call it file_1
    issue get command with ftp_local_file = file_1
    2.
    [ftp_local_file = file_1, loop i = 1]
    ftp_local_file = new QFile(); // file_2
    issue get command with ftp_local_file = file_2
    3. a file has been downloaded, now:
    ftp_local_file = file_2, you call close() on it (could be fine ONLY IF ftp_local_file is still file_2 ! so if there is more loop iterations, pointer to file_2 is lost and ftp_local_file points to another file)
    4. another file has been downloaded, ftp_local_file is still file_2 (or maybe another value), you call close() on it again
    file_1 may be downloaded, but never will be closed

    memory leaks from all QFile objects file_i, for i= 1, ..., N-1, where N is your download count.
    So it should work ok if you have only 1 download, but i dont know, maybe there are other bugs.

    Create a download queue and start next download in the get_finished() slot, will be far more easier to implement than searching here for bugs.

  7. #7
    Join Date
    Aug 2013
    Posts
    32
    Thanks
    2
    Thanked 1 Time in 1 Post
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: QFtp::Get, yet one more zero-length files thread

    I thank you for your effort, but I've been trying to deal with this for a week almost 24/7 and I am giving up. This really doesn't make any sense, more and more taking into account QFtp is going into oblivion.

    I've wasted one more hour trying to follow your suggestion and obviously QFtp is too complex for me. Crazy, considering I use libcurl without problems all the time. I'll be using libcurl once more since it just works. Obviously some defect in my mental model prevents me from being able to use QFtp.

    A last question. I am too tired to deal with this right now, but, in the case I want to try this in the future I will be using the newer QNetworkAccessManager stuff, which is supposedly the replacement for all this stuff in QT5. Do you know if there's a way to list remote files using that?

    Again, thanks for the dedication and the advice. I am just too tired to deal with this right now.

  8. #8
    Join Date
    Sep 2009
    Location
    Wroclaw, Poland
    Posts
    1,394
    Thanked 342 Times in 324 Posts
    Qt products
    Qt4 Qt5
    Platforms
    MacOS X Unix/X11 Windows Android

    Default Re: QFtp::Get, yet one more zero-length files thread

    I can perfectly understand you
    Maybe starting from scratch could be refreshing, try to define a data structure representing a single download, when looping through a list of all files to download, dont issue get() requests yet, but first put them all into a list. Next, call a doJob() function :
    Qt Code:
    1. // pseudocode..
    2.  
    3. void nextDownload(){
    4. if (!this->_listOfDownloads.isEmpty()){
    5. this->_currentDownload = _listOfDownloads.takeFirst();
    6. this->initializeDownload(_currentDownload); // this should start ftp->get() , connect signals etc
    7. ...
    8. }
    9. }
    10.  
    11. void downloadFinished(){
    12. if (this->_currentDownload) {
    13. _currentDownload->file->close();
    14. delete _currentDownload;
    15. }
    16. this->nextDownload();
    17. }
    18.  
    19. //...
    To copy to clipboard, switch view to plain text mode 
    just an idea of how you can organize this in a different way, sometimes it really helps to start with a fresh perspective.

    Do you know if there's a way to list remote files using that?
    Unfortunately, I don't know. But downloading files from ftp works ok with QNAM I remember searching for a way to issue a "list" command via QNAM in Qt4 ~1 year ago and I couldn't find a solution. I don't know, maybe something changed in Qt5.

  9. #9
    Join Date
    Aug 2013
    Posts
    32
    Thanks
    2
    Thanked 1 Time in 1 Post
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: QFtp::Get, yet one more zero-length files thread

    Thank you, I might try, but not today.

    As for QNAM, well, if it can't even list files, it's useless for me. You know, fetching a file without knowing its name is only slightly less difficult than rewriting gcc using qBasic.

    I have another question though, in case you or anyone else can answer. I've use QEventLoop in another tool to force sync here, and it worked ok. However, this time it doesn't work. The snippet would be something in the lines of:

    Qt Code:
    1. connect(ftp, SIGNAL(commandFinished(int,bool)), &loop, SLOT(quit()));
    2.  
    3. QString item;
    4. QFile *file;
    5. foreach(item, *ftp_file_list)
    6. {
    7. QString img_local_path = QString(photoDir->path().append("/%1").arg(item));
    8. file = new QFile(img_local_path);
    9.  
    10. if(!file->open(QIODevice::WriteOnly))
    11. {
    12. delete file;
    13. return;
    14. }
    15.  
    16. QString ftp_file = QString("/%1").arg(item);
    17. ftp->get(ftp_file, file);
    18.  
    19. loop.exec();
    20. file->close();
    21. }
    To copy to clipboard, switch view to plain text mode 

    But this never goes beyond the first iteration, because it gets stuck in the loop. The qtcreator debugging tools show that I enter the loop, and I can proceed until the loop instruction. After that, nothing else (QFTp-related) happens. It just sits there forever.

    But, anyway...


    Added after 1 30 minutes:


    This seems to work.

    Qt Code:
    1. void kooker::fetchFtpList()
    2. {
    3. qDebug() << Q_FUNC_INFO << "Entering...";
    4. if(!ftp_file_list->isEmpty())
    5. {
    6. QString item = ftp_file_list->takeFirst();
    7. QString img_local_path = QString(photoDir->path().append("/%1").arg(item));
    8. ftp_local_file = new QFile(img_local_path);
    9.  
    10. if(!ftp_local_file->open(QIODevice::WriteOnly))
    11. {
    12. delete ftp_local_file;
    13. return;
    14. }
    15.  
    16. QString ftp_file = QString("/%1").arg(item);
    17. ftp->get(ftp_file, ftp_local_file);
    18. }
    19. //}
    20. }
    21.  
    22. void kooker::my_commandFinished(int command, bool error)
    23. {
    24. if(error)
    25. {
    26. qDebug() << QString("FTP command %1 returned: '%2'")
    27. .arg(command)
    28. .arg(ftp->errorString());
    29. }
    30. else
    31. {
    32. if(ftp->currentCommand() == QFtp::List)
    33. fetchFtpList();
    34.  
    35. if(ftp->currentCommand() == QFtp::Get)
    36. {
    37. qDebug() << Q_FUNC_INFO << QString("closing file associated with command %1, %2 bytes")
    38. .arg(command)
    39. .arg(ftp_local_file->size());
    40. ftp_local_file->close();
    41. fetchFtpList();
    42. }
    43. }
    44. }
    To copy to clipboard, switch view to plain text mode 
    Last edited by i92guboj; 15th November 2013 at 15:20.

Similar Threads

  1. QFtp get multiple files?
    By hexie in forum Qt Programming
    Replies: 0
    Last Post: 12th March 2012, 08:24
  2. QFtp appending of files
    By DIMEDROLL in forum Qt Programming
    Replies: 8
    Last Post: 2nd March 2009, 15:15
  3. QFtp hidden files/folder's list
    By jay in forum Qt Programming
    Replies: 1
    Last Post: 26th December 2008, 12:12
  4. Replies: 0
    Last Post: 23rd September 2007, 11:54

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Qt is a trademark of The Qt Company.