How to check the file to download from ftp site exists ?

[wysota]

I checked FTP is working fine.

What do you consider "fine" when speaking about a protocol? If you know how ftp dictates the servers to manage the connection state, why are you asking those questions? The protocol clearly states how to abort a running command and what the state of the connection is afterwards.

[davethomaspilot]

Read this thread and didn't see anything useful. The problem seems non-trivial, yet the responses suggest it's only difficult if you don't understand ftp. I disagree!

My situation is similar, except I want to do an ftp of a file and create a directory structure on the remote host based on a part of the local file's path. Which means I need to ftp->mkdir to create directories as needed before doing the ftp->put().

For example, I have a file named /la/lb/d/e/f/file.jpg. I want it to end up on the ftp server in something like /ftptop/d/e/f/file.jpg

So, there needs to be a sequence of ftp->mkdir, ftp->cd commands to create the directory structure on the remote host before doing the final ftp->put.

Something like this doesn't work:

QStringList dirs2Make = ftpDir.split("/");
      qDebug()<< dirs2Make;
      foreach (QString d,dirs2Make)
      {
          qDebug() << "making directory " << d;
         ftp->mkdir(d);
         ftp->cd(d);
      }
      ftp->put ftpFile;

QStringList dirs2Make = ftpDir.split("/");
      qDebug()<< dirs2Make;
      foreach (QString d,dirs2Make)
      {
          qDebug() << "making directory " << d;
         ftp->mkdir(d);
         ftp->cd(d);
      }
      ftp->put ftpFile;

To copy to clipboard, switch view to plain text mode 

Because if directories already exist in the remote path, the sequence will terminate. So, if a directory exists, cd into it. If not mkdir first, then cd into. Sounds simple, but it's not that easy.

The QFtp commands are asynchronous. Which means you have to schedule a sequence and wait for it to complete or for a command in the sequence to fail. Then create another sequence based on if and how the prior scheduled sequence completed.

Figuring out which command in a sequence failed requires keeping an association of a sequence number to the each of the commands in the sequence.

So, unless I'm missing something, there's more too it than just understanding how ftp works. I'm thinking it would be easier to just shell out to a script that does the ftp commands--that way the ftp commands can more easily be dynamically created based on the pass/fail result of a prior command.

Basically, what I'd need is a way to do everything but the actual put synchronously. Commands like login, cd, mkdir typically complete quickly and often alter what should happen next based on whether they pass or fail.

[wysota]

So don't schedule a sequence of commands but rather schedule the next one only after the previous one completes. What exactly is the problem?

[davethomaspilot]

The problem is the relative complexity of the code.

What would you suggest the code that must be in the commandFinished slot? There's no userdata * that can be passed to the slot. All you get is a sequence number and an error code. So, I guess the slot has to have code to code that depends on external class variables to figure out what to do on fail.

Just an unnecessarily complex implementation, IMHO. Versus a simple return code from a synchronous command. Even a simple flag to NOT abort the sequence on command failure would simplify things for me signficiantly. Then, the equivalent of mkdir -p in ftp would just become ftp->mkdir mydir; ftp->cd mydir.

I really don't care if the mkdir fails if it's because it already exists. And if the ftp->mdkir fails for some other unknown reason, I don't how I'd handle the error versus handling the error in the subsequent ftp->cd.

Thanks for the reply,

Dave Thomas

---

Added after 4 minutes:

Maybe I don't have the best practice for making an asynchronous command synchronous.

Say my application can do nothing useful except keep the GUI responsive until a command complets (or set of commands in this specific example). Should I enter a spin loop with a QApplication:rocessEvents looking for a flag that gets changed in a slot when a particular signal gets emitted? What controls how much cpu resource gets spent spinning in the event loop versus getting the commands done (when they are on the same thread).

[anda_skoa]

First, a common pattern to wrap asychronous operations together with their context is the job or command pattern.
Each command object has its context, can be started and signals success/failure.
A sequence command can then get a list of smaller commands and start one after the previous one has succeeded.
Things like "check if dir exists, create if not" can also be combined in a composite command which can then be used in the sequence.

See also this blog http://cmollekopf.wordpress.com/2014...re-it-belongs/

Second, any event loop based asynchronous operation in Qt can, with some caution, behave synchronous by using a local event loop instance, e.g. using a helper function

void runCommand(QFtp *ftp, int commandId)
{
    QEventLoop loop;
    connect(ftp, SIGNAL(commandFinished(int, bool)), &loop, SLOT(quit()));
    loop.exec();
}

void runCommand(QFtp *ftp, int commandId)
{
    QEventLoop loop;
    connect(ftp, SIGNAL(commandFinished(int, bool)), &loop, SLOT(quit()));
    loop.exec();
}

To copy to clipboard, switch view to plain text mode 

runCommand(ftp, ftp->mkdir(d));

runCommand(ftp, ftp->mkdir(d));

To copy to clipboard, switch view to plain text mode 

Alternatively use something like a QProgressDialog to provide both the local event loop and keeping the user from interacting with the rest of the application.

Cheers,
_

[davethomaspilot]

From the URL you provided (and thanks for that):

What you’ll notice immediately that this involves a lot of boilerplate code. It also introduces a lot of complexity in a seemingly trivial task.....So while KJob gives us a tool to wrap asynchronous operations in a way that they become reusable, it comes at the cost of quite a bit of boilerplate code. It also means that what can be written synchronously in a simple function, requires a class when writing the same code asynchronously

That's really my point--complexity versus a simple synchronous call. So, that's why I'm thinking use of the QFtp class for my purposes isn't a good idea versus doing it externally in a script where synchronous ftp commands ARE available.

The idiom you posted for making a synchronous command is better than what I'm doing--at least the spin loop isn't visible in my code ;-). Thanks!

But, I'm assuming the overhead to loop and process events in the exec is small enough not to significantly impact execution of the commands themselves? That's the objection I have about using an asynchronous command when a synchronous operation is desired.

And, there's considerably more to the implementation than understanding ftp commands.

Thanks,

Dave Thomas

[wysota]

You don't need KJob and you don't need any boilerplate code. KJob aims to be a general purpose mechanism, your case is very specific so you only need code tailored to your exact situation. You can have a queue of commands to be executed and each command can be in form of if-then, e.g. if directory X doesn't exist, create directory X. Then you schedule a command from the tip of the queue to ftp and when the command is finished, you check the condition, pop the command off the queue and repeat the whole process with the new tip. You only need the data structure for the queue and a slot connected to a signal emitted when ftp command finishes execution.

[davethomaspilot]

this thread has had LOTS of reads--I think it is a very common problem.

situation. You can have a queue of commands to be executed and each command can be in form of if-then, e.g. if directory X doesn't exist, create directory X.

Well, yes at at high enough abstraction level.

I know this is obvious to you but there's a bit of code behind the "if-then stuff". Declare signals and slots. Connect signals and slots , Schedule the command. Go into an event loop. Execute code in slot that gets called when command finishes. Set something in the slot that let's the code that scheduled the ftp command know if it's passed or failed.

Versus checking a return code of a syncrhonous command in an your favorite scripting language and calling with something like QProcess.

I come out thinking the script is much simpler.

[wysota]

this thread has had LOTS of reads--I think it is a very common problem.

Or someone has set a periodic download of this exact page

I know this is obvious to you but there's a bit of code behind the "if-then stuff". Declare signals and slots. Connect signals and slots , Schedule the command. Go into an event loop. Execute code in slot that gets called when command finishes. Set something in the slot that let's the code that scheduled the ftp command know if it's passed or failed.

No, not really, you are overcomplicating things. You need one slot connected to QFtp::commandFinished() and in that slot pass the result of the ftp command to the job on the front of the queue, let it decide whether to execute the "then" command or not, pop it from the queue, execute the command from the current tip of the queue, rince and repeat. No need for any signal declarations, signal connecting or disconnecting, no need for any extra event loops.

[anda_skoa]

That's really my point--complexity versus a simple synchronous call.

Sure, these are just the side effects of different use cases.
Qt applications are usually faced with the tasked of doing more than one thing at the same time, e.g. having a UI and doing network communication, or doing timeout guarded network communication, etc.

So Qt's classes for potentially long running operations are designed to be asynchronous to make that possible.

So it seems your program is a non-GUI/console tool that does not need any of that and you might indeed be better of by using a script instead.

Cheers,
_