C++11/Qt & Dropbox v2 API in dropboxQt

[osoft]

We recently uploaded to github Qt5 adaptation of Dropbox v2 API, see https://github.com/osoftteam/dropboxQt and homepage http://prokarpaty.net/dropbox.php

Would be interesting to hear valuable feedbacks.
It's called dropboxQt. What is special about this code is that it was mostly generated from something called Dropbox Stone documentation. It is kind a big, because API itself contains about 400 classes, but we tried hard to bring structure into code and create clean interface.

Some details about implementation - we used lambdas for slots, unique_ptr and factories for result types, also exceptions are heavily used (there are couple dozen of them) because they fit into Dropbox solutions. There are also convenience functions that don't throw exceptions but provide top level access, for example:

bool downloadFile(QString dropboxFilePath, QString localDestinationPath);
bool uploadFile(QString localFilePath, QString dropboxDestinationPath);
bool fileExists(QString dropboxPath);
bool folderExists(QString dropboxPath);

[TorAn]

Wow! This is so cool!

[anda_skoa]

It is not a very Qt like API, it seems to have blocking I/O calls.

In Qt almost everything that causes I/O with non deterministic times requirements it asynchronous.

As an application developer I would expect some form of status and progress notification when dealing with an API that accesses a network service.

See QNetworkAccessManager for an idea how to encapsulate long running asynchronous operations with the "job" pattern.

Cheers,
_

[osoft]

2 anda_skoa

That is correct, we use it in separate thread but for GUIsh application it's a problem.

I am thinking this:

downloadFileAsync(QString dropboxFilePath, QString localDestinationPath)
.setProgressCallback(std::function )
.setErrorCallback(std::function )
.start();

to implement it we would need internal working thread, queue of request and exception handling and maybe command pattern for selected API calls.

For progress we already have signal void ApiClient::progress(qint64 bytesProcessed, qint64 total);
and to cancel ApiEndpoint::cancel();

but cancellation of async calls might need revision too.

Thank you for comment, it make sense.

[anda_skoa]

That is correct, we use it in separate thread but for GUIsh application it's a problem.

This is of course an option, but not very common in Qt.



I am thinking this:

downloadFileAsync(QString dropboxFilePath, QString localDestinationPath)
.setProgressCallback(std::function )
.setErrorCallback(std::function )
.start();

That looks nice for a non-Qt C++ program but very unwieldy for a Qt program which is usually based on using signals to report change of an active component.



to implement it we would need internal working thread, queue of request and exception handling and maybe command pattern for selected API calls.

And an easy way to determine inside the callbacks which command they are associated with, a clear documentation which thread is going to execute the callbacks, etc.
All things you get "for free" when using a "response" object with signals.



For progress we already have signal void ApiClient::progress(qint64 bytesProcessed, qint64 total);
and to cancel ApiEndpoint::cancel();

I saw that and it looked quite out of place since there were not signals for finish or error, nor any way to determine for which call the progress was being reported.

Aside from the "job" or "response/reply" pattern, another patttern/idiom used by Qt for asynchronous calls is a "future" mechanism.
E.g. QDBusPendingReply, which can either be used blockingly or asynchronously (by using an async signal helper, in this case QDBusPendingCallWatcher.

Cheers,
_

[osoft]

You are right, second thread looks like over-engineering, progress signal - OK. QDBusPendingReply something worth look at.
Maybe we can just propagate QNetworkAccessManager signals into specialized callbacks and specialized exceptions callbacks without raising exception.

So it would look like this:
CreateArg arg("new_folder");
void createDropboxFolder_Async(arg,
[](std::unique_ptr<FolderCreatedResult> result)
{
cout << "created folder " << result->toString();
},
[](std::unique_ptr<FolderCreatedError> error)
{
cout << "error creating folder " << error->toString();
}
)

and then if needed have also blocking companion function:
const CreateArg arg("new_folder");
FolderCreatedResult> result = createDropboxFolder(arg);

[osoft]

just an update on development - we added async functions as discussed here.

using namespace dropboxQt;
DropboxClient dbox("ACCESS_TOKEN");
files::CreateFolderArg arg("path_to_new_folder");
dbox.getFiles()->createFolder_Async(arg,
[](std::unique_ptr<files::FolderMetadata> res)
{
qDebug() << "folder created, id=" << res->id();
},
[](std::unique_ptr<DropboxException> e)
{
qDebug() << "Exception: " << e->what();
});

and preserved blocking functions with exceptions, now they are implemented as call to async functions with callback processing and call to evenloop exec

[anda_skoa]

Very nice non Qt C++ API.

For the Qt API or wrapper I would recommend a more Qt like approach to be familiar for Qt developers.
Either a response handler object with signals or a "future" style return value that can be used with a "watcher" object for signals.

Cheers,
_

[osoft]

Taking into account latest suggestion by anda_skoa, we added one more async function that returns dynamically allocated object of

DropboxTask<T> type, similiar to QNetworkReply

The object has two signals to connect:
void completed();
void failed();

also two functions to query state:
bool isCompleted()
bool isFailed()


and two function to get access to Result class or Exception object in case of failure:
DropboxException* error()
T* get()

The object of DropboxTask<T> type should be deleted from 'completed' or 'failed' slots via 'deleteLater', similiar to QNetworkRepy.

The blocking kind of APIs works as before, the light lambda-friendly APIs got new suffix 'AsyncCB'. So that we have, for example, 3 functions to create folder:

//blocking with exception
std::unique_ptr<FolderMetadata> createFolder(const CreateFolderArg& );
//asynchronous with task as QObject-derived
DropboxTask<FolderMetadata>* createFolder_Async(const CreateFolderArg&);
//asynchronous with callbacks to register
void createFolder_AsyncCB(const CreateFolderArg&,
std::function<void(std::unique_ptr<FolderMetadata>)> completed_callback,
std::function<void(std::unique_ptr<DropboxException>)> failed_callback);

[anda_skoa]

Very nice!

Is the "completed" signal emitted in all cases of task end? I.e. it is also emitted when the task failed?

E.g. if you look at QNetworkReply it has a finished() signal that will always be emitted, no matter if the operation succeeded or failed.
The application programmer can therefore be sure that connecting to this signal will always be enough to catch the end of the operation.

Another example for that pattern in Qt is QProcess. It will also always emit the finished() signal, indepent of whether the process exited cleanly, with an error or crashed.

Cheers,
_

[osoft]

no, 'completed' is emited only in case of success, failed in case of 'error'. we can have 'finished' maybe 3rd signal emited in both cases

or maybe just have one signal 'finished' and let user check status of object, like in QNetworkReply

[anda_skoa]

no, 'completed' is emited only in case of success, failed in case of 'error'.

With two signals you need to remember to connect both or you could end up with obejcts hanging around and not being deleted.
For two signals you would need to return a shared pointer to be sure that the object is always deleted even if only one signal is being handled.

Cheers,
_

[osoft]

Converted two signal into one 'finished', seems to work but need a bit more testing.
Also replaced macros with new function
std::unique_ptr<RESULT> waitForResultAndRelease();

This function can be used to wait for async execution to finish. It auto releases Task object and moves result to caller, if available.
For example, new implementation of blocking call:

std::unique_ptr<Metadata> FilesRoutes::alphaGetMetadata(const AlphaGetMetadataArg& arg ){
return alphaGetMetadata_Async(arg)->waitForResultAndRelease();
}

[anda_skoa]

Nice, like a future's "value" getter.

Cheers,
_

[osoft]

There are certainly similarities. DropboxTask essentially can be described as typed representation of Json QNetworkReply data, something:

(Reply -> Json) -> (t, e)

where 't' is result type and 'e' is exception type, DropboxException derived.

On other note, we started something similar to dropboxQt for a subset of Google API - gdrive, gmail, gtask.

https://github.com/osoftteam/googleQt

Now we can compare both APIs. Dropbox at the low level (the endpoint of all routes) consists of 3 functions while Google of 1 function. It is possible to build Dropbox API on top of 1 function as well (dropbox API is even more consistent) but it was never objective. We found fascinating that all wrapper classes can be generated out of some declarative language (STONE), in case of Dropbox all Arguments, Results and Routes are generated, in case of Google Arguments are manually written and the rest generated.

2 anda_skoa - thank for your suggestions, it really helped to shape up the async stuff.