POST Request problems with IDrive EVS REST APIs

[PaceyIV]

I'm trying to use IDrive EVS REST APIs to upload a file with this service witout success.

This is the upload api (http://evs.idrive.com/web-file-upload-api.htm)
Upload local file(s) to your IDrive account
API Call: https://<server address>/evs/uploadFile
Method POST
Parameters uid=USERNAME&pwd=PASSWORD&pvtkey=OPTIONAL_PRIVATE_ KEY&p=DESTPATH&myfiles=FILENAME
enctype multipart/form-data

I try this piece of code but doesn't work. IDrive says "REQUEST 'ENCTYPE' IS NOT MULTIPART FORM DATA"

QFile *file = new QFile(sourcePath);
    file->open(QIODevice::ReadOnly);
 
    QHttpMultiPart *multiPart = new QHttpMultiPart(QHttpMultiPart::FormDataType);
    QHttpPart userNamePart;
    userNamePart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"uid\""));
    userNamePart.setBody(this->_userName.toUtf8());
    QHttpPart passwordPart;
    passwordPart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"pwd\""));
    passwordPart.setBody(this->_password.toUtf8());
    QHttpPart destPart;
    destPart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"p\""));
    destPart.setBody(destPath.toUtf8());
 
     QHttpPart imagePart;
     imagePart.setHeader(QNetworkRequest::ContentTypeHeader, QVariant("image/jpeg"));
     imagePart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"image\"; filename=\"" + file->fileName().toUtf8()+"\"\r\n\r\n\r\n"));
     imagePart.setBodyDevice(file);
     file->setParent(multiPart); // we cannot delete the file now, so delete it with the multiPart
 
     multiPart->append(userNamePart);
     multiPart->append(passwordPart);
     multiPart->append(destPart);
     multiPart->append(imagePart);
 
    QNetworkRequest req;
    req.setUrl(QUrl(QString("https://%0/evs/uploadFile").arg(this->_serverAddress)));
 
    if (this->syncPost(req, multiPart)) {
        return (this->_reply.result()["message"] == "SUCCESS");
    }
    return false;
 
//    req.setHeader(QNetworkRequest::ContentTypeHeader, "multipart/form-data;  charset=UTF-8;  boundary=" + boundary);
//    QFile file(sourcePath);
//    file.open(QIODevice::ReadOnly);
//    QByteArray postData(QString("uid=%0&pwd=%1&p=%3&myfiles=%4")
//                        .arg(this->_userName)
//                        .arg(this->_password)
//                        .arg(destPath)
//                        .arg(file.fileName()).toUtf8());
//    postData.append("--" + boundary + "\r\n");
//    postData.append("Content-Type: image/jpeg\r\n\r\n");
//    postData.append(file.readAll());
//    file.close(); // the file is opened earlier in the code
//    postData.append("\r\n");
//    postData.append("--" + boundary + "\r\n");
//    int contentLength = postData.length();
//    req.setHeader(QNetworkRequest::ContentLengthHeader, contentLength);
 
}

QFile *file = new QFile(sourcePath);
    file->open(QIODevice::ReadOnly);

    QHttpMultiPart *multiPart = new QHttpMultiPart(QHttpMultiPart::FormDataType);
    QHttpPart userNamePart;
    userNamePart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"uid\""));
    userNamePart.setBody(this->_userName.toUtf8());
    QHttpPart passwordPart;
    passwordPart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"pwd\""));
    passwordPart.setBody(this->_password.toUtf8());
    QHttpPart destPart;
    destPart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"p\""));
    destPart.setBody(destPath.toUtf8());

     QHttpPart imagePart;
     imagePart.setHeader(QNetworkRequest::ContentTypeHeader, QVariant("image/jpeg"));
     imagePart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"image\"; filename=\"" + file->fileName().toUtf8()+"\"\r\n\r\n\r\n"));
     imagePart.setBodyDevice(file);
     file->setParent(multiPart); // we cannot delete the file now, so delete it with the multiPart

     multiPart->append(userNamePart);
     multiPart->append(passwordPart);
     multiPart->append(destPart);
     multiPart->append(imagePart);

    QNetworkRequest req;
    req.setUrl(QUrl(QString("https://%0/evs/uploadFile").arg(this->_serverAddress)));

    if (this->syncPost(req, multiPart)) {
        return (this->_reply.result()["message"] == "SUCCESS");
    }
    return false;

//    req.setHeader(QNetworkRequest::ContentTypeHeader, "multipart/form-data;  charset=UTF-8;  boundary=" + boundary);
//    QFile file(sourcePath);
//    file.open(QIODevice::ReadOnly);
//    QByteArray postData(QString("uid=%0&pwd=%1&p=%3&myfiles=%4")
//                        .arg(this->_userName)
//                        .arg(this->_password)
//                        .arg(destPath)
//                        .arg(file.fileName()).toUtf8());
//    postData.append("--" + boundary + "\r\n");
//    postData.append("Content-Type: image/jpeg\r\n\r\n");
//    postData.append(file.readAll());
//    file.close(); // the file is opened earlier in the code
//    postData.append("\r\n");
//    postData.append("--" + boundary + "\r\n");
//    int contentLength = postData.length();
//    req.setHeader(QNetworkRequest::ContentLengthHeader, contentLength);

}

To copy to clipboard, switch view to plain text mode 

I also try with the commented code but I got the same error!

How should I do?

Albano

[wysota]

Check if the data is properly formatted before sending it.

[PaceyIV]

How can I see the raw data that I send?
I can't capture tcp packets with Wiresharsk becouse I have to use a https connection.

[wysota]

You don't have to send that data to this particular server, you know... you just need to see the outgoing request.

[PaceyIV]

I've sent the request to a my php page that should print all the post request but I can't understand the problem.

I can upload file with this piece of code

QFile fileUp("FILE PATH");
    fileUp.open(QIODevice::ReadOnly);
    QByteArray file(fileUp.readAll());
    fileUp.close();
 
    QByteArray boundary("AaB03x");
 
    QNetworkRequest req(QUrl("https://evsweb39.idrive.com/evs/uploadFile"));
    req.setRawHeader("Content-Type", QByteArray("multipart/form-data; charset=UTF-8; boundary=") + boundary);
 
    QByteArray postData;
    postData += "--" + boundary + "\r\n";
    postData += "Content-Disposition: form-data; name=\"uid\"\r\n\r\n";
    postData += userName + "\r\n";
 
    postData += "--" + boundary + "\r\n";
    postData += "Content-Disposition: form-data; name=\"pwd\"\r\n\r\n";
    postData += passWord + "\r\n";
 
    postData += "--" + boundary + "\r\n";
    postData += "Content-Disposition: form-data; name=\"p\"\r\n\r\n";
    postData += remotePath + "\r\n";
 
    postData += "--" + boundary + "\r\n";
    postData += "Content-Disposition: form-data; filename=\"" + fileName + "\";\r\nContent-Type: application/octet-stream\r\n\r\n";
    postData += file;
    postData += "\r\n";
 
    postData += "--" + boundary + "--\r\n";
 
    req.setHeader(QNetworkRequest::ContentLengthHeader, QString::number(postData.length()));

QFile fileUp("FILE PATH");
    fileUp.open(QIODevice::ReadOnly);
    QByteArray file(fileUp.readAll());
    fileUp.close();

    QByteArray boundary("AaB03x");

    QNetworkRequest req(QUrl("https://evsweb39.idrive.com/evs/uploadFile"));
    req.setRawHeader("Content-Type", QByteArray("multipart/form-data; charset=UTF-8; boundary=") + boundary);

    QByteArray postData;
    postData += "--" + boundary + "\r\n";
    postData += "Content-Disposition: form-data; name=\"uid\"\r\n\r\n";
    postData += userName + "\r\n";

    postData += "--" + boundary + "\r\n";
    postData += "Content-Disposition: form-data; name=\"pwd\"\r\n\r\n";
    postData += passWord + "\r\n";

    postData += "--" + boundary + "\r\n";
    postData += "Content-Disposition: form-data; name=\"p\"\r\n\r\n";
    postData += remotePath + "\r\n";

    postData += "--" + boundary + "\r\n";
    postData += "Content-Disposition: form-data; filename=\"" + fileName + "\";\r\nContent-Type: application/octet-stream\r\n\r\n";
    postData += file;
    postData += "\r\n";

    postData += "--" + boundary + "--\r\n";

    req.setHeader(QNetworkRequest::ContentLengthHeader, QString::number(postData.length()));

To copy to clipboard, switch view to plain text mode 

How can I do it using QHttpMultiPart?

[wysota]

Did you test the output you get using QHttpMultiPart?

[PaceyIV]

What I see with my custom php page is the same but I'm not sure. I've got some problems retrieving the raw post request by php.

[wysota]

Use netcat instead of the php servlet.

[PaceyIV]

Great! With netcat I can see the difference!

QHttpMultiPart add this header: MIME-Version: 1.0
Is this important? Why if I don't add this header I can upload this file. In the examples of RFC1867 they don't add the MIME version (http://www.faqs.org/rfcs/rfc1867.html). Why Qt does it?

[wysota]

QHttpMultiPart add this header: MIME-Version: 1.0
Is this important?

Apparently so.

Why Qt does it?

It's a standard header so I don't see why it would cause your message to fail. I would rather assume there is some other difference you are missing.

[PaceyIV]

You're right!
I add the MIME version header with my working code and it continues to work, so I found the real difference! QHttpMultipart doesn't add "\r\n" to the end of the request. If I don't do this the server rejects the request!

What I can do?

[PaceyIV]

I'm studying RFC standards and and I think I found a bug in QHttpMultipart class.

In the Qt documentation class I found this lines:

QHttpMultiPart::FormDataType corresponds to the "multipart/form-data" subtype, meaning the body parts contain form elements, as described in RFC 2388.

reference: http://qt-project.org/doc/qt-5.0/qht...ntentType-enum

In the RFC 2388 they say that:

3. Definition of multipart/form-data

The media-type multipart/form-data follows the rules of all multipart
MIME data streams as outlined in [RFC 2046].

reference: http://www.ietf.org/rfc/rfc2388.txt

In the RFC 2046 they say that:

The Content-Type field for multipart entities requires one parameter,
"boundary". The boundary delimiter line is then defined as a line
consisting entirely of two hyphen characters ("-", decimal value 45)
followed by the boundary parameter value from the Content-Type header
field, optional linear whitespace, and a terminating CRLF.

[...]

The boundary delimiter MUST occur at the beginning of a line, i.e.,
following a CRLF, and the initial CRLF is considered to be attached
to the boundary delimiter line rather than part of the preceding
part. The boundary may be followed by zero or more characters of
linear whitespace. It is then terminated by either another CRLF and
the header fields for the next part, or by two CRLFs, in which case
there are no header fields for the next part.

[...]

The boundary delimiter line following the last body part is a
distinguished delimiter that indicates that no further body parts
will follow. Such a delimiter line is identical to the previous
delimiter lines, with the addition of two more hyphens after the
boundary parameter value.

--gc0pJq0M:08jU534c0p--

NOTE TO IMPLEMENTORS: Boundary string comparisons must compare the
boundary value with the beginning of each candidate line. An exact
match of the entire candidate line is not required; it is sufficient
that the boundary appear in its entirety following the CRLF.

reference: http://www.ietf.org/rfc/rfc2046.txt

So the last line of a multipart/form-data MUST BE

"--" + BOUNDARY_TAG + "--" + "\r\n"

"--" + BOUNDARY_TAG + "--" + "\r\n"

To copy to clipboard, switch view to plain text mode 

QHttpMultipart doesn't add the last CRLF at the end of the packet.

[PaceyIV]

QTBUG-25429: https://bugreports.qt-project.org/browse/QTBUG-25429

https://codereview.qt-project.org/#change,24142