PDA

View Full Version : Http Posting using QNetworkAccessManager



Mohammadhzp
20th January 2014, 22:37
Hello
I'm facing another problem
it's about uploading file through HTTP posting
this is my code


from PyQt4 import QtCore, QtGui, QtNetwork
import sys
class Window(QtGui.QWidget):
def __init__(self, parent=None):
super(Window, self).__init__(parent)

self.netaccess = QtNetwork.QNetworkAccessManager(self)
self._uploaders = {}
row = 1 #this number will change per upload in real program,this is just demo
self.address = 'http://localhost/01/upload/demo.php'
stream = QtCore.QFile('/home/patriot/icon.png')
if stream.open(QtCore.QIODevice.ReadOnly):
data = stream

uploader = self._uploaders[row] = Uploader(row, self.netaccess)

uploader.upload(data, self.address)

class Uploader(QtCore.QObject):

def __init__(self, key, parent):
QtCore.QObject.__init__(self, parent)
self._key = key
self._reply = None


def upload(self, data, url):
if self._reply is None:

self._stream = data

self._multiPart = QtNetwork.QHttpMultiPart(QtNetwork.QHttpMultiPart. FormDataType)

fileName = QtCore.QFileInfo(self._stream.fileName()).fileName ()
key = 'file'

imagePart = QtNetwork.QHttpPart()
imagePart.setHeader(QtNetwork.QNetworkRequest.Cont entDispositionHeader,
"form-data; name=\"%s\"; filename=\"%s\"" % (key, fileName))
imagePart.setHeader(QtNetwork.QNetworkRequest.Cont entTypeHeader,
'image/png')
imagePart.setBodyDevice(self._stream)

self._multiPart.append(imagePart)

request = QtNetwork.QNetworkRequest(QtCore.QUrl(url))
request.setHeader(QtNetwork.QNetworkRequest.Conten tTypeHeader,
'multipart/form-data; boundary=%s' % self._multiPart.boundary())
request.setRawHeader('User-Agent','Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36')

self._reply = self.parent().post(request, self._multiPart)
self._reply.uploadProgress.connect(self.handleUplo adProgress)
self._reply.error.connect(self.handleError)
self._reply.finished.connect(self.handleFinished)



def handleUploadProgress(self, sent, total):
if sent >= total:
# prevent duplicated uploads
self._reply.close()

def handleFinished(self):
print('Content: ',self._reply.readAll())#no output here :(
self._stream.close()
self._multiPart.deleteLater()
self._reply.deleteLater()
self._reply = None
app.quit()

def handleError(self):
print('Error String :',self._reply.errorString())
print('Error number: ',self._reply.error())


app = QtGui.QApplication(sys.argv)
demo = Window()
sys.exit(app.exec_())

I use an answer in stackoverflow.com,the code needs to upload to a php script but I don't know why it's not working
anyone have any idea ?anything
thanks

edit: this is html form

<form action="demo.php" method="post"
enctype="multipart/form-data">
<label for="file">Filename:</label>
<input type="file" name="file" id="file"><br>
<input type="submit" name="submit" value="Submit">
</form>

ChrisW67
21st January 2014, 00:54
Install Wireshark (UNIX or Windows).
Open the HTML form in your browser,
Submit the form using the same data as your Python program (Make it a small file).
Inspect what exactly is sent.
Run your program.
Inspect what exactly is sent.

The differences will be informative.

Mohammadhzp
21st January 2014, 01:36
I'll look into this exactly
but my problem is I can't see anything from python(even a simple echo in php)
I think I can't even get into apache(for no reason)
I can upload with requests(a python library) with this headers,but in Qt I can't,nothing,something is not right I can't figure out what is that.hope members could help

ChrisW67
21st January 2014, 02:06
If your Qt code cannot even fetch a page from Apache (a GET) then you have more fundamental issues. You should work these out first.

Read the docs for QNetworkReply::uploadProgress() again and ask yourself what happens if the total size cannot be determined.

BTW: I do not see any reason from the close() call at all.

Mohammadhzp
21st January 2014, 02:27
I edited my first code,I was thinking because my code is long it's looks ugly,thanks for mention that


If your Qt code cannot even fetch a page from Apache (a GET) then you have more fundamental issues. You should work these out first.

Read the docs for QNetworkReply::uploadProgress() again and ask yourself what happens if the total size cannot be determined.

BTW: I do not see any reason from the close() call at all.

you see,I need to upload through HTTP,at first when I was writing my first Qt program and did not know about HTTP stuff,I read data as a QByteArray and pass it to Uploader,after processing was done I could fetch the page and I could see contents,so uploadProgress is Ok,when I added QHttpMultiPart first of all I change data from QByteArray to QFile then pass it to Uploader and append data to multipart,at the end I posted QHttpMultiPart and request
All I'm saying is Since I added QHttpMultiPart I can't fetch any pages,do you still think it's because QNetworkReply::uploadProgress() ?
If I don't use close() or deleteLater() program will crash(seagment crash) or if you mean in UploadProgress it's because PyQt sometimes will post file twice(after uploading it start uploading again) I don't know it's my code or PyQt bug,but that solved my problem

ChrisW67
21st January 2014, 06:04
Yes, I suspect it is. I suspect you are closing the reply at the first sign of progress. Print the value of total in handleUploadProgress.

Mohammadhzp
21st January 2014, 10:44
I print both total and sent and at the end the value were the same(sent = total)
Is this mean that closing reply is ok ? if yes do you have any idea ?


Oh Chris,you are great,I did remove self._reply.close() in my real program and I could see a variable which I echo in PHP,you were right
but still,can't send any file

anda_skoa
21st January 2014, 13:09
You don't have to close a QNetworkReply. It will emit its finished() signal when it is done. If you don't need to do anything else when it is done, just connect the signal to the reply's deleteLater() slot.
If you have other clean up to do, connect to a slot elsewhere.

Cheers,
_

Mohammadhzp
21st January 2014, 13:15
Thanks for reply
Yeah I got that but my main problem right now is not that,It's uploading file or data to a PHP script

ChrisW67
21st January 2014, 21:20
Can you see the data in the network transfer using Wireshark (or Fiddler)?

Mohammadhzp
21st January 2014, 22:22
Yes I did
these are the result:
html form :
9955

python program:
9956

some headers are not sending while In my code I'm sending them,:confused:

Mohammadhzp
22nd January 2014, 10:36
can someone drive me to right direction please ?

ChrisW67
23rd January 2014, 09:13
Your multipart form data needs to include the other elements of the form. In this case the submit button.


imagePart = QtNetwork.QHttpPart()
imagePart.setHeader(QtNetwork.QNetworkRequest.Cont entDispositionHeader,
"form-data; name=\"%s\"; filename=\"%s\"" % (key, fileName))
imagePart.setHeader(QtNetwork.QNetworkRequest.Cont entTypeHeader,
'image/jpeg')
imagePart.setBodyDevice(self._stream)

submitPart = QtNetwork.QHttpPart()
submitPart.setHeader(QtNetwork.QNetworkRequest.Con tentDispositionHeader,
"form-data; name=\"submit\"")
submitPart.setBody("submit")

self._multiPart.append(imagePart)
self._multiPart.append(submitPart)

Mohammadhzp
23rd January 2014, 11:12
I tried this but no differences.still not working
I don't know what to do anymore :(

ChrisW67
23rd January 2014, 23:55
Yes it is. Your code with only the web server and file path changed, the submit button encoded, and the close() call removed:


from PyQt4 import QtCore, QtGui, QtNetwork
import sys
class Window(QtGui.QWidget):
def __init__(self, parent=None):
super(Window, self).__init__(parent)

self.netaccess = QtNetwork.QNetworkAccessManager(self)
self._uploaders = {}
row = 1 #this number will change per upload in real program,this is just demo
self.address = 'http://dev/demo.php'
stream = QtCore.QFile('icon.png')
if stream.open(QtCore.QIODevice.ReadOnly):
data = stream

uploader = self._uploaders[row] = Uploader(row, self.netaccess)

uploader.upload(data, self.address)

class Uploader(QtCore.QObject):

def __init__(self, key, parent):
QtCore.QObject.__init__(self, parent)
self._key = key
self._reply = None


def upload(self, data, url):
if self._reply is None:

self._stream = data

self._multiPart = QtNetwork.QHttpMultiPart(QtNetwork.QHttpMultiPart. FormDataType)

fileName = QtCore.QFileInfo(self._stream.fileName()).fileName ()
key = 'file'

imagePart = QtNetwork.QHttpPart()
imagePart.setHeader(QtNetwork.QNetworkRequest.Cont entDispositionHeader,
"form-data; name=\"%s\"; filename=\"%s\"" % (key, fileName))
imagePart.setHeader(QtNetwork.QNetworkRequest.Cont entTypeHeader,
'image/png')
imagePart.setBodyDevice(self._stream)

submitPart = QtNetwork.QHttpPart()
submitPart.setHeader(QtNetwork.QNetworkRequest.Con tentDispositionHeader,
"form-data; name=\"submit\"")
submitPart.setBody("submit")

self._multiPart.append(imagePart)
self._multiPart.append(submitPart)

request = QtNetwork.QNetworkRequest(QtCore.QUrl(url))
request.setHeader(QtNetwork.QNetworkRequest.Conten tTypeHeader,
'multipart/form-data; boundary=%s' % self._multiPart.boundary())
request.setRawHeader('User-Agent','Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36')

self._reply = self.parent().post(request, self._multiPart)
self._reply.uploadProgress.connect(self.handleUplo adProgress)
self._reply.error.connect(self.handleError)
self._reply.finished.connect(self.handleFinished)



def handleUploadProgress(self, sent, total):
print sent, total

def handleFinished(self):
print('Content: ',self._reply.readAll())#no output here :(
self._stream.close()
self._multiPart.deleteLater()
self._reply.deleteLater()
self._reply = None
app.quit()

def handleError(self):
print('Error String :',self._reply.errorString())
print('Error number: ',self._reply.error())


app = QtGui.QApplication(sys.argv)
demo = Window()
sys.exit(app.exec_())

Using this simple PHP file upload receiver:


$ cat demo.php
<html>
<body>
<form method="post" enctype="multipart/form-data">
<label for="file">Filename:</label>
<input type="file" name="file" id="file" />
<br />
<input type="submit" name="submit" value="Submit" />
</form>
<p>
<?php
if(isset($_POST['submit'])) {
if ($_FILES["file"]["error"] > 0) {
echo "Error: " . $_FILES["file"]["error"] . "<br />";
} else {
echo "Upload: " . $_FILES["file"]["name"] . "<br />";
echo "Type: " . $_FILES["file"]["type"] . "<br />";
echo "Size: " . ($_FILES["file"]["size"] / 1024) . " Kb<br />";
echo "Stored in: " . $_FILES["file"]["tmp_name"];
}
}
?>
</p>
</body>
</html>

Gives this output with an artificially large icon.png:


16384 530779
32768 530779
...
524288 530779
530779 530779
('Content: ', PyQt4.QtCore.QByteArray('<html>\n <body>\n <form method="post" enctype="multipart/form-data">\n <label for="file">Filename:</label>\n <input type="file" name="file" id="file" /> \n <br />\n <input type="submit" name="submit" value="Submit" />\n </form>\n <p>\n Upload: icon.png<br />Type: image/png<br />Size: 517.9921875 Kb<br />Stored in: /tmp/phpKg8FG7 </p>\n </body>\n</html>\n'))

Mohammadhzp
24th January 2014, 01:52
Oh my God
I can't believe finally solution is here
I don't know how to say thank you,you helped me a lot,thanks Chris
it's working like a charm
but I also found the real issue in here
I tried your code in python3 for the first time and like always I didn't get any answer(no file uploaded) I become so upset since your code didn't working too !
anyway,I just thought maybe it's because PyQt4 have problem in python3,So I installed PyQt4 on python2 and Got exactly your result and file is uploading too
so it's obviously a PyQt4 bug in python3(the exact code is working on python2 but not in python3)
Do I need to report this ? how ?

ChrisW67
24th January 2014, 04:39
I am running Python 2.7.5 and Python 3.2.5 with PyQt 4.10.2 on 64-bit Linux.

My Python 3.2 on Linux also fails to send the data. The multipart message never makes it onto the wire correctly. Look closely at the two requests in Wireshark. With Python 2.7 the request goes on the wire with this header:


MIME Multipart Media Encapsulation, Type: multipart/form-data, Boundary: "boundary_.oOo._Nzc0MjMxOTU4MTgyMzc5MTk0Mw==NzIxMTQ zNjA4"

but with Python 3 the Boundary value gets mangled:


MIME Multipart Media Encapsulation, Type: multipart/form-data, Boundary: "b'boundary_.oOo._NjY2OTgyMTQ3MTg1NTk1MzgyNw==NDY5N jg1NTMw'"

resulting in a malformed payload (because the payload is encoded with boundary = "boundary_.oOo._NjY2OTgyMTQ3MTg1NTk1MzgyNw==NDY5Njg 1NTMw"). The PyQt4 under Python 3 is outputting the boundary as a bytes literal not a string. This could be a python wrapper type mapping issue.

You should address the issue to the PyQt mailing list (http://www.riverbankcomputing.com/mailman/listinfo/pyqt) or Riverbank Software directly if you have a commercial licence.

Mohammadhzp
24th January 2014, 18:00
I reported bug
Thanks for help Chris