fearu
3rd December 2010, 15:10
Hi,
I'm trying to automate the download of a series of html, plus some processing I want to do on them. The idea is to automate tools/libraries to do the subtasks for me. I was doing the task of downloading the htmls with imacros for firefox, but for various reasons it is imposible for me to automate it.
So I wonder if it is possible to use qtwebkit (maybe qwebpage?) to do this. The imacros script is like this:
VERSION BUILD=7031111 RECORDER=FX
TAB T=1
SAVEAS TYPE=CPL FOLDER=* FILE=+_{{!NOW:yyyymmdd_hhnnss}}
WAIT SECONDS=1
TAG POS=1 TYPE=A ATTR=ID:next
The last line is the most important, it is like clicking on a link for going to the next page of the series of html (so the next time it downloads the next page).
But I have no idea how to do this. I thought doing it with qtwebkit (not downloading directly with qhttp), because, looking at the source code, the link for clicking next is always the same, I don't know if the server uses the http referrer or anything else, because the page contains a lot of javascript.
I'm trying to automate the download of a series of html, plus some processing I want to do on them. The idea is to automate tools/libraries to do the subtasks for me. I was doing the task of downloading the htmls with imacros for firefox, but for various reasons it is imposible for me to automate it.
So I wonder if it is possible to use qtwebkit (maybe qwebpage?) to do this. The imacros script is like this:
VERSION BUILD=7031111 RECORDER=FX
TAB T=1
SAVEAS TYPE=CPL FOLDER=* FILE=+_{{!NOW:yyyymmdd_hhnnss}}
WAIT SECONDS=1
TAG POS=1 TYPE=A ATTR=ID:next
The last line is the most important, it is like clicking on a link for going to the next page of the series of html (so the next time it downloads the next page).
But I have no idea how to do this. I thought doing it with qtwebkit (not downloading directly with qhttp), because, looking at the source code, the link for clicking next is always the same, I don't know if the server uses the http referrer or anything else, because the page contains a lot of javascript.