How to run JavaScript with the contencs of a QNetworkReply

[WetCode]

Hello all I hope someone here can set me straight on this since I can't for the heck of it figure it out.
Am no pro so please bear with me.

I have a program that downloads a HTML file and parses out some information.
And its all working good, but some of the information is being displayed by a JavaScript.
So in the QString witch I have the complete source code of the page I see the code for the JavaScript.
Not the info I want, is there a way I can run the entire script so I can get the info I need?

here is a snip of the code showing how i store the html page.

// function executed when QNetworkReply emits finished()
void getHTMLFile::replyFinsihed(QNetworkReply *reply)
{
    QByteArray qbData = reply->readAll();
    sData.clear();
    sData = qbData;
    // continue with parsing data.
}

// function executed when QNetworkReply emits finished()
void getHTMLFile::replyFinsihed(QNetworkReply *reply)
{
    QByteArray qbData = reply->readAll();
    sData.clear();
    sData = qbData;
    // continue with parsing data.
}

To copy to clipboard, switch view to plain text mode 

I have googeld around for a while and found this.

QWebSettings::globalSettings()->setAttribute(..);

QWebSettings::globalSettings()->setAttribute(..);

To copy to clipboard, switch view to plain text mode 

But have not found a setting like

::evaluateJavaScript || ::enableScript

::evaluateJavaScript || ::enableScript

To copy to clipboard, switch view to plain text mode 

There is a function in

QWebView::evaluateJavaScript()

QWebView::evaluateJavaScript()

To copy to clipboard, switch view to plain text mode 

But from what i could find of information its not what i need (I may be wrong ofcorse..)

Any help and or guidance is much appreciated.

Cheers
WetCode

[wysota]

But from what i could find of information its not what i need (I may be wrong ofcorse..)

Get the page using QWebPage. It should run the script and you can query the page for results you need.

[WetCode]

Thank you for your reply much appreciated, i will try this when i get home from work.
But when you say query the page what do you mean exactly?
Since in my mind i query the page when i enter the URL for the web page.
Or are you refering to the parsing of the page after its been "downloaded" to the QWebView?

Cheers
WetCode

[wysota]

But when you say query the page what do you mean exactly?

I mean do with the page whatever you wanted to do with it in the first place ("parse the page" as you said in the opening post).

[WetCode]

Ok thank you for clarifying .

[WetCode]

So I have try the QWebView and got the script running with almost no effort thank you Qt.
But I have failed to get the QWebVeiw content in a QByteArray or QString, is there no readAll() for QWebView or QWebPage?

Thanks for any help.

Cheers
WetCode

[wysota]

A regular approach would be to use QWebElement API to parse page content.

[WetCode]

I have read the docs on QWebView , QWebPage , QWebFrame, QWebElement sevrel times now and i stile cant figur this out.
Take the QWebView that have finished and have all the data in the page \ Widget. (Widget is not needed btw its suposed to do this in the background).
So some JavaScript have been executed and all data is visable, but messed around with the classes i mentiond and cant get it into a QString.

I dont like to do this but, whould anyone be so kind to give me an example on how take all the text even the text that got generated by the JavaScript.
And place the data in a QString \ QWebView to QWebElement to QString if that is the more correct way?
Am so confiused right now and my Qt books are making me even more confiused.

Aprichiate all the help.

Cheers
WetCode

[wysota]

Why do you want to take all the text from the page? It is much easier to query the page's DOM for whatever data you require.

[WetCode]

Its not that its essential to get the entire page.
Am just having trouble with how to go about getting to the data.
When using the classes and reading the docs I seem to be missing functions.
This leads me to think am not doing it correctly and I may be asking the "wrong" questions.

Cheers
WetCode

[wysota]

You should start with QWebFrame::findFirstElement().

[WetCode]

Ok in the Documentation it says that I can use the classes with out the QWebView. (since I don't need to display the entire page.)
But I can't find ::setURL() or ::load(QUrl) on QWebPage or QWebFrame can't get it to work.
So when you say start with QWebFrame::findFirstElement(). Am guessing this is after I have loaded the page URL.

Since from what I managed to understand is that QWebPage is the entire page and the QWebFrame is the content on that page while QWebElement is used to search the content?

EDIT: I found this. http://www.qtcentre.org/threads/5383...parent-classes

void MainWindow::replyFinished(QNetworkReply *reply)
{
QWebPage page;
QWebFrame *frame = page.mainFrame();
 
QByteArray html = reply->peek(reply->size()); //~10kB
frame->setContent(html);
QWebElement htmlElement = frame->documentElement();
...

void MainWindow::replyFinished(QNetworkReply *reply)
{
QWebPage page;
QWebFrame *frame = page.mainFrame();
 
QByteArray html = reply->peek(reply->size()); //~10kB
frame->setContent(html);
QWebElement htmlElement = frame->documentElement();
...

To copy to clipboard, switch view to plain text mode 

I Think i know what to do now.

[wysota]

QWebFrame::load()

[WetCode]

QWebFrame::load()

Thanks for you patience wysota much appreciated.
I have been having some issues where the functions don't show even though I have the correct headers. ( Think)
Anyways I finally got the functions to show as they should think it was that I had forgot to run QMake.

But now my application crashes for some reason am I not using it correctly?
webTest1.pro:

#-------------------------------------------------
#
# Project created by QtCreator 2013-10-11T10:09:41
#
#-------------------------------------------------
 
QT       += core gui
QT       += webkit
QT       += network
QT       += webkit webkitwidgets
 
greaterThan(QT_MAJOR_VERSION, 4): QT += widgets
 
TARGET = webTest1
TEMPLATE = app
 
 
SOURCES += main.cpp\
        mainwindow.cpp
 
HEADERS  += mainwindow.h
 
FORMS    += mainwindow.ui

#-------------------------------------------------
#
# Project created by QtCreator 2013-10-11T10:09:41
#
#-------------------------------------------------

QT       += core gui
QT       += webkit
QT       += network
QT       += webkit webkitwidgets

greaterThan(QT_MAJOR_VERSION, 4): QT += widgets

TARGET = webTest1
TEMPLATE = app


SOURCES += main.cpp\
        mainwindow.cpp

HEADERS  += mainwindow.h

FORMS    += mainwindow.ui

To copy to clipboard, switch view to plain text mode 

mainwindow.h:

#ifndef MAINWINDOW_H
#define MAINWINDOW_H
 
#include <QMainWindow>
#include <QtWebKit/QWebElement>
#include <QtWebKit>
#include <QtWebKitWidgets>
 
namespace Ui {
class MainWindow;
}
 
class MainWindow : public QMainWindow
{
    Q_OBJECT
 
public:
    explicit MainWindow(QWidget *parent = 0);
    ~MainWindow();
    QWebPage *myPage;
    QWebFrame *myFrame;
    QWebElement *myElement;
 
private:
    Ui::MainWindow *ui;
};
 
#endif // MAINWINDOW_H

#ifndef MAINWINDOW_H
#define MAINWINDOW_H

#include <QMainWindow>
#include <QtWebKit/QWebElement>
#include <QtWebKit>
#include <QtWebKitWidgets>

namespace Ui {
class MainWindow;
}

class MainWindow : public QMainWindow
{
    Q_OBJECT
    
public:
    explicit MainWindow(QWidget *parent = 0);
    ~MainWindow();
    QWebPage *myPage;
    QWebFrame *myFrame;
    QWebElement *myElement;
    
private:
    Ui::MainWindow *ui;
};

#endif // MAINWINDOW_H

To copy to clipboard, switch view to plain text mode 

mainwindow.cpp:

#include "mainwindow.h"
#include "ui_mainwindow.h"
 
MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);
    myPage->mainFrame()->load( QUrl("http://google.com") );
}
 
MainWindow::~MainWindow()
{
    delete ui;
}

#include "mainwindow.h"
#include "ui_mainwindow.h"

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);
    myPage->mainFrame()->load( QUrl("http://google.com") );
}

MainWindow::~MainWindow()
{
    delete ui;
}

To copy to clipboard, switch view to plain text mode 

I also tryed:

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);
    myFrame->load(QUrl("http://google.com"));
}

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);
    myFrame->load(QUrl("http://google.com"));
}

To copy to clipboard, switch view to plain text mode 

[anda_skoa]

But now my application crashes for some reason am I not using it correctly?

Hint: calling a method on a pointer requires that the pointer point to a valid object.

Cheers,
_