Hi there guys, compliments of the season. I'm creating this web crawler that mines data off websites. Here is my problem, I am using QNAM to pull the HTML from a website and then I create a QWebEnginePage object where I set the HTML to the page for the purpose of using the QWebEnginePage::runJavaScript() to extract the data I need from the HTML. The problem is that the QWebEnginePage::runJavaScript() function doesn't work if I run it immediately after the call to QWebEnginePage::setHtml(). So I resorted to calling a lambda within QObject::connect() where the signal is QWebEnginePage::loadFinished() and the outcome was that QWebEnginePage::runJavaScript() will retrieve data that is typically at the top of the page (e.g. "document.title;" will run) but the function return an empty string for the data that is in the mid-section to the end of the HTML document. This gives me an impression that I am trying to access elements that are not loaded by the page yet, which is confusing because I am running a lambda on a loadFinished() signal. Please see code below.
void CPT_Page::getTenderSite()
{
page_1 = new QWebEnginePage(this);
page_1->setHtml(tenderPage_reply->readAll());
QWebEngineView *view_1 = new QWebEngineView();
view_1->setPage(page_1);
view_1->show();
QObject::connect(page_1,
&QWebEnginePage
::loadFinished,
[&](){page_1
->runJavaScript
("document.getElementById(\"rfqsTable\").innerHTML;" qDebug() << data.toString() <<endl;
});});
void CPT_Page::getTenderSite()
{
page_1 = new QWebEnginePage(this);
page_1->setHtml(tenderPage_reply->readAll());
QWebEngineView *view_1 = new QWebEngineView();
view_1->setPage(page_1);
view_1->show();
QObject::connect(page_1, &QWebEnginePage::loadFinished, [&](){page_1->runJavaScript("document.getElementById(\"rfqsTable\").innerHTML;"
,[&](const QVariant &data){
qDebug() << data.toString() <<endl;
});});
To copy to clipboard, switch view to plain text mode
}
Bookmarks