Originally Posted by
anda_skoa
Are you sure the table is even there if you disable the script?
I.e. isn't jQuery creating it?
_
Because I am using QNAM to get the HTML, I get all the table data and all the page scripts. When I disable the scripts, the table is still there in pure html format but because the scripts are disabled, I am unable to run my own script. If I enable the scripts, something in JQuery interferes with my own script hence I cannot retrieve the data.
The url below is the site I am trying to get the data from (i.e. the data inside the table).
http://web1.capetown.gov.za/web1/TenderPortal/Tender/
I've been exploring a different approach where I try to strip the HTML of all in-page scripts (i.e. using regular expressions) before I set the HTML to the QWebEnginePage. I am still working on getting the correct regular expression though, (I.e. one that is going to match everything in-between script tags and then using QString::remove() I will remove the scripts from the HTML). But I am still struggling to get the right Regular Expression. See code below.
qDebug() << temp_html <<endl;
QFile linksFile
(QDir::currentPath().
append("/Include/Program_Files/tenderMainPage2.txt"));
if(!linksFile.
open(QFile::WriteOnly |
QFile::Text)) {
msgBox_2.setText("Tender Main File did not open...");
msgBox_2.exec();
return;
}
out << temp_html <<endl;
page_1 = new QWebEnginePage(this);
//page_1->settings()->setAttribute(QWebEngineSettings::JavascriptEnabled, true);
page_1->setHtml(temp_html);
QWebEngineView *view_1 = new QWebEngineView();
view_1->setPage(page_1);
view_1->show();
QObject::connect(page_1,
&QWebEnginePage
::loadFinished,
[&](){page_1
->runJavaScript
("document.getElementById(\"rfqsTable\").innerHTML;" ,QWebEngineScript
::MainWorld,
[&](const QVariant &data
){ qDebug() << data.toString() <<endl;
});});
}
QString html = (QString)tenderPage_reply->readAll();
QString temp_html = html.remove(QRegExp("\\b<script.*</script>\b"));
qDebug() << temp_html <<endl;
QFile linksFile(QDir::currentPath().append("/Include/Program_Files/tenderMainPage2.txt"));
if(!linksFile.open(QFile::WriteOnly | QFile::Text))
{
QMessageBox msgBox_2;
msgBox_2.setText("Tender Main File did not open...");
msgBox_2.exec();
return;
}
QTextStream out(&linksFile);
out << temp_html <<endl;
page_1 = new QWebEnginePage(this);
//page_1->settings()->setAttribute(QWebEngineSettings::JavascriptEnabled, true);
page_1->setHtml(temp_html);
QWebEngineView *view_1 = new QWebEngineView();
view_1->setPage(page_1);
view_1->show();
QObject::connect(page_1, &QWebEnginePage::loadFinished, [&](){page_1->runJavaScript("document.getElementById(\"rfqsTable\").innerHTML;"
,QWebEngineScript::MainWorld, [&](const QVariant &data){
qDebug() << data.toString() <<endl;
});});
}
To copy to clipboard, switch view to plain text mode
Bookmarks