Here is a page I am trying to parse. I am trying to extract the "Tender numbers" from the HTML table on the page, which I can do just fine. The problem is that the HTML table is segmented into 5 segments and can be navigated using segment numbers at the bottom of the table. At the moment my program extracts only "Tender numbers" on segment 1 and not the rest of the segments. When I open the web page in a native browser, the entire HTML table appears in one HTML source file not segmented, hence I was under the impression that if I use QWebEnginePage::runJavaScript() on the page to extract the "Tender numbers", the function would get all of them. How can I solve this problem. Thanking you in advance.
Qt Code: 
  1. void Cape_Town_Page::mine_to_tenders()
  2. {
  3.     QFile tenderRefs_file("C:/Users/C5248134/Desktop/Projects/Ithala/Include/tender_refs.txt");
  4.  
  5.     if(!tenderRefs_file.open(QFile::WriteOnly | QFile::Text))
  6.     {
  7.         qDebug() << "Tender refs file did not open ---- " <<tenderRefs_file.errorString() <<endl;
  8.     }
  9.     else
  10.         qDebug() << "opened" <<endl;
  11.  
  12.     QTextStream out(&tenderRefs_file);
  13.     QStringList refs_list;
  14.  
  15.     if(this->url().toString() == "http://web1.capetown.gov.za/web1/TenderPortal/Tender")
  16.     {
  17.         QString refs_script;
  18.  
  19.         refs_script.append("function getRefs(){"
  20.                            "var refs = document.body.querySelectorAll(\"tr.gridDetails\");"
  21.                            "var refArr = [];"
  22.                            "var i;"
  23.                            "for(i = 0; i < refs.length; i++){"
  24.                            "refArr[i] = refs[i].firstElementChild.textContent;"
  25.                            "}"
  26.                            "return refArr;"
  27.                            "}"
  28.                            "getRefs();"
  29.                            );
  30.  
  31.  
  32.         runJavaScript(refs_script, [&](const QVariant data){
  33.           qDebug() << data.toStringList() <<endl;
  34.  
  35.         });
  36.  
  37.     }
  38.     else
  39.         return;
  40.  
  41. }
To copy to clipboard, switch view to plain text mode