Results 1 to 5 of 5

Thread: Inspecting HTML elements in QWebEngine vs Chrome, Firefox, etc

  1. #1
    Join Date
    Jan 2017
    Posts
    3
    Qt products
    Qt5
    Platforms
    Windows

    Default Inspecting HTML elements in QWebEngine vs Chrome, Firefox, etc

    Hello, when accessing the html of an element on a webpage, QWebEngine gives different results to the "Inspect Element" dev tool in Chrome or Firefox. The Qt functions http://doc.qt.io/qt-5/qwebenginepage...unJavaScript-2 and http://doc.qt.io/qt-5/qwebenginepage.html#toHtml both seem to return the html before Javascript has run on the page, whereas Chrome & Firefox return it after.

    Example:

    Down the side of a youtube page eg https://www.youtube.com/watch?v=1nydxbGhgv8 is a list of similar videos with thumbnails. The brains at google store the image url for these thumbnails in a HTML5 attribute data-thumb, and swap this url into the <img src= using Javascript in the user's browser.

    When you inspect these thumbnails in Chrome & Firefox, you see a nice clean <img> tag, pointing to the correct source. You get the same thing when accessing the html of these thumbnails with JavaScript: var elems = $( "span.yt-uix-simple-thumb-wrap.yt-uix-simple-thumb-related:first" ); elems.html();.

    jquery issue.jpg

    In Qt with a QWebEngineView, when you do page()->toHtml(), you get the <img> tag before the source has been swapped using JavaScript. Similarly, when you load the jQuery library and then run:

    Qt Code:
    1. QString code = "var elems = qt.jQuery( 'span.yt-uix-simple-thumb-wrap.yt-uix-simple-thumb-related:first' ); elems.html();";
    2. webView->page()->runJavaScript(code, [&](const QVariant &v){ showResults(v.toString()); });
    To copy to clipboard, switch view to plain text mode 

    ... the variable v also contains the <img> tag before the source has been set.

    The question

    How does one view the final source code of such webpages, after all the JavaScript has been run, in the same way as the Chrome/Firefox dev tools?

  2. #2
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,416
    Thanks
    37
    Thanked 1,544 Times in 1,494 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Inspecting HTML elements in QWebEngine vs Chrome, Firefox, etc

    Maybe you are evaluating the toHtml() or runJavaScript() code too early as compared to when you are manually invoking a tool in the browsers.

    I.e. it could be more an issue of timing rather than web engine behavior.

    Cheers,
    _

  3. #3
    Join Date
    Jan 2017
    Posts
    3
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: Inspecting HTML elements in QWebEngine vs Chrome, Firefox, etc

    Hello, thank you for that suggestion, I've done some experiments and it turns out that you are correct.

    I am obviously waiting for the QWebEngineView's loadFinished signal before running any JavaScript. But my Qt browser widget is very small, so when the page loads these elements aren't visible to the user, because there isn't enough space to see them. When you scroll down and look at them, their <img src attribute gets set, and then inspecting them with JavaScript gives the same answer as in Chrome & Firefox.

    So now the question is: does one of the Qt classes have a method to force all elements on the page to be rendered, as if the user has scrolled down the page to look at them?

    I will have a look in the docs but if anyone knows the answer it would be much appreciated.

  4. #4
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,416
    Thanks
    37
    Thanked 1,544 Times in 1,494 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Inspecting HTML elements in QWebEngine vs Chrome, Firefox, etc

    Maybe if you are trying to "print" it, e.g. into a temporary PDF file.

    Cheers,
    _

  5. #5
    Join Date
    Jan 2017
    Posts
    3
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: Inspecting HTML elements in QWebEngine vs Chrome, Firefox, etc

    Here is what I've learned:
    - Off screen rendering is supported by the underlying chromium embedded framework but this has not been explicitly ported to Qt
    - A number of people have requested this functionality be added to QtWebEngine and coincidentally about two weeks ago it looks like they started work on it for Qt5.9 https://bugreports.qt.io/browse/QTBUG-44986
    - Various hacky solutions exist, such as scrolling down the page which forces it to be rendered, and resizing the QWebEngine widget so that it shows the whole page. I had no luck in getting scrolling to trigger the rendering, and wasn't able to test pdf printing because it would require an update of my Qt version. So I've ended up resizing the widget until some future release comes along which solves this officially:

    Qt Code:
    1. webView->page()->runJavaScript("document.documentElement.scrollWidth + '|' + document.documentElement.scrollHeight;", [=](QVariant result){
    2. std::string widthAndHeight = result.toString().toStdString();
    3. std::size_t idx = widthAndHeight.find("|");
    4. int newWidth = std::stoi(widthAndHeight.substr(0,idx));
    5. int newHeight = std::stoi(widthAndHeight.substr(idx+1));
    6. webView->resize(newWidth, newHeight);
    7. });
    To copy to clipboard, switch view to plain text mode 

Similar Threads

  1. Get Html element value with QWebEngine
    By danalex07 in forum Newbie
    Replies: 10
    Last Post: 5th February 2016, 04:18
  2. Replies: 4
    Last Post: 13th November 2015, 18:24
  3. Replies: 0
    Last Post: 8th July 2011, 00:43
  4. How to make custom menu for QWebView html elements ?
    By rsilva in forum Qt Programming
    Replies: 0
    Last Post: 10th May 2011, 01:18
  5. Qt debugging - inspecting properties?
    By gfunk in forum Qt Programming
    Replies: 6
    Last Post: 28th August 2006, 21:22

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.