Results 1 to 9 of 9

Thread: 30 fps desired but getting 30 frames every ~21 seconds.

  1. #1
    Join Date
    Jul 2016
    Posts
    18
    Thanks
    3
    Qt products
    Qt5 Qt/Embedded
    Platforms
    Unix/X11

    Default 30 fps desired but getting 30 frames every ~21 seconds.

    Hi All and thank you for reading this.

    I have a MainWindow with three widgets (QWidget, QDialog and QOpenGLWidget). The QWidget is a navigation bar at the bottom and the QDiaglog is for user input to later serial communitcation. Neither of these are doing anyting right now as it is the early stags of the project. I setup the structure and visual elements with no event handling for now. My main focus is the QOpenGLWidget as this is the view of one of many future cameras. I put in a frame counter in the paintGL () that once it hits 30 frames, give me a duration. Every 30 frames takes roughly 21 seconds. This is very unacceptable.

    As background, I have setup the default format as such in main...

    Qt Code:
    1. QSurfaceFormat format;
    2. format.setProfile ( QSurfaceFormat::CoreProfile );
    3. format.setRenderableType ( QSurfaceFormat::OpenGLES );
    4. format.setSwapBehavior ( QSurfaceFormat::TripleBuffer );
    5. format.setSwapInterval ( 0 ); // 0 = draw to vsync
    6. format.setVersion ( 3, 0 ); // imx6 vivante revision 5.0.11
    7. // must be called before the widget
    8. // or its parent window gets shown
    9. QSurfaceFormat::setDefaultFormat ( format );
    To copy to clipboard, switch view to plain text mode 

    I then setup my subclassed mainwindow through my own initialization function to propagate the items before show(). At the end of this, all items at created and initialized. They have their geomotry, everything is connected and we're ready to to be shown. show() then starts the visual loops and off we go. In the QOpenGLWidget, as standard, I follow the standard three overrides. Here is the top of my class in question...

    Qt Code:
    1. class AVC8000nano : public QOpenGLWidget, protected QOpenGLFunctions
    2. {
    3. Q_OBJECT
    4.  
    5. public:
    6. explicit AVC8000nano ( QWidget *parent = 0 );
    7. ~AVC8000nano ( void );
    8.  
    9. APP_STATUS Initialize ( WhichSetup* structWhichSetup );
    10. void DoWork ( WORKTODO myTask );
    11. bool SetCameraToViewPort ( const WHICH_CAMERA eCamera );
    12.  
    13. protected:
    14. void initializeGL () Q_DECL_OVERRIDE;
    15. void paintGL () Q_DECL_OVERRIDE;
    16. void resizeGL ( int width, int height );
    17. QOpenGLExtraFunctions* m_OGLES3Functions;
    18. ...
    To copy to clipboard, switch view to plain text mode 

    Inside of paintGL(), I do very little...

    Qt Code:
    1. void AVC8000nano::paintGL ()
    2. {
    3. if ( ( m_bAreWeExitting ) ||
    4. ( !m_bInitialized ) )
    5. {
    6. return;
    7. }
    8.  
    9. QOpenGLContext *oglContext = context ();
    10. if ( !oglContext ) // QOpenGLWidget not yet initialized
    11. {
    12. return;
    13. }
    14.  
    15. //GLenum enumError;
    16.  
    17. m_qimage2Paint = m_helperAVC8KV4L2->GrabImage ();
    18. if ( m_qimage2Paint != NULL )
    19. {
    20. QPainter qPainter ( this );
    21. qPainter.translate ( m_nDeltaX / 2, m_nDeltaY / 2 );
    22. qPainter.drawImage ( m_rectImage, *m_qimage2Paint );
    23.  
    24. // if ( ( enumError = m_OGLES3Functions->glGetError () ) != GL_NO_ERROR )
    25. // {
    26. // qDebug ( "OpenGL error: %d.", enumError );
    27. // }
    28. }
    29.  
    30.  
    31. #ifdef DEBUG_LIVE_TIMING
    32. m_nFrameGrabbed++;
    33. if ( m_nFrameGrabbed == 30 )
    34. {
    35. gettimeofday ( &m_tv_now, NULL );
    36. m_tk_Diff = TV_SECONDS ( m_tv_now, m_tv_begin );
    37. qDebug ( "30 frames took %lld seconds", m_tk_Diff, m_nFrameGrabbed );
    38. m_nFrameGrabbed = 0;
    39. gettimeofday ( &m_tv_begin, NULL );
    40. }
    41. #endif
    42.  
    43. delete m_qimage2Paint;
    44.  
    45. // Schedule composition. Note that this will use QueuedConnection, meaning
    46. // that update() will be invoked on the gui thread.
    47. QMetaObject::invokeMethod ( this, "update");
    48. //update ( m_rectWidget );
    49. }
    To copy to clipboard, switch view to plain text mode 

    I have placed inside of my /etc/appcontroller.conf file as well as my /etc/profile (appropriately edited) these lines...

    Qt Code:
    1. env=QT_QPA_PLATFORM=eglfs
    2. env=QT_QPA_PLATFORM_PLUGIN_PATH=/usr/lib/plugins/platforms
    3. env=QT_QPA_EGLFS_WIDTH=1920
    4. env=QT_QPA_EGLFS_HEIGHT=1080
    5. env=QT_QPA_EGLFS_PHYSICAL_WIDTH=1920
    6. env=QT_QPA_EGLFS_PHYSICAL_HEIGHT=1080
    7. env=QT_QPA_EGLFS_SWAPINTERVAL=0
    To copy to clipboard, switch view to plain text mode 

    My main point here is the SWAPINTERVAL matching the applications format.setSwapInterval above. I have tested setting these values to 34, 17, 1 and 0. None of these effect the 30 frames in 21 seconds measurement.

    For uboot, I set the video args as such for the kernel. Changing the @60 to @30 again makes no diff.

    vidargs=acpi=force mxc_hdmi.only_cea=1 video=mxcfb0:dev=hdmi,1920x1080M@60,if=RGB24 video=mxcfb1ff video=mxcfb2ff video=mxcfb3ff fbmem=32M

    So here I am now not sure where to look. Does anyone have any ideas as to where to look next? Thank you for your time.

    Cheers,
    Pete

  2. #2
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,363
    Thanks
    3
    Thanked 5,012 Times in 4,791 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: 30 fps desired but getting 30 frames every ~21 seconds.

    What is the point of using OpenGL if the only thing you do is display a 2d image?

    How much time does the following call take?

    Qt Code:
    1. m_qimage2Paint = m_helperAVC8KV4L2->GrabImage ();
    To copy to clipboard, switch view to plain text mode 
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  3. The following user says thank you to wysota for this useful post:

    PeterVE (20th September 2016)

  4. #3
    Join Date
    Jul 2016
    Posts
    18
    Thanks
    3
    Qt products
    Qt5 Qt/Embedded
    Platforms
    Unix/X11

    Default Re: 30 fps desired but getting 30 frames every ~21 seconds.

    Thank you wysota for the reply. The project is early on and it has a requirement to draw text on top of the image as well as image processing algorithms. Also, the target will be handling between 4-12 media streams at the same time and constant and forever from startup to shutdown. This has to be optimized for performance. Even now, I don't feel comfortable using QImage and QPaint. Anyways, your second question is exactly where I went right after posting this. It is in fact that the device is not providing an image as the product advertisements claim. I put the select() call on a loop to examine the issue and it is timing out with a 1s timeout. I am reviewing my port of their example as I type this and so far, I did find one missed line which is the setting of the resolution. I have also contacted the manufacturer's tech support to discuss possible things I have missed or is my expectations wrong. I also found that the adverts state they support RGB565 which is the closest to the default RGB8888 (RGB24) which is the internal of Qt as well as OpenGL. I found a readme in their driver that stated they only support YUYV. So I'm losing confidence in this device right now. I have done format conversions in vertex and fragment shaders as well as added an alpha channel. I'm not lost, but just debating on hardware at this point.

    No matter if I messed up or the product is sub-par, this thread is dead. Thank you again for the reply.

  5. #4
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,363
    Thanks
    3
    Thanked 5,012 Times in 4,791 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: 30 fps desired but getting 30 frames every ~21 seconds.

    Quote Originally Posted by PeterVE View Post
    Thank you wysota for the reply. The project is early on and it has a requirement to draw text on top of the image as well as image processing algorithms. Also, the target will be handling between 4-12 media streams at the same time and constant and forever from startup to shutdown. This has to be optimized for performance.
    That's not a reason to use OpenGL.

    Even now, I don't feel comfortable using QImage and QPaint. Anyways, your second question is exactly where I went right after posting this. It is in fact that the device is not providing an image as the product advertisements claim. I put the select() call on a loop to examine the issue and it is timing out with a 1s timeout.
    Grabbing frames and converting them to images is usually a very lengthy process. It's much quicker to make the device render content directly where you want it.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  6. #5
    Join Date
    Jul 2016
    Posts
    18
    Thanks
    3
    Qt products
    Qt5 Qt/Embedded
    Platforms
    Unix/X11

    Default Re: 30 fps desired but getting 30 frames every ~21 seconds.

    To be clear, I am using QOpenGL and not currently native OpenGL. The reasoning is because there are proven paths on the imx6 that a gpu loaded image can in fact have zero copies to the screen. They used QOpenGL to do so. Granted, I am not opengl native yet and am working out all of the easier paths right now, but it seems logical to follow what others have done. It is Qt's own website that pushed this info out to us. Also, shaders are (until we end this thread and am convinced otherwise) required. Not sure how you can write shaders and not be OpenGL within Qt. At least with the easiest path of hooks. Image processing as stated above means more than just displaying it to a screen. I am using 2D images to move robotics in 3d space and some of it automated for the user as well as algorithms to perform automated tedious tasks such as counting thousands of objects and sizes and a lot more. All of which I have done on DirectX inside of shaders and not on the CPU because of the floating point abilities. The GPU is more than just for rendering images, but for image processing as well. As a newb to this Qt and linux GPU programming, I am fully open to design options that will work with Qt as the GUI environment though.

    As a newb to Qt, I haven't seen and am not knowledgeable on best practices. I am assuming you are referring to QPaintDevice from the Widget itself to render directly to where I want it (?). I cannot see the path as to take a char buff[] directly to QPaintDevice as I believe you may be saying. I would greatly appreciate advise to a better performance path. Thank you.

  7. #6
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,363
    Thanks
    3
    Thanked 5,012 Times in 4,791 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: 30 fps desired but getting 30 frames every ~21 seconds.

    Quote Originally Posted by PeterVE View Post
    To be clear, I am using QOpenGL and not currently native OpenGL.
    There is no "native" and "non-native" OpenGL. QOpenGL calls such as your drawImage call are translated into regular OpenGL calls.

    The reasoning is because there are proven paths on the imx6 that a gpu loaded image can in fact have zero copies to the screen.
    Not if you draw QImage which is a purely software entity. Then you have at least two copy operations - first to obtain the data for QImage on the CPU and then to make a texture usable for a GPU out of it.

    Not sure how you can write shaders and not be OpenGL within Qt.
    OpenGL is an API. You can use shaders to render to an offscreen surface (i.e. an FBO) which can then be converted to a QImage and used without OpenGL. So having an OpenGL based widget is not a requirement. But that's irrelevant here. What's relevant is you are not leveraging any of OpenGL capabilities in the code you posted in this thread. A lot of conversions are made and that is why you have such poor performance.

    Image processing as stated above means more than just displaying it to a screen. I am using 2D images to move robotics in 3d space and some of it automated for the user as well as algorithms to perform automated tedious tasks such as counting thousands of objects and sizes and a lot more. All of which I have done on DirectX inside of shaders and not on the CPU because of the floating point abilities. The GPU is more than just for rendering images, but for image processing as well. As a newb to this Qt and linux GPU programming, I am fully open to design options that will work with Qt as the GUI environment though.
    This does not require you to use OpenGL widget for rendering the result to screen. You can of course have an OpenGL widget to render a texture you had earlier rendered to using shader computation but certainly not the way you have used in the code above.

    I have successfully used QtQuick with I.MX6, so assuming you can implement a QtQuick item that takes an existing GPU buffer and treats it as a texture for the item, that would give the best performance.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  8. #7
    Join Date
    Jul 2016
    Posts
    18
    Thanks
    3
    Qt products
    Qt5 Qt/Embedded
    Platforms
    Unix/X11

    Default Re: 30 fps desired but getting 30 frames every ~21 seconds.

    The qopengl i did is saved in another folder. This thread is not the full project. Yup, this post doesn't show the usage of opengl but on purpose. The full blown usage is not a part of the test/post. Simple tests show building blocks work. This flushed out that the camera hardware/driver is sub-par and won't work for us. Anyways, QtQuick seems to be your suggestion and I will look into it. Thanks for your time.
    Last edited by PeterVE; 22nd September 2016 at 15:43.

  9. #8
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,363
    Thanks
    3
    Thanked 5,012 Times in 4,791 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: 30 fps desired but getting 30 frames every ~21 seconds.

    Quote Originally Posted by PeterVE View Post
    This flushed out that the camera hardware/driver is sub-par and won't work for us.
    How did you test that? Because the code posted in this thread is not an adequate test.

    Anyways, QtQuick seems to be your suggestion and I will look into it.
    I said it might work for you, if you put some effort of building efficient custom items. I'm not suggesting you should use it, because I don't know the whole picture of what you are doing.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  10. #9
    Join Date
    Oct 2016
    Posts
    8
    Qt products
    Qt4 Qt/Embedded
    Platforms
    Windows Android

    Default Re: 30 fps desired but getting 30 frames every ~21 seconds.

    That's not a reason to use OpenGL.

Similar Threads

  1. Forcing desired space between two rectangles in GridLayout
    By TheIndependentAquarius in forum Qt Quick
    Replies: 2
    Last Post: 3rd May 2016, 07:30
  2. How to add mouse area at a desired frame
    By athulms in forum Qt Programming
    Replies: 1
    Last Post: 17th August 2011, 17:26
  3. How to add lines to desired frame only
    By athulms in forum Qt Programming
    Replies: 1
    Last Post: 16th August 2011, 08:43
  4. Replies: 6
    Last Post: 5th December 2009, 14:46
  5. failing to achieve desired size of a BitField structure
    By nass in forum General Programming
    Replies: 8
    Last Post: 13th February 2007, 13:29

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.