I'm trying to render camera stream, the problem is camera isn't recognized and all the Qt machinery is useless.
What i have is a pointer `void* data` that can be accessed for curret frame.
The solution i've came up is to follow this tutorial:
here
only I'm using `QOpenGLTexture` filled with video data stream.
Redering a frame looks like this:
- paint() is called
- inside of paint function updateTexture is called, followed by texture->bind()
- After that shader is linked and quad is painted.
- texture->release;
Two important functions are below. updateTexture has also overloaded version that fills that gets the data from device and calls the "main" updateTexture.
Code:
void RsFrameProvider::updateTexture(const void* data, int width, int height, rs::format format, int stride = 0) { stride = stride == 0 ? width : stride; QOpenGLTexture::PixelType pixType; QOpenGLTexture::PixelFormat pixFormat; QOpenGLTexture::TextureFormat texFormat; switch(format) { case rs::format::any: throw std::runtime_error("not a valid format"); case rs::format::rgb8: case rs::format::bgr8: // Display both RGB and BGR by interpreting them RGB, to show the flipped byte ordering. Obviously, GL_BGR could be used on OpenGL 1.2+ texFormat = QOpenGLTexture::RGBFormat; pixFormat = QOpenGLTexture::RGB; pixType = QOpenGLTexture::UInt8; std::cerr << "Texture data set."; //glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, width, height, 0, GL_RGB, GL_UNSIGNED_BYTE, data); break; default: { std::stringstream ss; ss << rs_format_to_string((rs_format)format) << " RS: Pixel format is not supported"; throw std::runtime_error(ss.str().c_str()); } } if (!texture->isStorageAllocated()) { texture->setSize(width, height); texture->setFormat(texFormat); texture->allocateStorage(pixFormat, pixType); } texture->setData(pixFormat, pixType, data); std::cerr << "VALUE: " << ((uchar*) data); // Texture updated ready to bind std::cerr << "Texture ready to bind, FPS:" << fps << " pixFormat" << pixFormat << std::endl; }
I'm sure that each frame there is non-zero texture produced since I can print the data from device on the cerr stream.
Paint function:
Code:
void Renderer::paint() { frameProvider.updateTexture(); frameProvider.texture->bind(0); if (!m_program) { initializeOpenGLFunctions(); m_program = new QOpenGLShaderProgram(); m_program->addShaderFromSourceCode(QOpenGLShader::Vertex, "attribute highp vec4 vertices;" "varying highp vec2 coords;" "attribute highp vec4 texCoord;" "varying highp vec4 texc;" "void main() {" " gl_Position = vertices;" " coords = vertices.xy;" " texc = texCoord;" "}"); m_program->addShaderFromSourceCode(QOpenGLShader::Fragment, "uniform lowp float t;" "varying highp vec2 coords;" "varying highp vec4 texc;" "uniform sampler2D tex;" "void main() {" " lowp float i = 1. - (pow(abs(coords.x), 4.) + pow(abs(coords.y), 4.));" " i = smoothstep(t - 0.8, t + 0.8, i);" " i = floor(i * 20.) / 20.;" " highp vec3 color = texture2D(tex, texc.xy).rgb;" " gl_FragColor = vec4(color, 1.0);" "}"); m_program->bindAttributeLocation("vertices", 0); m_program->bindAttributeLocation("texCoord", 1); m_program->link(); } m_program->bind(); m_program->enableAttributeArray(0); m_program->enableAttributeArray(1); float values[] = { -1, -1, 1, -1, -1, 1, 1, 1 }; float texCoords[] = { 0, 0, 1, 0, 0, 1, 1, 1 }; m_program->setAttributeArray(0, GL_FLOAT, values, 2); m_program->setAttributeArray(1, GL_FLOAT, texCoords, 2 ); m_program->setUniformValue("t", (float) m_t); m_program->setUniformValue("tex", 0); glViewport(0, 0, m_viewportSize.width(), m_viewportSize.height()); glDisable(GL_DEPTH_TEST); glClearColor(0, 0, 0, 1); glClear(GL_COLOR_BUFFER_BIT); glEnable(GL_BLEND); glBlendFunc(GL_SRC_ALPHA, GL_ONE); glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); m_program->disableAttributeArray(0); m_program->disableAttributeArray(1); frameProvider.texture->release(); m_program->release(); // Not strictly needed for this example, but generally useful for when // mixing with raw OpenGL. m_window->resetOpenGLState(); }
All I get is a black screen.
If I use non-dynamic texture created from Image by QOpenGLTexture( url ...) binded the same way as dynamic one - I can see textured quad.
Is it even correct way to approach video rendering?
I want good performance and access to video frame data.
What is wrong?