Dynamic QOpenGLTexture with arbitrary data

[pietrko]

I'm trying to render camera stream, the problem is camera isn't recognized and all the Qt machinery is useless.
What i have is a pointer `void* data` that can be accessed for curret frame.
The solution i've came up is to follow this tutorial:
here (http://doc.qt.io/qt-5/qtquick-scenegraph-openglunderqml-example.html)
only I'm using `QOpenGLTexture` filled with video data stream.

Redering a frame looks like this:

paint() is called
inside of paint function updateTexture is called, followed by texture->bind()
After that shader is linked and quad is painted.
texture->release;


Two important functions are below. updateTexture has also overloaded version that fills that gets the data from device and calls the "main" updateTexture.


void RsFrameProvider::updateTexture(const void* data, int width, int height, rs::format format, int stride = 0) {
stride = stride == 0 ? width : stride;

QOpenGLTexture::PixelType pixType;
QOpenGLTexture::PixelFormat pixFormat;
QOpenGLTexture::TextureFormat texFormat;

switch(format)
{
case rs::format::any:
throw std::runtime_error("not a valid format");
case rs::format::rgb8: case rs::format::bgr8: // Display both RGB and BGR by interpreting them RGB, to show the flipped byte ordering. Obviously, GL_BGR could be used on OpenGL 1.2+
texFormat = QOpenGLTexture::RGBFormat;
pixFormat = QOpenGLTexture::RGB;
pixType = QOpenGLTexture::UInt8;
std::cerr << "Texture data set.";
//glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, width, height, 0, GL_RGB, GL_UNSIGNED_BYTE, data);
break;

default:
{
std::stringstream ss;
ss << rs_format_to_string((rs_format)format) << " RS: Pixel format is not supported";
throw std::runtime_error(ss.str().c_str());
}
}

if (!texture->isStorageAllocated()) {
texture->setSize(width, height);
texture->setFormat(texFormat);
texture->allocateStorage(pixFormat, pixType);
}
texture->setData(pixFormat, pixType, data);

std::cerr << "VALUE: " << ((uchar*) data);
// Texture updated ready to bind
std::cerr << "Texture ready to bind, FPS:" << fps << " pixFormat" << pixFormat << std::endl;
}


I'm sure that each frame there is non-zero texture produced since I can print the data from device on the cerr stream.

Paint function:



void Renderer::paint()
{
frameProvider.updateTexture();
frameProvider.texture->bind(0);


if (!m_program) {
initializeOpenGLFunctions();

m_program = new QOpenGLShaderProgram();
m_program->addShaderFromSourceCode(QOpenGLShader::Vertex,
"attribute highp vec4 vertices;"
"varying highp vec2 coords;"
"attribute highp vec4 texCoord;"
"varying highp vec4 texc;"
"void main() {"
" gl_Position = vertices;"
" coords = vertices.xy;"
" texc = texCoord;"
"}");
m_program->addShaderFromSourceCode(QOpenGLShader::Fragment,
"uniform lowp float t;"
"varying highp vec2 coords;"

"varying highp vec4 texc;"
"uniform sampler2D tex;"
"void main() {"
" lowp float i = 1. - (pow(abs(coords.x), 4.) + pow(abs(coords.y), 4.));"
" i = smoothstep(t - 0.8, t + 0.8, i);"
" i = floor(i * 20.) / 20.;"
" highp vec3 color = texture2D(tex, texc.xy).rgb;"
" gl_FragColor = vec4(color, 1.0);"
"}");

m_program->bindAttributeLocation("vertices", 0);
m_program->bindAttributeLocation("texCoord", 1);

m_program->link();

}
m_program->bind();

m_program->enableAttributeArray(0);
m_program->enableAttributeArray(1);
float values[] = {
-1, -1,
1, -1,
-1, 1,
1, 1
};
float texCoords[] = {
0, 0,
1, 0,
0, 1,
1, 1
};
m_program->setAttributeArray(0, GL_FLOAT, values, 2);
m_program->setAttributeArray(1, GL_FLOAT, texCoords, 2 );

m_program->setUniformValue("t", (float) m_t);
m_program->setUniformValue("tex", 0);

glViewport(0, 0, m_viewportSize.width(), m_viewportSize.height());

glDisable(GL_DEPTH_TEST);

glClearColor(0, 0, 0, 1);
glClear(GL_COLOR_BUFFER_BIT);

glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE);

glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

m_program->disableAttributeArray(0);
m_program->disableAttributeArray(1);


frameProvider.texture->release();
m_program->release();

// Not strictly needed for this example, but generally useful for when
// mixing with raw OpenGL.
m_window->resetOpenGLState();

}

All I get is a black screen.
If I use non-dynamic texture created from Image by QOpenGLTexture( url ...) binded the same way as dynamic one - I can see textured quad.

Is it even correct way to approach video rendering?
I want good performance and access to video frame data.
What is wrong?