Render 2d Density graph with OpenGL

[d_stranz]

As far as I understood the situation is about data that changes frequently, what would be the opposite of "keep it there". And I would expect, that preparing the data that can be uploaded to the GPU for each frame is not faster than creating the image itself - it requires the same steps beside of the final mapping of each pixel into a RGB value.

And a GPU might have more cores to do things in parallel, but beside that I don't see why it should be faster than the CPU for this situation.

Ah, my point was that if you can do the image calculation on the GPU side, then there is no need to move the data around. If you can't do that, then there may be no performance difference. The OP doesn't state whether his data is coming from an external source or is being computed.

One thing that could be faster if performed on the GPU is bilinear interpolation for resampling, and there are published algorithms for that.

[kkaz]

The data come from an external source, which means that the basic flow is the following:
A separate thread:
1. Wait for data from external device (More than 70 measurements per second)
2. Process data (This is of minor importance. ~2-3 ms)
3. Flush all data to disk without waiting to actually be flushed
3. Store the processed data in a 2d double array "ToRenderStructObject"

and the main GUI thread will paint the "toRenderStructObject" every once in a while, meaning that i am not losing painted data, but i don't paint them as they come. Painting requires to set the data to the SpectrogramData object that overrides the value() function, set some intervals, and call replot()

The resolution of the data is relatively high, 1080x500 in one occasion, and 2000x2000 in another occasion, and they are painted in rather small resolution widgets-graphs since they can't both fit in the screen.
I have not re implemented pixelHint(). Is that something that would help me?

My implementation of the value() function:

double Spectrogram::SpectrogramData::value(double x, double y) const { 
 
    /* X represents the column number in our data, and Y the row */
    double column = x;
    double row = y;
 
    QwtInterval interval_x = interval(Qt::XAxis);
    QwtInterval interval_y = interval(Qt::YAxis);
 
    if (x > interval_x.maxValue() || x < interval_x.minValue()) {
        /* This should not be happening */
        return std::numeric_limits<double>::quiet_NaN();
    }
    if ((y > interval_y.maxValue()) || (y < interval_y.minValue())) {
        /* This should not be happening */
        return std::numeric_limits<double>::quiet_NaN();
    }
 
    /* Linear interpolation to map the values to our data range */
    double index_x = 0 + (data_.size() - 1 - 0) * ((row - interval_y.minValue()) / (interval_y.maxValue() - interval_y.minValue()));
    double index_y = 0 + (data_.at(0).size() -1 - 0) * ((column - interval_x.minValue()) / (interval_x.maxValue() - interval_x.minValue()));
 
    if (index_x < 0 || index_x >= data_.size()) {
        dt::ConsoleInfoL(dt::CRITICAL, "Spectrogram rendering, index_x out of boundaries");
    }
    if (index_y < 0 || index_y >= data_.at(0).size()) {
        dt::ConsoleInfoL(dt::CRITICAL, "Spectrogram rendering, index_y out of boundaries");
    }
 
    /* Pick the nearest neighbour */
    return data_[std::round(index_x)][std::round(index_y)]; 
}

double Spectrogram::SpectrogramData::value(double x, double y) const { 

    /* X represents the column number in our data, and Y the row */
    double column = x;
    double row = y;
    
    QwtInterval interval_x = interval(Qt::XAxis);
    QwtInterval interval_y = interval(Qt::YAxis);
    
    if (x > interval_x.maxValue() || x < interval_x.minValue()) {
        /* This should not be happening */
        return std::numeric_limits<double>::quiet_NaN();
    }
    if ((y > interval_y.maxValue()) || (y < interval_y.minValue())) {
        /* This should not be happening */
        return std::numeric_limits<double>::quiet_NaN();
    }
    
    /* Linear interpolation to map the values to our data range */
    double index_x = 0 + (data_.size() - 1 - 0) * ((row - interval_y.minValue()) / (interval_y.maxValue() - interval_y.minValue()));
    double index_y = 0 + (data_.at(0).size() -1 - 0) * ((column - interval_x.minValue()) / (interval_x.maxValue() - interval_x.minValue()));

    if (index_x < 0 || index_x >= data_.size()) {
        dt::ConsoleInfoL(dt::CRITICAL, "Spectrogram rendering, index_x out of boundaries");
    }
    if (index_y < 0 || index_y >= data_.at(0).size()) {
        dt::ConsoleInfoL(dt::CRITICAL, "Spectrogram rendering, index_y out of boundaries");
    }

    /* Pick the nearest neighbour */
    return data_[std::round(index_x)][std::round(index_y)]; 
}

To copy to clipboard, switch view to plain text mode 

So the the question about CUDA was, if i squeeze into the separate thread a CUDA QImage building algorithm, and inside the main GUI thread paint the image into the Qframe, if it would be any faster. There is no required goal of refreshes per second that needs to be achieved, just wondering if i could squeeze anywhere a little gpu time.

[Uwe]

Store the processed data in a 2d double array "ToRenderStructObject"

• Could you please also show the definition of ToRenderStructObject and did you check how much time it takes to fill it ?
• What type of color map do you use ?
• Did you enable DEBUG_RENDER and what values do you see ?
• And are you using more than one core ?

The resolution of the data is relatively high, 1080x500 in one occasion, and 2000x2000 in another occasion, and they are painted in rather small resolution widgets-graphs since they can't both fit in the screen.
I have not re implemented pixelHint(). Is that something that would help me?

The image is rendered in the minimum of the resolutions of data/screen. But without implementing the pixelHint() the resolution of the data is not known and it will always end up in screen resolution.
Implementing a pixelHint helps, when the data resolution is below the resolution of the plot canvas ( = size in pixels ) - so in your case probably not.

My implementation of the value() function:

This method is called for every pixel - for a resolution of 1000x1000 this will be a million times and removing any pointless operation from it will have a significant effect.

• Better do these "will never happen" checks in debug mode only.
• Everything that can be calculated in advance should be done in YourRasterData::initRaster. F.e something like "(data_.size() - 1) * interval_y.maxValue() - interval_y.minValue()" is constant and does not need to be calculated again and again.
• Don't use std::round() - casting to int should be good enough.

But the implementation is not totally bad and accessing data_ might be the bottleneck.

Why don't you simply use QwtPlotMatrixData - or at least copy the code and strip it down to the minimum you need ? It is faster than your code and the main reasons for not using it ( calculate the values on the fly or returning values from the original buffer without having an extra copy ) seem to be irrelevant in your situation.

Concerning using the GPU: as long as you don't have an OpenGL canvas you would need to:

1. prepare data
2. upload data
3. create something with the GPU
4. download the result
5. translate it into an image
6. draw the image

Uwe

[kkaz]

In one case the the external source produces 1d vectors, so the ToRenderStruct is just a 2d vector with older measurements that i shift whenever a new one comes (using the vector assignment constructor), and in the other case the external source produces 2d array data, so i just use the assignment constructor to set the 2d vector. So the separate data gathering and processing thread always produces a 2d vector and the main gui thread renders that 2d vector. I have not measured how much time it takes to fill it, but it must be irrelevant to the rendering process, since it's done on the separate data gathering thread and this thread is able to achieve even 100 measurements per second.

The color map is use:

color_map_ = new QwtLinearColorMap(QColor(0, 0, 141), QColor(131, 0, 0));
color_map_->addColorStop(8.0 / 64.0, QColor(0, 0, 252));
color_map_->addColorStop(24.0 / 64.0, QColor(0, 254, 254));
color_map_->addColorStop(40.0 / 64.0, QColor(255, 252, 0));
color_map_->addColorStop(56.0 / 64.0, QColor(254, 0, 1)); 
...
QwtPlotSpectrogram * spectrogram = ...
...
spectrogram->setColorMap(...)

color_map_ = new QwtLinearColorMap(QColor(0, 0, 141), QColor(131, 0, 0));
color_map_->addColorStop(8.0 / 64.0, QColor(0, 0, 252));
color_map_->addColorStop(24.0 / 64.0, QColor(0, 254, 254));
color_map_->addColorStop(40.0 / 64.0, QColor(255, 252, 0));
color_map_->addColorStop(56.0 / 64.0, QColor(254, 0, 1)); 
...
QwtPlotSpectrogram * spectrogram = ...
...
spectrogram->setColorMap(...)

To copy to clipboard, switch view to plain text mode 

With DEBUG_RENDER i get, renderImage QSize(489, 193) 15, for a 2d vector with size 700*1280

I run on a i5 750 processor, but the final version will run on a i7 U laptop processor.

The main reason for not using QwtMatrixRasterData is that i have to use setValueMatrix() in order to set the data, which requires a QVector() with 1d configuration of the 2d array which provides a big copy overhead given that everything else in our code uses std vectors and 2d std vectors.

I think i'll leave the gpu for the moment, and stick to qwt plots, since it's more important to have accurate axes and names and stuff which would require significant amount of time to produce with a custom CUDA-created image. I will try to optimize value() with your suggestions.

[Uwe]

With DEBUG_RENDER i get, renderImage QSize(489, 193) 15

Considering, that this only for 489x193 pixels ( 5% of the number of pixels in my test ) this doesn't sound that good - so it might be worth to do optimizations for your value() method.

On the other hand you wrote, that you run into trouble with 10Hz, what indicates, that updates take >= 100ms. But considering, that you have 2 plots we would have 30ms only.
So there are > 70ms you lose somewhere else !

Uwe