Very high speed data acquisition with realtime chart update. How to manage this?

[asgard20032]

I got my first electrical engineering internship (altought I just did one semester since I got into that degree). Although I love programming, I usually do my programming in C on micro-controller or with VHDL or Verilog on FPGA. Never really worked with GUI and threading. Also, never really did true object oriented.

During my internship, I had to do some FPGA work. That FPGA side send around 2000 frame per second, each one containing 1024*32bits (I am using jumbo frame). I need to plot this data in real-time but also store it in a log. For the log part, a simple CSV will do. For the plotting, I am using QCustomPlots, with QCPColorMap to do an heat-map like. Vertical scale representing channel, horizontal scale representing time. And color representing the channel value at that time. My current implementation would work fine, if only it would be faster :

1: Read data from network
2: Insert new data frame into my frame vector containing 2000 frame. Require a whole vector rotation.
3: If my current frame counter == 20, update the graph(taking 1 frame each 20 frame) and reset counter (thus updating the graph each 20 frame, and drawing only 1 frame out of 20, so frame skipping, but also skipping some data when plotting)
4: increment frame counter
5: Log data to CSV

The problem is if drawing the chart take too much time, my program is too much busy to read the data from network, thus I will miss new data. Need some help for restructuring the whole thing. Need a simple implementation, that is fast enough. At the moment, it take more than 5 second to render 1 second worth of data.

Need suggestion.

[Sintegrial]

1. Put data acquisition into a separate thread.
2. Do not rotate the vector - use two pointers on frame start and frame end instead. These pointers will be incremented until end of the vector is reached. Then just set the pointer to 0 and repeat again.
3. Run a QTImer in the GUI thread and do UI updates only when timeout() signal comes. Just render the data between the two pointers.