QwtPanner grabs the content of the canvas into a pixmap and moves the pixmap only. This is done for performance reasons, because rendering the plot items might be too slow for mouse movements. If you don't want this - don't use the panner.

Instead catch all mouse events of the plot canvas using an event filter. Translate the mouse movements into coordinates of the plot using the canvas maps ( QwtPlot::canvasMap ) and adjust the scales.