I suggest you manipulate with QGraphicsView::ScrollHandDrag a bit. And in your event handlers change modes and let the events flow as usual - simply act depending on the current mode instead of ignoring events, that's not a good way to go.

Panning you'll have for free using aforementioned QGraphicsView::ScrollHandDrag. For zooming you'll have to implement a separate mode which you'll be handling in an event or slot that is called when a specific action you want zooming to be triggered with is performed (for instance wheelEvent is you want zooming to be triggered by the mouse wheel).