Thank you!
Got it working, I was trying to be too clever in the zoomIn and -Out methods. A plain call to scale(factor, factor) did it.
For the next step I may have to start over. The labels for the axis must be visible at all times, so in effect only the grid lines must move on zoom and pan. I've thought about subclassing QGraphicsTextItem for the X and Y labels, and QGraphicsItem for the gird. The current GridScene class then not needed anymore as GridScene::drawBackGround method will be done in the subclassed items, using QGraphicsAnchorLayout to keep the three items in their proper places in the GridView. Not sure how to proceed though. Forward mouse, and button events to the item subclasses? Is this the best way to do it?