I don't know if there is any kind of library that supports the objects you need to draw. One of the most complete in number of features is WildMagic. It is an educational tool, so it supports a lot of things that are not really needed in the gaming world.

As for mouse manipulation, that bit is not so hard. I once created an open source 3d editor and I implemented all the mouse manipulation tools. Here are they on a QGLWidget:

http://sourceforge.net/p/sceneengine...GLViewport.cpp
http://sourceforge.net/p/sceneengine.../MouseView.cpp

Good luck!