I don't understand the (syntactical) problem, why can't you access it via a pointer as your subject states?
But I would say that would be a wrong way of doing it (design-wise) since you are decreasing the servers member in another class.
It would be better to connect a slot that belongs to the server to the disconnected() signal, and do it there.
Bookmarks