If you scale the canvas in milimeters, the font size will also have to be given in milimeters but scaled with a factor matching the one between the window and the viewport (in other words you have to calculate how many "pixels" or "points" fit into one "milimeter").
Bookmarks