Not that easy.
My solution is to use QListView instead. Create a custom item delegate and reimplement sizeHint and minimumSize for it. Next you must set it to the list view.
This way you will get your minimum size.

But this also means you will have to actually write some code .
Designer is just a tool of Qt, it is not meant to "create" fully functional applications with it.

Regards