Results 1 to 2 of 2

Thread: set float precision

  1. #1
    Join Date
    Feb 2012
    Location
    Armenia/Yerevan
    Posts
    400
    Thanks
    15
    Thanked 16 Times in 15 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11

    Default set float precision

    application outpus 1.02122e-38
    to get it to 1.02 I added
    Qt Code:
    1. lcd.display(QString::number(digit, 'f', 2));
    To copy to clipboard, switch view to plain text mode 

    unfortunately, now it outputs 00.0

    what am I missing?

  2. #2
    Join Date
    Mar 2008
    Location
    Kraków, Poland
    Posts
    1,536
    Thanked 284 Times in 279 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: set float precision

    You are missing that 1.02122e-38 is very small number. It is almost 0 not 1.02.

Similar Threads

  1. precision of float field in qtableview
    By mandal in forum Newbie
    Replies: 4
    Last Post: 25th March 2020, 01:23
  2. Multi defined QVector<float>::QVector<float>
    By lni in forum Qt Programming
    Replies: 1
    Last Post: 6th May 2013, 08:58
  3. Replies: 4
    Last Post: 18th May 2011, 18:59
  4. QPolygonF precision?! is it possible?!
    By fatecasino in forum Newbie
    Replies: 11
    Last Post: 27th January 2011, 03:36
  5. Replies: 1
    Last Post: 10th February 2009, 10:42

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.