Results 1 to 3 of 3

Thread: <double> has very low resolution

  1. #1
    Join Date
    Sep 2011
    Posts
    9
    Thanks
    1
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Post <double> has very low resolution

    After several hours browsing the forums and rebuilding a dozen times with qDebugs, it's time to ask.
    All my <double> values appear with two decimal points only. This is a plain XP system, latest SDK, US English. No currency is involved.

    A clear example is comparing a new <double> to the value in a QDoubleSpinBox with 3 decimals.
    Their QString representations are
    "100.001" and "100.002"
    Their doubles are
    100.0 and 100.0
    The outcomes are consistent using any Qt conversion method and Qt display method, including direct output from qDebug(), QString.toDouble(), QLocale, QVariant, etc. (I said, I've been searching!)
    Other test values? "123.456" is converted to 123.46 -- looks like rounding.
    This seems like a simple environment setting that I've missed. Where should I look?
    Thanks, Scott

  2. #2
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,359
    Thanks
    3
    Thanked 5,015 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: <double> has very low resolution

    The "rounding" sometimes takes place during output to qDebug(), the values are fine until you cast then to float. By the way, "double" is a C type, not a Qt type.

    Both these work fine:

    Qt Code:
    1. #include <QtGui>
    2.  
    3. class DSB : public QDoubleSpinBox {
    4. Q_OBJECT
    5. public:
    6. DSB() : QDoubleSpinBox() {
    7. connect(this, SIGNAL(valueChanged(double)), this, SLOT(display(double)));
    8. setDecimals(4);
    9. }
    10. public slots:
    11. void display(double val) {
    12. qDebug() << "direct:" << val;
    13. qDebug() << "transformed:" << val*1000;
    14. }
    15. };
    16.  
    17. #include "main.moc"
    18.  
    19. int main(int argc, char **argv) {
    20. QApplication app(argc, argv);
    21. DSB sb;
    22. sb.show();
    23. return app.exec();
    24. };
    To copy to clipboard, switch view to plain text mode 
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  3. #3
    Join Date
    Sep 2011
    Posts
    9
    Thanks
    1
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: <double> has very low resolution

    Thanks, wysota.
    I noticed that your example uses only 4 decimals, indeed, more than was shown in my post but not enough for my actual app. In several cases there are *1000 and /1000 operators between doubles and QString.setNum.

    That did get me thinking...a little more experimentation revealed a combination of my error and the default behavior of qDebug.
    setNum() without parameters #2 and #3 defaults to 'g', 6.
    Scott assumed the default '6' was decimals. Yes, format 'f' is 6 decimals, format 'g' is 6 total digits.
    Qt apparently uses that default ('g', 6) when executing double conversions in qDebug. That's logical, I suppose.

    Here's part of the test and results...

    Qt Code:
    1. // expanded testing inside the app
    2. ui->SpinBox->setValue(7777.777);
    3. sz1 = "7777.777";
    4. dbl1 = sz1.toDouble();
    5. dbl2 = ui->SpinBox->value();
    6.  
    7. sz2.setNum(dbl1);
    8. sz3.setNum(dbl2);
    9. sz4.setNum(dbl1, 'g', 6);
    10. sz5.setNum(dbl1, 'g', 8);
    11. qDebug() << dbl1 << dbl2 << sz1 << sz2 << sz3 << sz4 << sz5;
    12.  
    13. // qDebug output:
    14. // 7777.78 7777.78 "7777.777" "7777.78" "7777.78" "7777.78" "7777.777"
    To copy to clipboard, switch view to plain text mode 

    Now try to "break" your example. ( Um...uh...but it's a really gooood example !! )

    Qt Code:
    1. // added
    2. setSingleStep(33.0001); // large steps, and make decimals visible
    3. setMaximum(10000000); // default is 100
    4.  
    5. // qDebug output - note the disappearing decimal digits
    6. // direct: 99.0003
    7. // transformed: 99000.3
    8. // direct: 132
    9. // transformed: 132000
    10. // direct: 165.001
    11. // transformed: 165001
    To copy to clipboard, switch view to plain text mode 

    I'll be a little wary of qDebug. Of course it needs to run as fast as possible and there are tradeoffs in there.
    Hope this helps someone else.
    Cheers,
    Scott

Similar Threads

  1. Program not finding an existing double in a QList<double>
    By aarelovich in forum Qt Programming
    Replies: 2
    Last Post: 9th May 2011, 20:59
  2. Resolution Spectrogram
    By edney in forum Qwt
    Replies: 3
    Last Post: 13th April 2010, 07:44
  3. Replies: 2
    Last Post: 24th June 2009, 15:38
  4. how to scale according to the used resolution
    By rmagro in forum Qt Programming
    Replies: 1
    Last Post: 3rd July 2007, 10:43

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.