PDA

View Full Version : <double> has very low resolution



ScottBell
13th October 2011, 19:18
After several hours browsing the forums and rebuilding a dozen times with qDebugs, it's time to ask.
All my <double> values appear with two decimal points only. This is a plain XP system, latest SDK, US English. No currency is involved.

A clear example is comparing a new <double> to the value in a QDoubleSpinBox with 3 decimals.
Their QString representations are
"100.001" and "100.002"
Their doubles are
100.0 and 100.0
The outcomes are consistent using any Qt conversion method and Qt display method, including direct output from qDebug(), QString.toDouble(), QLocale, QVariant, etc. (I said, I've been searching!)
Other test values? "123.456" is converted to 123.46 -- looks like rounding.
This seems like a simple environment setting that I've missed. Where should I look?
Thanks, Scott

wysota
13th October 2011, 21:02
The "rounding" sometimes takes place during output to qDebug(), the values are fine until you cast then to float. By the way, "double" is a C type, not a Qt type.

Both these work fine:


#include <QtGui>

class DSB : public QDoubleSpinBox {
Q_OBJECT
public:
DSB() : QDoubleSpinBox() {
connect(this, SIGNAL(valueChanged(double)), this, SLOT(display(double)));
setDecimals(4);
}
public slots:
void display(double val) {
qDebug() << "direct:" << val;
qDebug() << "transformed:" << val*1000;
}
};

#include "main.moc"

int main(int argc, char **argv) {
QApplication app(argc, argv);
DSB sb;
sb.show();
return app.exec();
};

ScottBell
14th October 2011, 14:44
Thanks, wysota.
I noticed that your example uses only 4 decimals, indeed, more than was shown in my post but not enough for my actual app. In several cases there are *1000 and /1000 operators between doubles and QString.setNum.

That did get me thinking...a little more experimentation revealed a combination of my error and the default behavior of qDebug.
setNum() without parameters #2 and #3 defaults to 'g', 6.
Scott assumed the default '6' was decimals. Yes, format 'f' is 6 decimals, format 'g' is 6 total digits.
Qt apparently uses that default ('g', 6) when executing double conversions in qDebug. That's logical, I suppose.

Here's part of the test and results...



// expanded testing inside the app
ui->SpinBox->setValue(7777.777);
sz1 = "7777.777";
dbl1 = sz1.toDouble();
dbl2 = ui->SpinBox->value();

sz2.setNum(dbl1);
sz3.setNum(dbl2);
sz4.setNum(dbl1, 'g', 6);
sz5.setNum(dbl1, 'g', 8);
qDebug() << dbl1 << dbl2 << sz1 << sz2 << sz3 << sz4 << sz5;

// qDebug output:
// 7777.78 7777.78 "7777.777" "7777.78" "7777.78" "7777.78" "7777.777"


Now try to "break" your example. ( Um...uh...but it's a really gooood example !! )



// added
setSingleStep(33.0001); // large steps, and make decimals visible
setMaximum(10000000); // default is 100

// qDebug output - note the disappearing decimal digits
// direct: 99.0003
// transformed: 99000.3
// direct: 132
// transformed: 132000
// direct: 165.001
// transformed: 165001


I'll be a little wary of qDebug. Of course it needs to run as fast as possible and there are tradeoffs in there.
Hope this helps someone else.
Cheers,
Scott