PDA

View Full Version : QtTest and asserts



doberkofler
17th September 2010, 14:25
I was wondering how to deal with asserts fired from a function that is tested in a QTest module. Imagine, you want to test a method bool xyz() that internally uses a precondition that might fire if something is inconsistent. QCOMPARE(xyz(), true) would never be executed because xyz() already exists with an assert.

Is this simply not supported in the Qt test environment, does it need to somehow managed manually (using some global semaphore for example) or did I just not get it?

lyuts
17th September 2010, 15:38
I think that a function whose assert fires is not working properly and is not ready for testing at all. Besides, there are not so many cases when you should keep asserts in your production code.

doberkofler
17th September 2010, 16:03
I get the point but:
- I would actually also want to test if the pre-conditions, post-conditions, invariants and asserts are working as supposed in my test
- I'm very much convinced (and 20 years of C++ experience never proved me wrong) that assertions are even more important in production code then in debug version

lyuts
18th September 2010, 09:17
Could you post some sample code of such function, demonstrating what you are trying to do?

doberkofler
20th September 2010, 06:47
The following example should show what I'm talking about. my_assert would be my own assert macro that usually logs the error and reports it to the user before aborting the application.

I was now wondering, how to test a function that is supposed to fail (by asserting or raising an exception) in QtTest?



#include <QtTest/QtTest>

static float myFunction(float value)
{
my_assert(value > 0.0); // check precondition
return 2 / value;
}

class tst_QDate : public QObject {
Q_OBJECT
public:
tst_myClass() {}
virtual ~tst_myClass() {}
public slots:
void init() {}
void cleanup() {}
private slots:
void myClass();
};

void tst_QDate::myClass()
{
QCOMPARE(float(1), myFunction(2.0));

// Next QCOMPARE should fail (e.g. raise an exception)
QCOMPARE(float(1), myFunction(0.0));
}

QTEST_APPLESS_MAIN(tst_myClass)

#include "moc_tst_myClass.cxx"

lyuts
20th September 2010, 08:51
How is it better than:


static float myFunction(float value)
{
if (value <= 0.0) // check precondition
{
return eNAN;
}
return 2 / value;
}

?

I realize that after your function is completely polished you can remove your assert and doing "if" will not take time. But, I think that your function should be error-resistable, since it gets value inputted by someone. Your function should take care of exceptional/errorneous situation (in your case, it is division by zero).
Lets pretend that your app reads numbers from some file and does divisions for them. This file is created by somebody who put 0 in that file.
Way 1: Your application is crashed by your assert only because someone was not careful enough and put 0 in that file.
Way 2: On processing that division by 0, your application returns NAN value and reports that wrong arguments have been specified.

What would you choose? I think that the 2nd way is more correct.

doberkofler
20th September 2010, 09:43
We seem to be moving a little away from my actual question but I'm very interested in this more generic topic as well. My answer to your question would (as always) be: IT DEPENDS!
It depends on what the contract between invoking function and the invoked one specifies and this can be to either not to care at all (most common), return a status and leave it to the caller to decide (your suggestion) or impose a strict rule of parameter passing using a precondition (this is what I wanted to do).

A more realistic (longer) version of myClass::myFunction would typically look like this:


float myClass::myFunction(float value)
{
/* test consistency of class */
my_class_invariant(...);

/*test consistency of parameter */
my_precondition(value > 0, "you are using an invalid value=("+toString(value)+")");

/* usually the algorithms are "little" more complex */
float temp = 10.0 / value;

/* test consistency of intermediate result */
my_assert(temp > 0.0 && ...);

/* test consistency of result */
my_postcondition(fuzzy_compare(temp * value, 10.0);

return temp;
}

wysota
20th September 2010, 10:25
- I'm very much convinced (and 20 years of C++ experience never proved me wrong) that assertions are even more important in production code then in debug version
If an assert triggers during tests it means it will fire in the production code. All asserts should be removed prior to testing (which is easy - just build your app in release mode). If you still want to test the conditions, substitute asserts with regular checks and exit the function gracefully.

If you insist on asserts not killing your application, redefine the assert/Q_ASSERT macro to do something else than abort the process.

doberkofler
20th September 2010, 13:51
If an assert triggers during tests it means it will fire in the production code. All asserts should be removed prior to testing (which is easy - just build your app in release mode). If you still want to test the conditions, substitute asserts with regular checks and exit the function gracefully.

As I explained in my reply to lyuts, I very strongly believe that there is no rule or best practice on how to deal with assertions. It really depends and I have made some very positive experience over a long period of time in keeping all asserts in the production code.


If you insist on asserts not killing your application, redefine the assert/Q_ASSERT macro to do something else than abort the process.

Thank you for making this clear. I just wanted to make sure, that there is no explicit functionality in QtTest that would intercept an exception instead of doing this myself.

wysota
20th September 2010, 14:17
As I explained in my reply to lyuts, I very strongly believe that there is no rule or best practice on how to deal with assertions.
Assert, by definition, kills your app. If you want an assert that will not kill your app but instead will return an error, it will not be an assert anymore.


Thank you for making this clear. I just wanted to make sure, that there is no explicit functionality in QtTest that would intercept an exception instead of doing this myself.
Asserts don't throw exceptions. They call abort(). So there is nothing to "intercept". Even as the manual for abort says - "abort() does not return". If you want, just throw an exception instead of asserting and you'll be able to catch that in QtTestLib.

doberkofler
20th September 2010, 21:02
Assert, by definition, kills your app. If you want an assert that will not kill your app but instead will return an error, it will not be an assert anymore.
I agree! My "assert" would log an error, show the error and then abort.


Asserts don't throw exceptions. They call abort(). So there is nothing to "intercept". Even as the manual for abort says - "abort() does not return". If you want, just throw an exception instead of asserting and you'll be able to catch that in QtTestLib.
Sorry for the misunderstanding, but I used the term exception in a generally and did not specifically refer to the C++ exception handling. More precisely put, I was unsure if QtTest might intercept the assert.

Intercepting my own assert handling is actually what I have done now and it seems to give me expected results. QtTets can actually catch C++ exception thrown by the test methods.

lyuts
21st September 2010, 08:22
The only place where I use asserts is in my internal classes. By saying internal, I mean those that do not provide public interface and is used only by me in my modules. Otherwise, I try to do regular checks.