Cmake with QT Tests

[DAVC]

Hi.
So I heard somewhere that testing your code is a good idea. :eek:

In my project I am using CMake with visual studio 2015.

I found out how to add my QT Tests to my project, using CMake, though it wasn't easy.

My problem is that when a tests fails, it causes CTest itself to fail:


1> Test project C:/work/foo/build/msvc2015_64
1> Start 1: foo
1> 1/1 Test #1: foo...............***Failed 0.05 sec
1>
1> 0% tests passed, 1 tests failed out of 1
1>
1> Total Test time (real) = 0.06 sec
1>
1> The following tests FAILED:
1> 1 - foo(Failed)
1> Errors while running CTest
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.Cp pCommon.targets(133,5): error MSB3073: The command "setlocal
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.Cp pCommon.targets(133,5): error MSB3073: "C:\Program Files\CMake\bin\ctest.exe" --force-new-ctest-process -C Debug
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.Cp pCommon.targets(133,5): error MSB3073: if %errorlevel% neq 0 goto :cmEnd
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.Cp pCommon.targets(133,5): error MSB3073: :cmEnd
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.Cp pCommon.targets(133,5): error MSB3073: endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.Cp pCommon.targets(133,5): error MSB3073: :cmErrorLevel
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.Cp pCommon.targets(133,5): error MSB3073: exit /b %1
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.Cp pCommon.targets(133,5): error MSB3073: :cmDone
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.Cp pCommon.targets(133,5): error MSB3073: if %errorlevel% neq 0 goto :VCEnd
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.Cp pCommon.targets(133,5): error MSB3073: :VCEnd" exited with code 8.

If no tests fail, then there is no problem.

The test fails:



void TestSignals::testgetName() {
QCOMPARE(s.getName(), "dfsdf ");
}

QTEST_MAIN(TestSignals);


The test passes:



void TestSignals::testgetName() {
QCOMPARE(s.getName(), "");
}

QTEST_MAIN(TestSignals);


Another problem is, that I don't get any diagnostics. In the documentation it states that QCOMPARE should give a more descriptive error message, but as you can see, I get nothing.

I suppose the missing messages are caused by the fact that CTest fails. Is there a way to make CTest not fail just because a test fails?

Best Regards
DAVC

[anda_skoa]

Another problem is, that I don't get any diagnostics. In the documentation it states that QCOMPARE should give a more descriptive error message, but as you can see, I get nothing.


You can either run ctest with the "-V" ("--verbose") option or you run the test individually.

Once you have multiple tests it is usually nice to have ctest just run through all with "summary" output and then one can investigate any test that fails more closely.

Cheers,
_

[DAVC]

Thanks for your reply anda_skoa.

Can you elaborate on how to invoke the verbose option for CTest in CMakeLists.txt?


include_directories(${CMAKE_SOURCE_DIR})


SET( TEST_LIBRARIES fileparsing Qt5::Test)

SET( foo_SRCS foo.cpp )
ADD_EXECUTABLE( foo ${foo_SRCS} )
TARGET_LINK_LIBRARIES( foo ${TEST_LIBRARIES} )
ADD_TEST( NAME foo COMMAND foo )

set(CMAKE_CONFIGURATION_TYPES Debug;Release)

if(WIN32)

if($<CONFIG:Debug>)


get_target_property(WIDGETDLL Qt5::Test IMPORTED_LOCATION_DEBUG)


get_target_property(WIDGETDLL Qt5::Core IMPORTED_LOCATION_DEBUG)

else()


get_target_property(WIDGETDLL Qt5::Test IMPORTED_LOCATION_RELEASE)


get_target_property(WIDGETDLL Qt5::Core IMPORTED_LOCATION_RELEASE)

endif()


add_custom_command(


TARGET foo POST_BUILD


COMMAND ${CMAKE_COMMAND} -E copy_if_different



$<TARGET_FILE:Qt5::Test>



$<TARGET_FILE:Qt5::Core>



$<TARGET_FILE_DIR:foo>

)
endif()

Best Regards
DAVC

[anda_skoa]

Can you elaborate on how to invoke the verbose option for CTest in CMakeLists.txt?


The CMakeLists.txt is for building the tests, their dependencies.

The mention option is a commandline argument for when you run the ctest program.

For example if you run ctest from a shell then

$ ctest

will run all tests and just report status (passed, failed, etc) for each one, while

$ ctest -V

should also show the output of each test.

Cheers,
_