Change log

Boost.Test releases:

Boost.Test v3.10 / boost 1.70

New features

new documentation section about custom command line arguments
Custom datasets are not required to declare a inner type sample anymore
Boost.Test does not depend on Boost.Timer any more (which was pulling also Boost.Chrono and Boost.System as transitive dependencies).
Now Boost.Test raises an exception when the test case times-out on Windows. Prior to this release, times-out on Windows were not failing the test cases. Note that signaling is not available on Windows, and it is not possible to interrupt a test even in case of time out.
Time-out now applies to test-suites as well: a test-suite is marked as timed-out if it exceeds the allocated time. The test units that were not executed at the time-point of the time-out are skipped.
It is now possible to pass several values for the same context via the tool BOOST_TEST_CONTEXT.
A new macro BOOST_TEST_INFO_SCOPE let define a context for the current scope in a sticky way.
It is now possible to use floating point comparison without being required to cast both operands to floating point types. Now Boost.Test uses floating point comparisons for expressions such as
```
BOOST_TEST(3.0001 == 3);
```
See this section for more information.

Breaking changes

Boost.Test minimal.hpp is now showing a deprecation warning. minimal.hpp has been deprecated for a long time already, and will be removed in the near future. Please switch to eg. the header only variable of Boost.Test. Tests using minimal.hpp can readily be converted to the header variant. For instance, the following code:
```
#include <boost/test/minimal.hpp>
int test_main( int, char *[] )
{
  ...
}
```
may be rewritten as:
```
#include <boost/test/included/unit_test.hpp>
BOOST_AUTO_TEST_CASE(test_main)
{
  ...
}
```
The floating point comparison behaviour change may use this type of comparison while previously using straight relational operator comparison. In particular this may causes new warnings.
the member function boost::unit_test::unit_test_log_formatter::log_build_info has slightly changed to accept an additional boolean argument. If you have a custom logger, you will need to update its signature.

Bugfixes and feature requests

#GH-133 Timeout effect on Windows
#GH-138 expected_failures doesnt work for BOOST_DATA_TEST_CASE
#GH-141 Support for Boost.MP11 and Boost.Hana type lists
#GH-157 Test name should handle const-volatile specifiers
#GH-160 suppress -Wformat-overflow when optimization is enabled on GCC 8.2.0
#GH-174 UBSAN identified a problem at exit time by gcc-8 only
#GH-176 [snippet_dataset1_3] seems to be broken
#GH-177 boost_check_equal-str-test failed on llvm
#GH-180 Unreachable code warning on MSVC builds in test matrix
#GH-181 doc_example22 (and 23) are expected to fail, but do not on clang with release variant builds
#GH-194 master_test_suite declared twice
#GH-196 junit report: test error is also reported as failure
#GH-198 Support BOOST_UNIT_TEST_FRAMEWORK_DYN_LINK et al
#GH-199 Runtime type_mismatch after upgrade to 1.69
#GH-202 boost/timer.hpp is deprecated
#GH-203 Test cases with datasets and fixtures don't support flexible fixture interface
#GH-204 Feature Request: Allow specifying timeouts for test cases with datasests.
#GH-208 Incorrect handling of timed-tests on Windows
#GH-211 windows.h should be lower case
#GH-212 Comment ar parameter of assertion_result to avoid warning
#GH-213 BOOST_SYMBOL_VISIBLE cannot be used for enums with Sun Studio
#PR-171 Correct library name in test runner help screen
#PR-172 Check for non-used variables when NDEBUG is defined
#PR-182 fix use of bind1st in example 12
#PR-183 remove superfluous semicolon in example 04
#PR-184 fix example to use the correct variable and avoid unused variable warning
#PR-185 Added CI framework
#PR-190 fix warning on gcc-7.3 in cygwin claiming master_test_suite is declared differently
#PR-195 Fix MinGW compilation problems
#PR-197 Feature Request: BOOST_TEST_CONTEXT that doesn't require introducing a new scope with braces
#PR-205 Fix MinGW vsnprintf compile errors and warnings
#PR-214 Fixes an issue with sun_cc lacking the __global attribute for enums
#7397 Boost.Test, since boost 1.48 is using the deprecated Boost.Timer class (solved via #GH-202)
#9434 error: namespace boost::timer {} redeclared as different kind of symbol (solved via #GH-202)
#13106 libs/test/tools/console_test_runner does not compile
#13418 Request: allow general typelist types in BOOST_AUTO_TEST_CASE_TEMPLATE()

Boost.Test v3.9 / boost 1.69

New features

Official support of header-only variant of Boost.Test with multiple translation units. This feature was available but needed to be properly documented (here and here).
It is now possible to manually add a test case by specifying its name, with BOOST_TEST_CASE_NAME
Better logging of messages in boost::exception

Breaking changes

Bugfixes and feature requests

#GH-149 Setting color_output=no does not disable the output of color format codes
#GH-150 Some headers fail to compile independently
#GH-156 close_at_tolerance always returns false for comparisons of infinity
#GH-158 Detecting boost_test_print_type does not work when testing a type with an explicit conversion to bool
#GH-163 Significant start slowdown on MSVC x64/Debug after upgrade to v1.68.0
#PR-147 Catch block for boost::exception appears after std::exception in execution_monitor::execute()
#PR-148 Colored output contradiction
#PR-151 Fix warning: BOOST_CLANG is not defined, evaluates to 0
#PR-154 When specifying --color_output=no, don't output color codes
#PR-161 add a self-containment test
#13380 data-driven tests' join operator + corrupts first column (duplicates #12216)
#13625 Boost.test fail to compile with -Werror=missing-declarations on some architectures
#13637 Fix for Bug #12597 causes a problem with BOOST_TEST_CASE

Boost.Test v3.8 / boost 1.68

New features

The tests generated from a dataset are now instanciated during the framework setup. This let the dataset generator access the argc and argv of the master test suite. For indicating a dataset that should be instanciated in a delayed manner, a new data::make_delayed helper has been introduced.
It is now possible to create a dataset with data::make, with variable number of arguments. As the datasets are monomorphic, those should be castable to the first element type.

Breaking changes

the master_test_suite_t is not copyable anymore.
As datasets can now be delayed, it might be that additional copies of the dataset arguments are performed. This is especially the case for datasets created out of an std::initializer_list.

Bugfixes and feature requests

#PR-143 Fix exception_api.run-fail.cpp doc example
#PR-145 Fix build of library on recent Cygwin editions
#12095 disabling test with precondition leads to error
#12953 access to master_test_suite().{argc, argv}
#13504 [Boost::Test] short form of catch_system_errors not working
#13525 Boost Test 1.67.0: Compilation error with GCC 4.6.3
#13528 Boost Test 1.67 crashes when the --report_sink command-line parameter is used

Boost.Test v3.7 / boost 1.67

Breaking changes

Now colour is on by default for the output streams that are either std::cout or std::cerr. This can be disabled by passing --no_color_ouput (or just --no_color) to the command line.
Adding test cases with the same name to the same test suite is reported as an error. This impacts template and parametrized test cases, as well as manually registered tests. Make sure you have no duplicate names.

New features

Colour output on Windows
Improved and clearer command line help
BOOST_AUTO_TEST_CASE_TEMPLATE now accepts a sequence of types in an std::tuple

Bugfixes and feature requests

#PR-112 Deliberate-failure tests shouldn't be optimized
#PR-118 Update VxWorks support
#PR-118 [clang] Fix [-Wc++11-narrowing] error
#PR-121 fix compiler warning
#PR-122 Fix some fallthrough warnings with gcc >= 7
#PR-125 Prevent 2 unused parameter warnings
#PR-127 Silence 'unused variable' warning
#PR-134 Fix stdcerr file creation on shutdown
#PR-136 Change Windows.h include to all-lowercase (MinGW)
#12092 Request: allow std::tuple typelists in BOOST_AUTO_TEST_CASE_TEMPLATE
#12596 Sanitize metacharacters in test names
#12597 Report tests with clashing names
#12969 Problem linking print_helper_t<nullptr_t> under Clang
#13058 errors.hpp in Boost Test requires warning C4946 to be off
#13149 Dependency decorators on parent suites
#13170 BOOST_AUTO_TEST_CASE_TEMPLATE don't want typedef for list
#13181 Boost test can't compare classes which have begin and end but not const_iterator
#13371 Use-after-free with --log_sink=file
#13387 Test header fails to compile
#13398 Log format JUNIT generates invalid XML files
#13407 Boost.Test appears to crash under Cygwin
#13435 BOOST_TEST_GLOBAL_CONFIGURATION (result report shutdown time)
#13443 Boost.Test data driven test fails to compile when number of samples greater than 9

Boost.Test v3.6 / boost 1.65

Breaking changes

BOOST_GLOBAL_FIXTURE is flagged as deprecated and will be removed in a later version
Using test assertions and macros is not allowed when used inside BOOST_GLOBAL_FIXTURE. Please use BOOST_TEST_GLOBAL_FIXTURE instead (see below).
the interface for loggers has slightly changed to take into account the current log level. This is for addressing #12631.

New features

VS2017 / C++17 compatibility (thanks to Daniela Engert)
Deprecating BOOST_GLOBAL_FIXTURE in favor of BOOST_TEST_GLOBAL_FIXTURE and BOOST_TEST_GLOBAL_CONFIGURATION. This helps separating the logic of the fixtures associated to the master test suite, from the one used for setting up the logging and reporting facility, and results in a general cleaner design.
It is possible to use now the BOOST_TEST check to comparing a collection with respect to regular arrays. See this section for more details.

Bugfixes and feature requests

#PR-106 replace deprecated binders and adapters, and random_shuffle by more modern equivalents
#5282 Test fixtures do not support virtual inheritance
#5563 using a test macro in a global fixture crashes Boost.Test
#11471 array is a sequence
#11962 BOOST_TEST_MESSAGE in fixture constructor - invalid XML
#12228 Some test headers fail to compile independently
#12631 BOOST_TEST_MESSAGE generates incorrect output when used in BOOST_DATA_TEST_CASE
#13011 BOOST_TEST broken with floating point relational operators

Boost.Test v3.5 / boost 1.64

New features

Now Boost.Test provides customization points for logging user defined types: this solution is less intrusive than forcing the definition of operator<< for a specific type.
JUnit output format can now have a log-level set between success and non-fatal error, and defaults to general information.
JUnit output format is now more performant in case a lot of checks are done in a test module.

Bugfixes and feature requests

#PR-107 BOOST_NO_EXCEPTIONS typo making throw_exception unusable under some circumstances
#PR-108 Change capital variable names to lowercase
#11756 boost.Test: non standards compliant use of FE_* macros (unable to compile boost test library on FPU-less arches) (reopened)
#12540 Provide customisation point for printing types in tests
#12712 BOOST_AUTO_TEST_SUITE: Generate unique names by using __COUNTER__
#12748 Boost.Test defines a variable called VERSION
#12778 Boost.Test is broken against left shift operator in certain cases (nullptr issue)

Boost.Test v3.4 / boost 1.63

Breaking changes

Now colons that appear in test case names are replaced with underscores. This affect mainly the template/typed test cases. The change is needed since the colon ':' is interpreted as a filter separators since 1.62, and it is otherwise not possible to execute the tests reported by --list_content. See #12531 for more details.

New features

Now per_element and lexicographic modifiers of BOOST_TEST can also be applied to string comparison. See string comparison for more details.

Bugfixes and feature requests

#PR-103 Syntactic change silences latest gcc warnings
#PR-105 Fix unused parameter warnings/errors with gcc 6
#11756 boost.Test: non standards compliant use of FE_* macros (unable to compile boost test library on FPU-less arches)
#11907 Why does BOOST_TEST() treat std::string as a collection?
#12339 Propose users given way to disable blink in colour output
#12506 typo in Boost.test report_sink description
#12507 Boost.test --report_sink parameter broken
#12530 No way to find out Boost.Test version without running any tests
#12531 --run_test in Boost 1.62 does not accept test names which contain ':'

Boost.Test v3.3 / boost 1.62

New features

Boost.Test now treats each sample of a dataset test case as being a uniquely named test case under the same test suite, which enables the (re)run of one particular sample from the command line interface. See here for more details,
Boost.Test learned to interpret ':' as a separator for the test filters: the string passed to --run_test generates tokens as if --run_test has been repeated, which enables the set up of several test filters through the associated environment variable BOOST_TEST_RUN_FILTERS
the Unit Test Framework learned to log the messages in the xUnit/JUNIT log format. See here for more details.
the Unit Test Framework learned to have several loggers at the same time, each of which with their own log level and log sink. See the associated command line switch --logger and corresponding environment variable BOOST_TEST_LOGGER for more details.
loggers are now able to indicate their default output stream and log level.

Bugfixes and feature requests

#PR-81 Possibility to remove the support of the alternative stack at compilation time. See BOOST_TEST_DISABLE_ALT_STACK for more details.
#8707 Provide Standard xUnit XML Output from Boost Test
#8834 Boost Test should be able to generate report in both XML and HRF together
#11128 [bb10/qnx failures] Build error
#11845 Ability to generate the unique and stable test name for every data set in BOOST_DATA_TEST_CASE
#11859 Wrong handling of "," in Run-Parameters
#12024 boost test depends on nonexisting abi::__cxa_demangle on android
#12093 Boost 1.60.0: Build fails (gcc 4.6)
#12103 Fix for gcc bug 58952 (getchar() is defined as a macro in uClibc)
#12224 Crash on MSVC with RTTI disabled
#12241 Data-driven testing over a range of std::tuple has broken
#12257 Incorrect line numbers in test_units generated from test_case_gen
#12378 Compilation errors with clang 3.8

Boost.Test v3.2 / boost 1.61

New features

now datasets support any arity, using the variadic template support of the compiler.
now datasets support fixtures through BOOST_DATA_TEST_CASE_F, see here for more details
now datasets honors move semantics of the types used for samples

Bugfixes and feature requests

#6767 Use of namespace qualifier with floating point exception functions breaks if they are macros
#8905 boost/test/impl/debug.ipp: Ignores return value from WaitForSingleObject
#9443 Runtime parameter Random seed for random order of test cases not respected correctly
#11854 Add fixture support in BOOST_DATA_TEST_CASE
#11887 BOOST_TEST(3u == (std::max)(0u, 3u)) fails
#11889 BOOST_DATA_TEST_CASE fails to compile for 4D and higher dimensional grids
#11983 Boost Test XML Report contains unescaped XML characters

Boost.Test v3.1 / boost 1.60

New major features

improved API for datasets
- it is now possible to use initializer lists
- the use of make as top left dataset is not necessary anymore
improved command line interface
- clearer help commands
- now proposes closest matching command in case of ambiguity
- reports invalid or ambiguous parameters: this might break existing calls when user defined commands are provided to the test module. The following calling convention should be adopted:
  - if the test module uses user supplied commands, those should be passed after an empty token --
  - all boost.test related commands should be passed before --, if any
  Example: the call
```
test_module --user-arg1=xy --log_level=test_suite
```
  should be rewritten to
```
test_module --log_level=test_suite -- --user-arg1=xy
```

Bugfixes and feature requests

#3384 Double-quoted arguments including spaces are divided by Boost.Test.
#3897 Test framework does not include <osreldate.h> before testing __FreeBSD_version (fixed in 1.59)
#6032 Program options within init_unit_test_suite are incorrect when using path and whitespaces
#6859 Boost.Test eats away last empty command line parameter
#7257 Boost.Test alters and does not restore ostream precision after any Test macro (fixed in 1.59)
#9228 Patch to make Boost.Test work with RTTI disabled (fixed in 1.59)
#10317 boost::test corrupts contents of argv if a paramter contains whitespace
#11279 invalid parameters should be reported
#11478 Boost Test Exception Assert Failure has poor message
#11571 Can't compile BOOST_TEST( ..., per_element() ) comparison of vector<string>
#11623 Clang rejects some simple BOOST_TEST() statements
#11624 BOOST_TEST( 0.0 == 0.0 ) fails under C++11 (GCC and Clang)
#11625 BOOST_TEST( ..., per_element() ) erroneously requires collections are comparable

Boost.Test v3 / boost 1.59

New major features

BOOST_TEST generic assertion
data driven test cases: supersedes the parametric test case (unary test cases)
test units attributes, that allow finer control over test units property and behavior
logical grouping of the test units using labels
support for declaring dependencies over test cases
attributes for enabling or disabling test execution based on static, compile-time or runtime rules
extended unit test filtering from the command line (negation, labels, ...)
color output with color_output
test bed listing with list_content
rewritten documentation using quickbook

Bugfixes and feature requests

#2018 Error in the documentation chapter "Runtime parameters reference"
#2450 equations in Floating-point comparison algorithms html are not rendered properly
#2600 Unit Test Framework - missed documentation
#2717 BOOST_<level>_EQUAL_COLLECTION docs typo
#2759 Typos in test new-year-resolution.html
#3182 _CrtSetReportFile can be used to redirect memory leaks report
#3316 Access violation when trying to log from init_tests_func
#3392 Boost::Test: Wrong contents for documentation of the BOOST_TEST_PASSPOINT macro
#3402 Invalid define name in docummentation (duplicates ##2717)
#3445 incorreect link in the docs
#3463 GT is GREAT!
#3542 Bug in documentation of detect_memory_leak parameter (duplicates ##2018)
#3481 Boost Testing doesn't work under Sun Solaris Containers (duplicates ##3592)
#3495 Boost::Test enters endless loop when running in vserver environment (duplicates ##3592)
#3592 under_debugger() goes into infinite loop
#3595 Typo (duplicates ##2759)
#3623 Boost Test Typo (duplicates ##2759)
#3664 SIGCHLD always considered fatal error
#3784 Documentation errors in Execution Monitor Compilation
#3785 Documentation errors in Program Execution Monitor implementation
#3811 global namespace polution
#3834 doc: probably incorrect HTML rendering (duplicates ##2450)
#3896 erroneous documentation in boost test commandline parameter description
#3932 Error in BOOST_<level>_GT description (duplicates ##3463)
#3938 doc: incorrect macro name (duplicates ##2759)
#3964 Documentation for BOOST_<level>_CLOSE_FRACTION is incorrect
#3978 Failed to completely redirect TestLog to file, bugfix appended
#3979 <boost/test/exception_safety.hpp> requires additional includes
#4161 spelling mistakes...
#4275 Documentation error Boost.Test (duplicates ##2717)
#4389 Enable boost_test to run specific tests with any required dependent tests.
#4434 BOOST_AUTO_EST_CASE typos in docs
#4587 Broken link in website
#4806 Invalid link (examples not showing up in documentation)
#4911 ENH: boost.test output the exception real type name.
#4923 Missing semicolon in documentation example
#4924 Minor typo in Boost::Test docs
#4982 Boost.Test has misspelled Gennadiy Rozental e-mail address
#5008 Boost.Test does not do checkpointing of entry/exit of test cases
#5036 Boost.Test VC memory leak report should direct to stderr
#5262 Run tests by name utility doesn't support negation
#5374 Errors from Boost.Test are no more shown in the Error list in VS2010
#5412 XML formatter in test library processes strings with subsequences ]]> incorrectly
#5563 using a test macro in a global fixture crashes Boost.Test
#5582 There is a memory leak in the BOOST_AUTO_TEST_CASE_TEMPLATE
#5599 boost::test documentation gives poor instruction
#5718 broken link to unit testing framework examples
#5729 Missing static_cast in fpt_limits
#5870 The warning stack is not maintained
#5972 Support program option to only dump the test-tree in text to output stream
#6002 Failed to completely redirect TestLog to file (duplicates)
#6071 Boost Test (Boost 1.46.0) GCC 4.6.1 error: ambiguous overload for ‘operator[]’
#6074 warnings-as-errors not usable with Boost.test in release mode
#6161 SunOS: bad putenv declaration (duplicates #6766)
#6766 incorrect declaration for putenv in config.hpp
#6712 Eliminate warnings with GCC
#6748 Link in the documentation points to wrong page
#7046 Output full error message, not just 512 chars
#7136 Correct documentation for BOOST_<level>_CLOSE_FRACTION is not reflected into released documents
#7410 Test Units (Cases and Suites) in Boost.Test do not capture __FILE__ and __LINE__ at declaration point making it impossible to provide source file linking using external test management tools
#7894 Boost.Test documentation contains no linking instructions
#8201 Broken link in document
#8272 BOOST_REQUIRE_CLOSE fails to compile with boost::multiprecision::cpp_dec_float_100 (duplicates ##11054)
#8467 Incorrect link in document (duplicates ##6748)
#8679 Boost.Test pollutes boost namespace with it's own enable_if/disable_if templates
#8862 Boost.Test typo in documentation
#8895 English error in test collection comparison
#9179 Documentation: broken link (unable to find const_string.hpp/const_string_test.cpp)
#9272 boost::test BOOST_<level>_GT documentation bug (duplicates ##3463)
#9390 Incomplete BOOST_TEST_DONT_PRINT_LOG_VALUE
#9409 Some source code examples are missing
#9537 const_string_test example fails
#9539 Floating-point comparison algorithms aren't formatted correctly
#9581 Squassabia reference link gives 404 not found
#9960 Warnings on Clang
#10256 [boost test] - issue: sigaltstack
#10318 Minor documentation fix
#10394 Broken links in Boost Test documentation
#10888 Assertion failures don't show up in the errors pane in VS 2010, VS 2012 or VS 2013 (duplicates)
#11054 Floating-point comparison of multiprecision values fails if expression template is on
#11347 DS identifier causes test failures in doc/examples/dataset_example*.cpp
#11358 Boost.Test v3 warning could helpfully be suppressed.
#11359 BOOST_CHECK_EQUAL_COLLECTIONS: can't control output operator (duplicates #9390)
#11425 use-of-uninitialized-value (obsolete)