Build and Test


There is a bewildering array of build systems. None of them are really satisfactory. I had hoped to avoid imposing any requirement as to the build/test system and leave the option to each library author to choose what he felt was most convenient. Some experimentation with my prototype library safe_numerics convinced me that this wasn't realistic and that I would have to require a specific build/test system. I wanted a system which:

  • would work on the wide variety of environments on which C++ is run.

  • would gracefully handle build variants such as debug/release, dll/static libraries, etc.

  • didn't require any customization for each environment.

  • didn't require elaborate installation and/or scripting.

  • minimized the learning curve.

  • would include the ability to support both building AND testing of libraries.

    This last has turned out to be particularly problematic. Repeatable, portable unit testing is essential to development of robust software. We require a system which supports the following:

    • Run by developer in his local enviroment before code is committed.

    • Run by prospective library users when library is downloaded.

    • Log of test results from various users posted in a public place which is easily viewable by library developers, users and reviewers.

    • Is scalable - doesn't require some centralized testing setup.

    • Tests with a wide variety of platforms and compilers.

I looked at a few specific alternatives.

Make

In the spirit of keeping things as simple as possible, this was my first choice. But starting with a basic make file, one would need to add a lot of extras functionality to address the requirements in the above list. Basic make doesn't have enough functionality to support all we need so we're forced to specify a build/test system.

Autotools

Autotools is the traditional Unix solution to the problem of build/test/deployment. It has a long history and is widely used. There is lot's of literature on it as well. It garners a number of complaints about how hard it is to use - but then it seems all of these build systems share that trait. And it doesn't do windows which is sort of deal killer for us. None the less, I wanted to take a look at it. First I looked at my books. It was explained but it did seem pretty complicated. I trolled the net and found a couple of references:

The first one has a graphic on page 16 which helped make it clear how all the pieces of this system fit together. On the other hand, this graphic didn't make it look very easy. I went through the process with my unix-like CYGWIN installation and things seemed to run well until the last step. Again, I had the feeling I could make this work if I was willing to put in more time, but I didn't feel like doing it. An interesting experience but not a good solution for us.

Summary and Conclusion

After trying and/or investigating many of the alternatives, I've settled on CMake/CTest/CDash for Safe Numerics. I feel comfortable strongly recommending this for those who want to make libraries which they hope to submit to Boost. It supports our list of requirements as well as any of the other alternatives considered. AND it's the only alternative which supports our specific requierments for logging test results in a commonly accessable place. As I write this, I see CMake/CTest/CDash as indispensible for libraries posted on this website.

Note the above doesn't preclude support for other build systems as well. In particular BJAM might be a candidate if one's library evolves to the point of being a serious candidate for inclusion into boost.

Here is a list of miscellaneous links to sites that I found useful in my investigation of Build systems.

Comment on This Page