The TestingLite Library provides a small set of test execution tools and test building tools for quickly setting up a program to run automated tests.

The TestingLite Library

• Is written in C++
• Does not use any third-party libraries, including MFC.
• Is only incompatible with MFC when its memory leak detection is turned on (by default it is not on). See the header MemLeakDetect.h for details.
• Is distributed in binary form with builds for Visual C++ 5.0 and 6.0
• Is distributed in binary forms compatible with the three possible Run-Time Library settings available in VC++ for Debug builds
• Cannot be linked into Release builds
• Currently is available for MS Win32 platforms
• Is designed to be small and robust

Go to the samples subdirectory of the installation and pick a sample based on the kind of testing you need to do. Currently the choices are:

• Process Test – the Software Under Test (SUT) is a standalone executable. The test program launches that executable and feeds it inputs and captures its outputs.
• Class Test – the SUT is a class that is linked into the test program itself.

The installation for the TestingLite library contains two subdirectories required for setting up an application that uses the library:

• include – public headers for the library
• lib – compiled static-link libraries

The include path must be added to your application’s preprocessor include path and the lib path must be added to your application’s linker input path. The individual .lib filenames are not important, so don’t add them to the compiler’s link settings. Thanks to a special #pragma statement used by TestingLite the correct .lib file will automatically be linked into your program.

Using TestingLite

The TestingLite library contains the following categories of functionality:

1. Test Running – command-line parsing and invocation of user-defined test functions
2. Memory Leak Detection – sets up the Microsoft Run-Time library for memory leak detection
3. Test Writing Utilities – lots of powerful goodies for making tests
4. Logging – class to record data to a text file

The TestingLite library provides a set of macros for building a "TestMap". The TestMap can then be passed to the TestMain() function, along with the command-line arguments from the program’s main() routine, to invoke individual tests or groups of tests.

TestMap Macros:

Simple Example TestMap:

What does this do?

• declares a variable with a name derived from "FooTest" that is an array of structures
• initializes the "FooTest" array variable with two structures, one with a reference to a user-defined function, the other with flag values to indicate it is the final structure in the array
• ties a the following function into the TestMap (note the "_test" modification):

void FooTestFunction_test(void)

• makes it possible to make the following function call inside your app’s main():

What does TestMain() do?

• parses the input command-line arguments looking for options it recognizes. Currently the following options are accepted:
• /? displays the program usage message
• /l (this is a lower-case L) prints the list of available tests and test suites
• /p causes the program to pause when and wait for user input before exiting
• /r forces TestingLite to maintain .success files, for use with makefiles
• gets a list of test names and/or test suite names that are to be executed, and executes them. The test and test suites names are not case sensitive. There can be any number of names on the command-line, even repetitions of names.

Example Command-Line for FooTest:

This command causes the FooTestFunction_test() routine to be called, and when it completes, a prompt is given to the user prior to exiting.

What About Test Suites?

A test suite is a named collection of tests. TestingLite’s idea of a test suite is pretty simple. It requires the suite to be delimited in the TestMap using the following TEST_MAP_SUITE_BEGIN() and TEST_MAP_SUITE_END macros. The suite name used in the ..._BEGIN() macro can be any string that won’t goof up the command-line. Basically, stick to letters, numbers, and punctuation like underscores and colons.

One nice way of setting up test suites is to make each one a C++ namespace. This way, the list of tests viewable with the /l option always looks like a list of C++ functions.

Test Suite Example TestMap:

What does this do?

• ties the following functions into the TestMap:

void SimpleTests::FooTestFunction1_test(void)

void SimpleTests::FooTestFunction2_test(void)

void FooSmokeTest_test(void)

void ComplexTests::FooTestFunction1_test(void)

• makes it possible to use any comma-separated combination of the following strings to run tests on the command-line:

SimpleTests::FooTestFunction1

SimpleTests::FooTestFunction2

FooSmokeTest

ComplexTests::FooTestFunction1

SimpleTests

ComplexTests

More on TestSuites:

• they can be nested arbitrarily deep
• the suite name does not need to be prefixed on each test inside the suite. This is only necessary when each suite is matched with a C++ namespace
• a test function can appear in more than one suite

The header file MemLeakDetect.h can be included into a single source file in your project, if you are NOT using MFC, in order to turn on the C RunTime Library’s memory leak detection feature.

TestingLite consists of the following categories of Test Writing Utilities:

1. File Comparison

• Simple Binary Comparison (See BinaryCompare.h)
• Regression Comparison (See Below for more on this topic)

1. File Manipulation (See FileTools.h)
2. Directory Manipulation (See DirTools.h)
3. Process Launching (See ProcessTools.h)
4. Stream Manipulation (See StreamRedirectJanitors.h)

The TestingLite Library contains some very advanced file comparison features for performing Regression Tests. This part of the library is designed for tests that generate output files such that the output file for one run of the test can be compared to the output file from that last "correct" build of the software to determine if the software has Regressed. When software Regresses it loses features and/or gains bugs.

The Regression Comparison system centers on the FileRegCompare.h header. It declares the CFileRegCompare class. This class allows clients to plug-in custom facilities for doing the following:

• File Comparison – simply determining if two output files are "equal"
• Sample plug-in: BinaryFileCompare.h
• File Viewing – displaying a single file
• Sample plug-in: NotepadView.h
• Visual File Differencing – visually displaying the differences between two output files
• Sample plug-ins: VisDiff.h and WinDiff.h
• Interacting – controlling how the system interacts with the user (console vs. GUI)
• Sample plug-ins: GUIInteraction.h and ConsoleInteraction.h

To use the Regression Comparison system you must do the following:

• Call CFileRegCompare::SetDefaultObjects() (a static method), with the default plug-ins.
• During a test, make a call like the following:

See the ProcessTest sample for an example of using the Regression Comparison system.

The Logger.h header includes a simple class for logging text to a disk file.

Currently the TestingLite library has two sample programs. They are in the samples directory under the release.

<no problems known at this time>

To summarize it, you can modify and redistribute this software as long as you give credit to the authors, and you don’t charge for it.

If you have ideas for corrections, enhancements, etc. for the library please send them to the authors (see the Credits seciont). If you have already made changes to the library that you think are globally useful, send them in and they may be added to the library in future releases (Wow! that sounded really professional).

This software was developed with the generous support of Pyramid Solutions, Inc. (www.pyrasol.com).