Sam Hooke

Setting up CMocka with CMake for Linux and Windows

All notes in this series:

CMocka is a unit testing framework for C. These notes cover how to configure CMake for an existing C project to use CMocka, for both Linux and Windows, in the following steps:

  • Install CMocka.
  • Update your CMake modules to find CMocka.
  • Update your CMakeLists.txt to link against CMocka.
  • Update your project structure to add a tests directory.
  • Add a basic CMocka test to your project.
  • Run the test.
  • Add a more advanced test which hooks into your project.

In these notes I’m using Windows 10 and Debian 9.3 “stretch”. For a terminal, on Windows I’m using Git Bash.

Install CMocka §

Clone CMocka §

First things first: CMocka must be built and installed on your system. Your C program will then link against it. You do not need to copy the CMocka source into your program (except for the CMake modules, which is covered later).

Start by cloning cmocka from source and change directory into the clone:

$ git clone https://git.cryptomilk.org/projects/cmocka.git
$ cd cmocka

Generate makefiles §

Before running the next step of generating the makefiles, note that the default CMocka install locations are:

  • C:/Program Files (x86) on Windows.
  • /usr/local on Linux.

If desired, change the install location with -DCMAKE_INSTALL_PREFIX. On Windows, I installed to C:/c_libs/cmocka (partly because I don’t want to clutter Program Files, partly because it avoids needing administrator privileges in the install step later, and partly because I still distrust spaces in path names):

$ cmake -S . -B build -G "MinGW Makefiles" -DCMAKE_INSTALL_PREFIX="C:/c_libs/cmocka" -DCMAKE_BUILD_TYPE=Debug

On Linux I used /usr as the install location. I omit the -G "MinGW Makefiles" since that’s only needed for Windows:

$ cmake -S . -B build -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Debug

Running this command generates the makefiles (but does not yet actually build or install CMocka).

Following is example output from generating the makefiles on Windows:

-- The C compiler identification is GNU 8.1.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/mingw-w64/i686-8.1.0-posix-dwarf-rt_v6-rev0/mingw32/bin/gcc.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Could NOT find NSIS (missing: NSIS_MAKE)
-- Looking for assert.h
-- Looking for assert.h - found
-- Looking for inttypes.h
-- Looking for inttypes.h - found
-- Looking for io.h
-- Looking for io.h - found
...
-- Looking for snprintf
-- Looking for snprintf - found
-- Looking for vsnprintf
-- Looking for vsnprintf - found
-- Performing Test HAVE_GCC_THREAD_LOCAL_STORAGE
-- Performing Test HAVE_GCC_THREAD_LOCAL_STORAGE - Success
-- Performing Test HAVE_MSVC_THREAD_LOCAL_STORAGE
-- Performing Test HAVE_MSVC_THREAD_LOCAL_STORAGE - Success
-- Performing Test HAVE_CLOCK_REALTIME
-- Performing Test HAVE_CLOCK_REALTIME - Success
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
CMake Error: failed to create symbolic link 'D:/PublicRepos/cmocka/compile_commands.json': operation not permitted
-- Configuring done
-- Generating done
-- Build files have been written to: D:/PublicRepos/cmocka/build

Build CMocka §

Next, perform the actual build:

cmake --build build

Install CMocka §

Then install cmocka. It will install to the location you specified earlier (or the default if unspecified):

$ cmake --install build
-- Install configuration: "Debug"
-- Up-to-date: C:/c_libs/cmocka/lib/pkgconfig/cmocka.pc
-- Up-to-date: C:/c_libs/cmocka/lib/cmake/cmocka/cmocka-config-version.cmake
-- Up-to-date: C:/c_libs/cmocka/include/cmocka.h
-- Up-to-date: C:/c_libs/cmocka/include/cmocka_pbc.h
-- Up-to-date: C:/c_libs/cmocka/lib/libcmocka.dll.a
-- Up-to-date: C:/c_libs/cmocka/bin/cmocka.dll
-- Up-to-date: C:/c_libs/cmocka/lib/cmake/cmocka/cmocka-config.cmake
-- Up-to-date: C:/c_libs/cmocka/lib/cmake/cmocka/cmocka-config-debug.cmake

If it fails with Maybe need administrative privileges. (likely on Windows if you’re using the default installation location of C:/Program Files (x86)) then run again as administrator.

Configure PATH §

On Windows, update your system PATH to add the CMocka bin directory. This is required so that at runtime Windows will be able to find the cmocka.dll:

C:\c_libs\cmocka\bin

On Linux, if you used /usr as your install location, then you shouldn’t need to make any PATH changes.

Configuring your C application to use CMocka §

Copy CMake files §

Copy the CMocka provided *.cmake files into your project. These are needed to enable CMocka commands such as add_cmocka_test. In my project, I store CMake modules here:

<project root>/cmake/<module name>

To do the same, go to the CMocka git clone and copy everything from cmocka/cmake/Modules/* into <project root>/cmake/cmocka/. This includes:

$ ls cmake/cmocka/ -l
total 40
-rw-r--r-- 1 sam 197609  866 Nov 20 00:44 AddCCompilerFlag.cmake
-rw-r--r-- 1 sam 197609 4266 Nov 20 00:44 AddCMockaTest.cmake
-rw-r--r-- 1 sam 197609 1349 Nov 20 00:44 COPYING-CMAKE-SCRIPTS
-rw-r--r-- 1 sam 197609 1952 Nov 20 00:44 CheckCCompilerFlagSSP.cmake
-rw-r--r-- 1 sam 197609  811 Nov 20 00:44 DefineCMakeDefaults.cmake
-rw-r--r-- 1 sam 197609 3973 Nov 20 00:44 DefineCompilerFlags.cmake
-rw-r--r-- 1 sam 197609  585 Nov 20 00:44 DefinePlatformDefaults.cmake
-rw-r--r-- 1 sam 197609 1480 Nov 20 00:44 FindNSIS.cmake
-rw-r--r-- 1 sam 197609  680 Nov 20 00:44 MacroEnsureOutOfSourceBuild.cmake

Add FindCMocka.cmake §

To help support Windows builds, we want to add a CMOCKA_PATH option which allows passing in the path to where CMocka is installed when generating the makefiles. While there are other approaches here (such as ensuring the CMocka libraries and include dirs are on your PATH) I prefer to explicitly pass in -DCMOCKA_PATH="C:/c_libs/cmocka" on Windows.

Create a new file <project root>/cmake/cmocka/FindCMocka.cmake and paste in the following:

set (CMOCKA_PATH "" CACHE STRING "Custom CMocka path")

# Search for the CMocka include directory
find_path(CMOCKA_INCLUDE_DIR
  NAMES cmocka.h
  PATHS ${CMOCKA_PATH}/include
  DOC "Where the CMocka header can be found"
)
set(CMOCKA_INCLUDE_DIRS "${CMOCKA_INCLUDE_DIR}")

# Search for the CMocka library directory
find_library(CMOCKA_LIBRARY
  NAMES cmocka
  PATHS ${CMOCKA_PATH}/lib
  DOC "Where the CMocka library can be found"
)
set(CMOCKA_LIBRARIES "${CMOCKA_LIBRARY}")

# Set CMOCKA_FOUND (if all required vars are found).
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(CMocka DEFAULT_MSG CMOCKA_INCLUDE_DIRS CMOCKA_LIBRARIES)

# Hide variables from cmake GUIs.
mark_as_advanced(CMOCKA_PATH CMOCKA_INCLUDE_DIR CMOCKA_INCLUDE_DIRS CMOCKA_LIBRARY CMOCKA_LIBRARIES)

Update top-level CMakeLists.txt §

Now update your project’s top-level CMakeLists.txt.

First, ensure it includes all the CMake modules we just added:

# Add CMocka CMake modules
list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake/cmocka)

Then, below that add a UNIT_TESTING option, which includes a tests/ directory in the build (we’ll create that directory in a moment). This conditional means that unit tests will only be built if you include the option -DUNIT_TESTING=ON:

if (UNIT_TESTING)
     find_package(cmocka 1.1.0 REQUIRED)
     include(AddCMockaTest)
     add_subdirectory(tests)
endif()

Create a simple test §

Create a tests/ directory, then create a simple_test.c inside that directory:

#include <stdarg.h>
#include <stddef.h>
#include <setjmp.h>
#include <stdint.h>
#include <cmocka.h>

/* A test case that does nothing and succeeds. */
static void null_test_success(void **state) {
    (void) state; /* unused */
}

int main(void) {
    const struct CMUnitTest tests[] = {
        cmocka_unit_test(null_test_success),
    };

    return cmocka_run_group_tests(tests, NULL, NULL);
}

Note that this test will not be picked up by CMocka until we do the next step of hooking it up in CMake.

(For more example tests, see cmocka/example in the CMocka code base).

Hook up test to CMake §

Inside the tests/ directory create a CMakeLists.txt. Within this file, put:

add_cmocka_test(simple_test
                SOURCES simple_test.c
                COMPILE_OPTIONS ${DEFAULT_C_COMPILE_FLAGS}
                LINK_LIBRARIES "${CMOCKA_LIBRARIES}")
add_cmocka_test_environment(simple_test)
target_include_directories(simple_test PUBLIC "${CMOCKA_INCLUDE_DIRS}")

This registers the CMocka test with CMake, links it to the CMocka library (specified by CMOCKA_LIBRARIES) and ensures that cmocka.h is included (specified by CMOCKA_INCLUDE_DIRS).

Building and running the unit tests §

Building the unit tests §

As mentioned earlier, to compile tests, first use the option -DUNIT_TESTING=ON when generating the makefiles. For example, on Windows:

cmake -S . -B build -G "MinGW Makefiles" -DUNIT_TESTING=ON

On Linux the -G "MinGW Makefiles" can be omitted:

cmake -S . -B build -DUNIT_TESTING=ON

Then build as normal:

$ cmake --build build
Consolidate compiler generated dependencies of target myproject
[ 90%] Built target myproject
[ 95%] Building C object tests/CMakeFiles/simple_test.dir/simple_test.c.obj
[100%] Linking C executable simple_test.exe
[100%] Built target simple_test

This should have created a tests/simple_test.exe file. If you’re curious, you can just run that file manually to run that individual test, however there’s better ways to run the tests.

Running the unit tests §

To run all your built tests:

ctest --test-dir build

Use -V to see more details.

You may have noticed that our simple_test.c does not actually test any of our project’s code. Before we continue, let’s take a look at our project structure:

.
└── MyProject/
    ├── app/
    │   ├── CMakeLists.txt
    │   └── main.c
    ├── cmake/
    │   └── cmocka/
    │       ├── FindCMocka.cmake
    │       └── ...
    ├── include/
    │   ├── level.h
    │   ├── objects.h
    │   └── sprites.h
    ├── src/
    │   ├── CMakeLists.txt
    │   ├── level.c
    │   ├── objects.c
    │   └── sprites.c
    ├── tests/
    │   ├── CMakeLists.txt
    │   ├── test_simple.c
    │   ├── test_level.c
    │   ├── test_objects.c
    │   └── test_sprites.c
    └── CMakeLists.txt

As an overview:

  • The top-level CMakeLists.txt calls project(MyProject VERSION 0.1.0) to define the project.
    • It calls add_subdirectory(src) to add the src directory.
    • It also calls add_subdirectory(app) to add the app directory.
  • The src/CMakeLists.txt:
    • Calls file(GLOB HEADER_LIST CONFIGURE_DEPENDS "${MyProject_SOURCE_DIR}/include/*.h") to populate HEADER_LIST with all the header files (level.h, objects.h, sprites.h)
    • Calls add_library(my_lib STATIC level.c objects.c sprites.c ${HEADER_LIST}) to create a library called my_lib which includes all the source files in src/ with the matching header files.
    • Calls target_include_directories(my_lib PUBLIC "${MyProject_SOURCE_DIR}/include") to include the directories with the header files.
  • The app/CMakeLists.txt:
    • Calls add_executable(my_game main.c) to define the executable. The main() method is in main.c.
    • Calls target_link_libraries(my_game PUBLIC my_lib) to link my_game to our library my_lib.

So, to test my_lib we must make the following changes to tests/CMakeLists.txt.

  • Link to the library we want to test (i.e. my_lib).
  • Also include the my_lib source code so that we can test the static (private) functions, by doing something like #include "objects.c".
    • Typically you would only ever include a header (*.h) file, but including the source file (*.c) is a useful trick for unit testing private functions.

In this example, the code being tested is the target named my_lib. We modify our tests/CMakeLists.txt from earlier to support the above two points:

  • Create a new list TEST_LIBS, which contains CMOCKA_LIBRARIES and now also my_lib.
  • Create a new list TEST_DIRS, which contains CMOCKA_INCLUDE_DIRS and now also the src/ directory.
list(APPEND TEST_LIBS "${CMOCKA_LIBRARIES}")
list(APPEND TEST_LIBS my_lib)

list(APPEND TEST_DIRS "${CMOCKA_INCLUDE_DIRS}")
list(APPEND TEST_DIRS "${MyProject_SOURCE_DIR}/src")

add_cmocka_test(
    simple_test
    SOURCES simple_test.c
    COMPILE_OPTIONS ${DEFAULT_C_COMPILE_FLAGS}
    LINK_LIBRARIES "${TEST_LIBS}")
add_cmocka_test_environment(simple_test)
target_include_directories(simple_test PUBLIC "${TEST_DIRS}")

It is now possible to do the following from within simple_test.c:

  • Use #include "objects.h", and write tests against the public functions available in that header file.
  • Use #include "objects.c", and write tests against the public and private functions available in that source file.

Multiple unit tests §

So far we have just one unit test, simple_test.c. Adding new unit tests by copying and pasting the calls to add_cmocka_test, add_cmocka_test_environment and target_include_directories would add a lot of boilerplate code.

Instead, we can use the list approach with the unit tests. By defining a list named TEST_TARGETS which contains the filename of all the tests (omitting the .c extension), we can then use this for the CMake target name, and stick the extension back on for the source:

list(APPEND TEST_TARGETS test_simple)
list(APPEND TEST_TARGETS test_level)
list(APPEND TEST_TARGETS test_objects)
list(APPEND TEST_TARGETS test_sprites)

list(APPEND TEST_LIBS "${CMOCKA_LIBRARIES}")
list(APPEND TEST_LIBS my_lib)

list(APPEND TEST_DIRS "${CMOCKA_INCLUDE_DIRS}")
list(APPEND TEST_DIRS "${MyProject_SOURCE_DIR}/src")

foreach(TEST_TARGET IN LISTS TEST_TARGETS)
    add_cmocka_test(
        ${TEST_TARGET}
        SOURCES "${TEST_TARGET}.c"
        COMPILE_OPTIONS ${DEFAULT_C_COMPILE_FLAGS}
        LINK_LIBRARIES "${TEST_LIBS}")
    add_cmocka_test_environment(${TEST_TARGET})
    target_include_directories(${TEST_TARGET} PUBLIC "${TEST_DIRS}")

The above example will test: test_simple.c, test_level.c, test_objects.c and test_sprites.c in order.

Troubleshooting §

Unit tests build but fails at runtime §

If everything works until you run the unit tests, and then all the tests fail with exceptions, e.g.:

$ ctest --test-dir build -V
...
test 7
    Start 7: test_objects

7: Test command: D:\MyProject\build\tests\test_objects.exe
...
7: Test timeout computed to be: 1500
7/8 Test #7: test_objects .....................Exit code 0xc0000135
***Exception:   0.03 sec
...
0% tests passed, 8 tests failed out of 8

Total Test time (real) =   0.26 sec

The following tests FAILED:
...
          7 - test_objects (Exit code 0xc0000135
)
...
Errors while running CTest
Output from these tests are in: D:/MyProject/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.

Or if you get an error such as the following when manually running an individual test on Windows:

Error saying 'The code execution cannot proceed because cmocka.dll was not found. Reinstalling the program may fix this problem.'
"The code execution cannot proceed because cmocka.dll was not found"

Make sure that you have correctly got cmocka.dll available on your PATH. For example, if you installed CMocka to C:\c_libs\cmocka then your path should have C:\c_libs\cmocka\bin. Alternatively, as a quick workaround, copy the cmocka.dll from C:\c_libs\cmocka\bin\cmocka.dll into your tests/ directory.

Some useful CMake information:

All notes in this series: