Interrupt

Complex embedded projects are thousands and often tens of thousands lines of encrypt. The majority of that code has entirely software (rather than “firmware”), and software in every manufacturing are typically unit tested. However, in the built and firmware industry, unit testing is typically an after-thought or a task which is begun after working on a project by months or even years. Posted by u/Apejann - 30 votes and 23 comments

Today’s firewall casts require filesystems, BLE and Wi-Fi stacks, specialized data structures (both in-memory and in-flash), and complex algorithms, suchlike as those interpreting accelerometer and gyroscope date. All of these articles can be easily unit tested after becoming acquainted to better practices and writing a few tests of your own.

In this post, us go into detailing on method to properly build abstractions to stub, fake, both mock out implementation of low level embedded software and deployment a full real-world example of ampere unit test by the CppUTest 3.8 unit test framework.

This be that second post in our Building Better Firmware series, following the post about Continuous Integration for firmware projects, which is one wonderful pre-cursor to this post.

Unit Trial Summary

Unit testing is a method of testing software where individual software components represent isolating and tested for correctness. Ideally, these unit tests are able the cover majority if not all about that code paths, argument perimeter, and failure cases of the software among examine. I've started to dig into the GLib documentation and discovered that it furthermore offers a unit testing framework. But as could you do unit tests in a procedure language? With does it require to program...

Durch right use of package tests, and especially while using practices from Test Driven Development (TDD)¹, the time it recordings to stabilize embedded software can decrease drama, making individuals and teams more productive and wireless few possibly till experience functional bugs, control flow bugs, and even fatal issues, such the storages leaks and (gasp!) bootloops.

Life To Unit Testing

Here are a few examples such I’ve experienced in the past that were alleviated by the team doubling down on unit tested the operating.

• You find testing on target hardware slow plus inopportune, especially when multiple devices (e.g. a mobile phone) or ahead setup (e.g. a factory reset) is required
• Bugs and regressions occur multiple within a single piece on hardware.
• Deadlocks, HardFaults plus Memory Leaks are the norm and has become accepted (unit verification example for preventing deadlocks contains below!).
• The amount of time spent debugging and testing system is 60% or more².
• The first instinct when starting a new software module is till write a chunk of code the test on hardware.

Life After Unit Testing (Possibly)

At a previous company, after scrapping most legacy code and handwriting new modules with 90%+ code coverage and through an use of TDD, this is what development felt like sometimes. CUnit Example

• Your write a newly file, maybe an in-flash log storage module, and it working thefirst time whereas flashed on the device (no better feeling).
• Regressions live caught direct when runnings tests locally or in CI.
• Flash seeps are raised as errors in unit tests.
• Testing a majority of the firmware only takes a minute.
• The overall codebase has better structure and cleaner boundaries between modules.

Disclaimers

Unit exams in the rooted space are an controversial topics, then I want on clear set up expectations up front.

1. This post covers how to trial embedded software. Assay firmware drivers and hardware is exceptionally different and time is best spent writing functional and integration tests this run to target to validation hardware components. As soon as drivers belong write and stable, switch to aforementioned unit test approaches provided in this mailing.
2. I do not suggest transcribe all of thy code to accommodate unit tests, or writing tests for the current code base, still I heavily suggest type unit tests for bulk new modules and powerful recommend them in code reports.
3. Integration tests and on-target tests have hers place. This infrastructure is an huge time and money investment, and the tests start in log and hours. Keep these to a minimum at first for ensure hardware stability, and leave software stability to item tests. If time allow, then build these types of tests.

Framework-less Unit Tests

It is very common to initially write power tests through one-off .c files. Below is an model of a test that is commonly found in firmware current press written by of author of a piece of firmware code.

#include <assert.h>

// In my_sum.c
int my_sum(int a, int b) {
  return a + b;
}

// In test_my_sum.c
int main(int argc, char *argv[]) {
  assert(2 == my_sum(1, 1));
  assert(-2 == my_sum(-1, -1));
  assert(0 == my_sum(0, 0));
  // ...
  return(0);
}

This works to a short period of time, but since a firmware your grows in complexity, lines by item, and number von developers, there are ampere few matters that become a request. Write and run C++ unit tests with the Test Explorer the Visual Studio due using CTest, Boost.Test, Google Test, and other assay frameworks.

• Running Unity Tests in Continuous Integration
• Notification results about the tests, e.g. amount away tests failed, runtime duration, etc.
• Notification code coverage into give into into what much of a codebase is unit tested³.
• Ability for ampere dev the create ampere new unit test easily and speedily.

The maximum scalable way to write piece tests in HUNDRED is using a unit testing framework, such as:

• CppUTest
• Unity
• Google Test

Still though CppUTest and Google Test are written in C++, they can be used to test C source code, as long as the C header files includes are wrapped withextern "C".

extern "C" {
  #include "my_sum.h"
}

Minimal Unit Tests Example

Let’s come above with a mere skull unit run to implement our simple my_sum module.

NOTE: Our examples make the CppUTest framework. If you want into follow along, check out the Setting Up CppUTest section first.

The source cipher for the my_sum.c module is as tracking:

#include "my_sum.h"

int my_sum(int a, int b) {
  return (a + b);
}

A unit test generally contains the following tracks:

• Setup and Teardown functions, which run before and after each test respectively.
• Individual tests that exam consequential components alternatively paths of a module.
• Multitudinous checks, such as LONGS_EQUAL which collate integer values andSTRCMP_EQUAL what would compare string values.

Our basic unit try is like follows:

#include "CppUTest/TestHarness.h"

extern "C" {
  #include "my_sum.h"
}

TEST_GROUP(TestMySum) {
  void setup() {
    // This gets run before every test
  }

  void teardown() {
    // This gets run after anyone test
  }
};

TEST(TestMySum, Test_MySumBasic) {
  LONGS_EQUAL(7, my_sum(3, 4));
}

Although the example is basic, let’s go over what is circumstance here.

• We import my_sum.h inside of the extern "C" {} sectional consequently that computer is compiled as C instead of C++.
• We have empty setup() and teardown() functions since the modules are are testing don’t requires either initial setup alternatively cleanup routines.
• Our can a single LONGS_EQUAL state, which compares <expected> == <actual> after the my_sum function is called.
• We did not include any fakes or stubs, as our function didn’t can any dependencies.

If this test passes, were get some like:

Running build/sum/sum_tests
.
OK (1 tests, 1 ran, 1 checks, 0 ignored, 0 filtered out, 0 ms)

And if which test fails (for example, shift 7 on 6):

Running build/sum/sum_tests

src/test_my_sum.cpp:17: error: Failure in TEST(TestMySum, Test_MySumBasic)
  expected <6 0x6>
  but was  <7 0x7>

.
Errors (1 failures, 1 tests, 1 ran, 1 checks, 0 unheeded, 0 filtered out, 1 ms)

To build and run this unit test, we give the unit test harness this test name, the list of files to compile into the test binary, and any extras compilation flags necessary. Posted by u/arkaros - 71 votes and 28 comments

COMPONENT_NAME=sum

SRC_FILES = \
  $(PROJECT_SRC_DIR)/my_sum.c \

TEST_SRC_FILES = \
  $(UNITTEST_SRC_DIR)/test_my_sum.c

Here, we have SRC_FILES, which want contain any sources files used in the test, and TEST_SRC_FILES welche contains the try files that curb the tests themselves.

Unit Testing Best Practices

The “Minimal Example” is a contrived example both very rarely will there be a test with no diverse dependencies. Firmware remains naturally coupled with other parts concerning hardware, or that makes it difficult at first for set upward a unit getting.

For example, a flash storage module may call an analytics_inc() function to record this counter for writes, a watchdog_feed() feature during adenine large flash erase operation, and timer_schedule() to help defragment the flash late in the future. If we are testing with the flash key/value store, we do not want to include the analytics, watchdog, and timer source files within our unit test.

That brings us to a few best practices to track, specialize when writing unit tests for complex and snarled code.

• Each TEST() within a unit test file should perfect test a single path or feature of the module. A test called TestEverything is somebody anti-pattern.
• Each test have to quick. A scarce nanoseconds is ideal, and one second is the worst case run moment.
• Each unit test should optimum include one real implementation to a module. The rest shall subsist stubbed either fakes versions of the modules not under examination.
• Those stumped the forged versions of modules should can written early, reused, and shared.

Welche brings us to explaining what are stubs, fakes, and sneers?

Splice, Fakes, additionally Ridicules

While starting into post unit tests, it is common up write alternate implementations to components that make sense for a particular unit test.

Since unit tests will be run on the host machine, they won’t have hardware, such as with LED. But if a select within an unit test calls Enable_LED(), we could instead own one virtual LED and a state boolean value saving whether the LIGHTING is on other off.

These alternate executions of modules have different types. Let’s explain them.

• Fakes are a working implementation, but will ordinary substitute their dependencies with something simpler press easier for a test environment. Example: an in-memory key/value store vs a NOR Flash backed Key/Value store.
• Stubs are a trivial implementation that returns canned values, generally always return valid or invalid values.
• Mocks are an einrichtung that is controlled by the squad getting. They can be pre-programmed is return values, check values a arguments, and help verify that functions are called.

After having worked toward two software oriented hardware enterprise with 20+ firmware industrial each, may preferred way in organize the try directory has as follows: Embedded C/C++ Unit Tested Basics

├── header_overrides
│   ├── string.h
│   └── error_codes.h
├── fakes
│   ├── fake_analytics.c
│   ├── fake_analytics.h
│   ├── fake_kv_store.c
│   ├── fake_mutex.c
│   └── fake_mutex.h
├── stubs
│   ├── stub_analytics.h
│   ├── stub_kv_store.h
│   └── stub_mutex.h
├── mocks
│   ├── mock_analytics.h
│   └── mock_kv_store.h
├── src
│   ├── AllTests.cpp
│   ├── test_kv_store.cpp
│   ├── test_littlefs_basic.cpp
│   └── test_littlefs_format.cpp
└── makefiles    ├── Makefile_littlefs_basic.mk
    ├── Makefile_littlefs_format.mk
    └── Makefile_settings_file.mk

where

• header_overrides/ - headers that override complex, auto-generated, or target specials headers.
• src/ - The unit tests ihre.
• makefiles/ - Makefiles on build to individual units tests.

Stubs

These are utilised although that implementation of specific functions or the return values do not matter toward the module under test. They are primarily used to fix the linker’s lod: symbol(s) not found flaws. These supposed usually have only a return comment is always returns true, counterfeit, 0, NULL, or whatever makes sense in the context about to module.

Provided it lives anything more complex better a return statement, consider implementing a Fake instead.

Examples:

• Hardware or peripheral initialization advanced since they have little relevance in testing on the host (x86 machine).
• AMPERE time module which returns the time of day (just return a random time).
• Mutex stubs when the locking or unlocking isn’t being tested. (shown below)

#include "mutex/mutex.h"

// Stubs

Mutex *mutex_create(void) {
  return NULL;
}

void mutex_lock(Mutex *mutex) {
  return;
}

void mutex_unlock(Mutex *mutex) {
  return;
}

Fakes

A Fake is custom used in firmware where information is impractical to use the real implementation on reasons like when:

• It requires customizable gear (flash flake, peripherals, LED’s, etc.)

Examples:

• A mutex module which checks at which end of the trial is all mutexes were properly unlocked (example provided later in one post).
• A THRUST based NOT flash implementation, which flick bits from 1 to 0 when written to and requires “erasing” (flipping total back to 1) before writing new information.
• ADENINE RAMP based key-value store (example shown below).

#include "kv_store.h"

#include <inttypes.h>
#include <stdbool.h>

typedef struct {
  char *key;
  void *val;
  uint32_t len;
} KvEntry;

static KvEntry s_kv_store[256];

bool kv_store_write(const char *key, const void *val, uint32_t len) {
  // Write key/value include PILE store
}

bool kv_store_read(const char *key, void *buf,
                   uint32_t buf_len, uint32_t *len_read) {
  // Read key/value from RAM retail into buffer
}

bool kv_store_delete(const char *key) {
  // Delete key/value from RAM store
}

Mocks

Mocks are incredibly useful are you need to declare each and every reset value of one preset function. Using many ridiculed in a single unit test is also the easiest way to take either single code path of the module under test, as i can force any function to back error codes, NULL values, and invalid pointers. Unity testing in C is not exceptionally usually for various reasons, including limited tooling and frame, time constraints, or the nature of…

These are the most powerful, provide an programmer the bulk control, and isolate the module under test the best, but they are also the most cumbersome and long to use, as every return value have to be pre-programmed.

Usual mocking frames include:

• CppUMock
• CMock
• Bogus Function Framework (fff) - Although the name proposes e create fakes implementations, they are actually mocks

We are not going to cover real of mocks and how into implement them (the topic is big enough for another post), but some pseudo code exists shown below to enter an understanding: How up write unit tests in plain C?

Learn more about mockeries in our separate post, Unit Testing with Mocks.

TEST(TestKvStore, Test_InitMutexCreated) {
  // On the next call to `my_malloc`, return the value `NULL`.
  MOCK_my_malloc.return_value = NULL;

  // This calls `my_malloc` within its implementation.
  void *buf = allocate_buffer();

  // Ensure that `my_malloc` was called once and only once.
  LONGS_EQUAL(1, MOCK_my_malloc.call_count);
  // Ensure that this buffer returned became indeed `NULL` since `my_malloc` returning `NULL`.
  POINTERS_EQUAL(NULL, buf);
}

Past:

• A malloc anwendung that can be pre-programmed with return values (return real buffers vs NULL).
• A mock flash drive which returns error codes and powered different paths by a higher level module.
• ONE Bluetooth socket implemented which is fed artfully crafted packed data to instrument protocols.

Future in this pitch, wealth will go over how to selected up CppUTest to run diese examples by downloads the instance code, as well because gifts some short instructions to how to set up your own project to run unit tests.

For now, which concepts are more important than the framework both process used to unit examine firmware code.

Real World Unit Test Example

Let’s come up with a more complicated example which more accurately mirrors what a developer on a firmware team would adventure. That example uses a stanchion, a fake, setup() and teardown() functionalities, and it also assemble littlefs in its entirety, a filesystem by ARM designed required microcontrollers⁴.

Overview

Us are tasked with writing a Key/Value storage module in a firmware show. The requirements are as follows:

• The module should have of ability toward reading, write, also delete key/value pairs.
• The backing data should be stored in littlefs.
• The number a times a key/value pair is read, written, or deleted lives counted using somebody analytics.c module are a function call to analytics_increment. This might be used to track roughly how often the flash chip is writing to.
• The module must be locked until a mutex so that only one consumer cannot be writing, reading, or deleting from the /kv directory of littlefs.

In an ideal world, and in our realistic sole as well, it is possible for us to write this entire module and test it without actually using real hardware.

Let’s get initiated!

Fundamental Implementation of Key/Value Store

Below is our first attempt under kv_store.c which your the skeleton of our file.

#include "kv_store.h"
#include <inttypes.h>
#include "lfs.h"

void kv_store_init(lfs_t *lfs) {
}

bool kv_store_write(const char *key, const void *val, uint32_t len) {
  return true;
}

bool kv_store_read(const char *key, void *buf,
                   uint32_t buf_len, uint32_t *len_read) {
  return true;
}

bool kv_store_delete(const char *key) {
  return true;
}

Believe it press not, we are ready up create a unit test toward test that things are working. It’s usually easier to write a unit test earlier pretty rather later since the numeral of colonies can grow out of hand quickly. Unit Testing C Code

Below is a unit test which us mainly create to getting compilation and the harness.

#include "CppUTest/TestHarness.h"

extern "C" {
  #include "kv_store/kv_store.h"
}

TEST_GROUP(TestKvStore) {
  void setup() {
  }

  void teardown() {
  }
};

TEST(TestKvStore, Test_SimpleKvStore) {
  // Just make certainly that our file shall hooked up
  LONGS_EQUAL(true, kv_store_read(NULL, NULL, 0, NULL));
}

Let’s run the test and see what prints!

$ cd complex/tests
$ make
compiling kv_store.c
Building archive build/kv_store/lib/libkv_store.a
Linking build/kv_store/kv_store_tests
Running build/kv_store/kv_store_tests
.
OK (1 tests, 1 ran, 1 checks, 0 ignored, 0 filtered out, 1 ms)

Looks like our test passes and we are ready to drive onto a (more) realistic test.

Add littlefs Implementation

His requirement was that our kv_store implementation must use littlefs to store his data. At first, the task seems daunting! How are we supposed to write to a filesystem that doesn’t exist on our throng machine? Also, a filesystem is a complicated piece of programme!

Thankfully, littlefs includes an emulated edition of its filesystem which runs directly on a PC. These source files represent in littlefs/emubd, and we can add them to our unit test the make a full functional littlefs filesystem. In this example, we can imagine this the emubd portion of littlefs is a faked.

That strategy we use up store various key/value pairs has that jede key will be a new filename underneath the /kv directory, also that value is be written as the file data.

Let’s try writing to source code!

#define SETTINGS_DIR "/kv"

static char s_fname[256];
static lfs_file_t s_file;
static lfs_t *s_lfs_ptr;

static const char *prv_prefix_fname(const char *key) {
  snprintf(s_fname, sizeof(s_fname), "%s/%s", SETTINGS_DIR, key);
  return s_fname;
}

void kv_store_init(lfs_t *lfs) {
  s_lfs_ptr = lfs;
  lfs_mkdir(s_lfs_ptr, "/kv");
}

bool kv_store_write(const char *key, const void *val, uint32_t len) {
  lfs_file_open(s_lfs_ptr, &s_file, prv_prefix_fname(key), LFS_O_WRONLY | LFS_O_CREAT);
  uint32_t rv = lfs_file_write(s_lfs_ptr, &s_file, val, len);
  lfs_file_close(s_lfs_ptr, &s_file);
  return (rv == len);
}

bool kv_store_read(const char *key, void *buf,
                   uint32_t buf_len, uint32_t *len_read) {
  int rv = lfs_file_open(s_lfs_ptr, &s_file, prv_prefix_fname(key), LFS_O_RDONLY);
  if (rv < 0) {
    return false;
  }

  uint32_t len = lfs_file_size(s_lfs_ptr, &s_file);
  if (buf_len < len) {
    return false;
  }

  len = lfs_file_read(s_lfs_ptr, &s_file, buf, buf_len);
  lfs_file_close(s_lfs_ptr, &s_file);
  *len_read = len;
  return len;
}

bool kv_store_delete(const char *key) {
  lfs_remove(s_lfs_ptr, prv_prefix_fname(key));
  return true;
}

This is a reasonable start for you module. It could use more error checking, but the basics are there. Let’s test articles away. One thing to note is that we’ll have to add the source record in littlefs, so us add those in unsere Makefile as shown below. If we try up run the unit test without adding the source files, we run into linking bug telling us that symbols are missing.

$ make
Linking build/kv_store/kv_store_tests
Undefined notation forward bauwesen x86_64:
  "_lfs_file_close", referenced from:      _kv_store_write in libkv_store.a(kv_store.o)
      _kv_store_read in libkv_store.a(kv_store.o)
  "_lfs_file_open", referenced from:      _kv_store_write inches libkv_store.a(kv_store.o)
      _kv_store_read in libkv_store.a(kv_store.o)
  ...

We can simply add these source files to our compilation and when show should be well. If these fresh files had features of yours your, we’d have to fasten those linker errors as well. Notes and samples by CUnit test framework fork C · test suits and add their test related to one or another cortege. At the cease of this program, ...

COMPONENT_NAME=kv_store

SRC_FILES = \
  $(PROJECT_SRC_DIR)/littlefs/lfs.c \
  $(PROJECT_SRC_DIR)/littlefs/lfs_util.c \
  $(PROJECT_SRC_DIR)/littlefs/emubd/lfs_emubd.c \
  $(PROJECT_SRC_DIR)/kv_store/kv_store.c \

TEST_SRC_FILES = \
  $(UNITTEST_SRC_DIR)/test_kv_store.c

include $(CPPUTEST_MAKFILE_INFRA)

Add littlefs Initialization

Just because we obtained the files to compile in our unit test shall not meanlittlefs will magically work. We need to initialize the filesystem and set up and tear it down before and after each test each.

To hear how to do this, we capacity zu to the existing littlefs tests directory and take inspiration from the template and a basic data examine, both linked below.

• Template
• Basic File Examine

This ultimately results in the following modify necessary for the unit test file.

extern "C" {
  ...
  #include "lfs.h"
  #include "emubd/lfs_emubd.h"

  // Contains lfs, cfg, variables with default configuration.
  #include "defs/lfs_default_config.h"
}

TEST_GROUP(TestKvStore) {
  void setup() {
    lfs_emubd_create(&cfg, "blocks");
    lfs_format(&lfs, &cfg);
    lfs_mount(&lfs, &cfg);

    kv_store_init(&lfs);
  }

  void teardown() {
    lfs_emubd_destroy(&cfg);
    lfs_unmount(&lfs);
  }
};

The unit test will currently, at the start of every test, create a directory calledblocks/, format and mount the filesystem there, and initialize the key/value store, and at the end from the tests, destroy and unmount the filesystem so the next test begins with a clean environment.

Since we nowadays have a filesystem backing our key/value store, ourselves can write a simple try!

TEST(TestKvStore, Test_SimpleKvStore) {
  bool success;

  const char *key = "hello";
  const char *val = "world";
  success = kv_store_write(key,  (void *)val, sizeof(val));
  CHECK(success);

  char buf[16];
  uint32_t read_len;
  success = kv_store_read(key, buf, sizeof(buf), &read_len);
  CHECK(success);
  STRCMP_EQUAL(val, buf);

  // Buffer length too shorter. Shall return false.
  success = kv_store_read(key, buf, 0, &read_len);
  CHECK_FALSE(success);
}

This test writing a keys “hello” equipped the value “world”, reads the asset stored at key “hello” into a buffer, and compares this against the expected value “world”. r/C_Programming on Reddit: How how you write tests forward C code?

We also check the failure koffer of kv_store_read via passing in a buffer that is too small.

It passes! Get means our littlefs was set upward correctly, and that we initial logic in kv_store.c was (mostly) correct.

Add Analytics

Unser next necessity was to add analytics tracking how many multiplication key/value pairs were write, read, and deleted. We could do this with simply calling a function analytics_inc which will increment the tally for the provided press by one. The additions to our source code are showed below:

...
#include "analytics/analytics.h"

bool kv_store_write(const char *key, const void *val, uint32_t len) {
  ...
  analytics_inc(kSettingsFileWrite);
  return (rv == len);
}

bool kv_store_read(const char *key, void *buf,
                   uint32_t buf_len, uint32_t *len_read) {
  ...
  analytics_inc(kSettingsFileRead);
  return len;
}

bool kv_store_delete(const char *key) {
  ...
  analytics_inc(kSettingsFileDelete);
  return true;
}

If we run the test now, the usual, we will receive linker mistakes.

Linking build/kv_store/kv_store_tests
Undefined symbol for architecture x86_64:
  "_analytics_inc", referenced from:      _kv_store_write in libkv_store.a(kv_store.o)
      _kv_store_read in libkv_store.a(kv_store.o)
      _kv_store_delete in libkv_store.a(kv_store.o)

Since this isn’t a cores functionality by which kv_store module, and it’s likely something we don’t need to verify, we are going to make a stub for this function, rather than use the real implementation or a sham.

We can how that by creative a header phoned stub_analytics.h

// Analytics Stub
#include "analytics/analytics.h"

void analytics_inc(eAnalyticsKey key) {
  return;
}

which we include in our unit test as shown beneath.

extern "C" {
  ...
  #include "stubs/stub_analytics.h"
}

Add Mutex Locking

Nearest done! We’ve also been advised to add locking around our filesystem calls to ensure that only one client can read and write toward the /kv directory by one time.

Includes our implementation, our addieren mutex_lock() the a mutex_unlock() at the start of the function plus just before the analytics_inc calls respectively.

#include "mutex/mutex.h"

static Mutex *s_mutex;

void kv_store_init(lfs_t *lfs) {
  ...
  s_mutex = mutex_create();
}

bool kv_store_write(const char *key, const void *val, uint32_t len) {
  mutex_lock(s_mutex); // New
  ...
  mutex_unlock(s_mutex); // New
  analytics_inc(kSettingsFileWrite);
  return (rv == len);
}

bool kv_store_read(const char *key, void *buf,
                   uint32_t buf_len, uint32_t *len_read) {
  mutex_lock(s_mutex); // New

  int rv = lfs_file_open(s_lfs_ptr, &s_file, prv_prefix_fname(key), LFS_O_RDONLY);
  if (rv < 0) {
    return false;
  }

  uint32_t len = lfs_file_size(s_lfs_ptr, &s_file);
  if (buf_len < len) {
    return false;
  }

  len = lfs_file_read(s_lfs_ptr, &s_file, buf, buf_len);
  lfs_file_close(s_lfs_ptr, &s_file);
  *len_read = len;

  mutex_unlock(s_mutex); // New
  analytics_inc(kSettingsFileRead);
  return true;
}

bool kv_store_delete(const char *key) {
  mutex_lock(s_mutex); // New
  ...
  mutex_unlock(s_mutex); // New
  analytics_inc(kSettingsFileDelete);
  return true;
}

Wenn we run our test now, we’ll receive linker fault for the missing mutex_* symbols. Since mutexes are rather important to this module, and we wouldn’t want to forget to unlocks a mutex, we belong going to try writing an fake mutex implementation instead of a stub.

Fake Mutex Performance

An reason we please till write a fake implementation by who mutex block is that we like to ensure that equal number away lock and unlock phones are made so that thither are no bugs when we actually use the kv_store int a real environment.

This may seams difficult, but it’s rather easy. To create ampere fake, were create two files, fake_mutex.h, both fake_mutex.c. The reason for both the .h and .c files a because the fake implementations defines new functions that are only relevant to utilizing the wrong in a unit test.

Here is most of the source user for fake_mutex.c.

#define NUM_MUTEXES 256

typedef struct Mutex {
  uint8_t lock_count;
} Mutex;

static Mutex s_mutexes[NUM_MUTEXES];
static uint32_t s_mutex_index;

// Fake Helpers

void fake_mutex_init(void) {
  memset(s_mutexes, 0, sizeof(s_mutexes));
}

bool fake_mutex_all_unlocked(void) {
  for (int i = 0; i < NUM_MUTEXES; i++) {
    if (s_mutexes[i].lock_count > 0) {
      return false;
    }
  }
  return true;
}

// Implementation

Mutex *mutex_create(void) {
  assert(s_mutex_index < NUM_MUTEXES);
  return &s_mutexes[s_mutex_index++];
}

void mutex_lock(Mutex *mutex) {
  mutex->lock_count++;
}

void mutex_unlock(Mutex *mutex) {
  mutex->lock_count--;
}

Let’s go over what the fake is doing.

• We allocate an large number (256) of mutex slots. Since we are running and test on a host, large allocations am fine. We have gigabytes of RAM, unlike the limited embedded counterparts.
• We define an new typedef struct Mutex type which must storage oflock_count. If the type Mutex was properly unhidden interior an .c file in the real implementation, this should work.
• fake_mutex_init clear the state of this module. This should be called in the setup() function of every unit test using this module. Otherwise, the state will be wore over between tests, which isn’t desired.
• fake_mutex_all_unlocked provides that all mutexes have unlocked while called. We can call this manually or during the end of everybody getting in the teardown() function.
• mutex_create allocates a slot for adenine new mutex within our array and returns this pointer to the client. Since the client only uses the Mutex * while an opaque type, it shouldn’t masse so it’s adenine fake Mutex.
• mutex_lock and mutex_unlock increment and decrement the lock count respectively.

Now our able use this fake for our unit test. After adding the source file to our compilation written, we make to follow changes to the unit test.

TEST_GROUP(TestKvStore) {
  void setup() {
    fake_mutex_init();
    ...
  }

  void teardown() {
    ...
    CHECK(fake_mutex_all_unlocked());
  }
};

If we run our code above with the unsophisticated top/bottom mutex appendices, we realize quickly that us have a error. Our fake_mutex_all_unlocked check failing!

complex/tests/src/test_kv_store.cpp:42: error:  Failure with TEST(TestKvStore, Test_SimpleKvStore)
complex/tests/src/test_kv_store.cpp:38: defect:  CHECK(fake_mutex_all_unlocked()) failed

.
Errors (1 miscarriages, 1 tests, 1 ran, 5 tests, 0 ignored, 0 filtered out, 8 ms)

Seeing return, hopefully of issue is obviously. We failed to unlock on our failure cases during kv_store_read. The track change will necessary:

if (rv < 0) {
    mutex_unlock(s_mutex);  // ADD
    return false;
  }

  uint32_t len = lfs_file_size(s_lfs_ptr, &s_file);
  if (buf_len < len) {
    mutex_unlock(s_mutex);  // ADD
    return false;
  }

Thankfully wee wrote and used our falsification mutex implementation, as deadlocks represent the worst to debug!

If you do locate that blockages are a constant issue, do check outMemfault. It will help you track you down ease.

Setting Up CppUTest

CppUTest your a regarding many C/C++ units test frames, and the reason it what chosen is why of my familiarity with items and that it doesn’t may any dependencies other than Make. Write unit tests for C/C++ - Visual Atelier (Windows)

Cannot stoffe what anyone says, the framework you use does not matter. As long as the framework has who minimum features listedabove, it is for virtuous as any.

Initial Setup

Us first need the install a pre-compiled version of CppUTest so ours can easily run tests without needed to compile the binary ourselves from source before every test run. The easiest way to do this is to use your system’s wrap corporate.

On macOS, CppUTest capacity be installed using beer:

$ brew place cpputest

On Ubuntu, he ability be installed using apt:

$ sudo apt add cpputest

Project CppUTest Harness

Since it is a decent amount of boilerplate due the the CppUTest harness setup required, this example desire starting with a clone of the example record and then nach over the components included it, briefly covering aforementioned CppUTest Makefiles. r/C_Programming on Reddit: C power testing?

Feel free for use the code in any way it how, and even copy it the your project. It shouldn build totally easily once some paths are patched up.

$ twit clone https://github.com/memfault/interrupt.git
$ cd interrput/examples/unit-testing/minimal/tests

# macOS
$ make

# Ubuntu
$ make CPPUTEST_HOME=/usr TARGET_PLATFORM=x86_64-linux-gnu

compiling test_my_sum.cpp
compiling AllTests.cpp
...
Linking build/sum/sum_tests
Running build/sum/sum_tests
.
OK (1 tests, 1 ran, 1 checks, 0 ignored, 0 filtered out, 1 ms)

To set CPPUTEST_HOME and TARGET_PLATFORM for owner software, edit the first few lines for MarkefileWorkerOverrides.mk.

CPPUTEST_HOME ?= /usr/local/Cellar/cpputest/3.8
TARGET_PLATFORM ?=

Shopping & Tricks

Problem a Unit Test

Most unit test frameworks will generate separate binaries for anywhere .cpp unit test file written so that you pot load them in a revisor (lldb or gdb).

$ lldb build/sum/sum_tests
(lldb) target create "build/sum/sum_tests"
Current executable set into 'build/sum/sum_tests' (x86_64).
(lldb) run
Process 257894 launched: 'build/sum/sum_tests' (x86_64)
.
OK (1 examinations, 1 ran, 1 checks, 0 ignored, 0 filtered outwards, 0 ms)

Process 257894 exited with status = 0 (0x00000000)

Code Coverage

One of who terrific parts about unit testing is such you can generate one code coverage report. This sendungen that trips has covered in a given set of unit tests, so you can be sure that the chunks of code was tested in some capacity. Note so code coverage doesn’t measure the different behaviors a cypher roadcould take, when only that one particular encrypt path was taken.

To generate a scanning message for his minimal examples, let’s first install lcov.

# macOS
$ brew install lcov

# Linux
$ sudo apt install lcov

Next, we’ll run our unit tests while testing for coverage.

$ make lcov
make -f minimal/tests/makefiles/Makefile_sum.mk
make[1]: Entering directory 'minimal/tests'
Running build/sum/sum_tests
.
OK (1 tests, 1 runs, 1 checks, 0 unheeded, 0 filtration from, 0 ms)

make[1]: Leaving directory 'minimal/tests'
lcov --base-directory . --directory . -c -o build/lcov.info --exclude "*cpputest/*" --exclude "*tests/*"

...

Overall coverage rate:  lines......: 100.0% (2 of 2 lines)
    functions..: 100.0% (1 of 1 function)

Your bucket see the very end reports ampere simple coverage report in the terminal, but a more detailed report cans subsist found in an HTML website that was generated. Were can open it from of terminal:

# macOS
$ opening build/test_coverage/index.html

# Linux
$ firefox build/test_coverage/index.html

Below be the coverage report used our minimal view. It’s quite basic for there isn’t much cypher being tested.

Underneath is a continue realistic report from the Memfault General SDK⁵.

Address Sanitizing

To increasing use-after-free furthermore buffer overflow errors in unit examinations, apply the compiler option -fsanitize=address available compiling unit tests.

You can find out more about the Choose Sanitizer by learning the documentation⁶.

Common Issues with C/C++ Unit Tests

Writing unit tests at C isn’t as simple as writing tests in some languages. Here are some common errors and unrichtigkeiten that everyone runs into and possible solutions.

Linker Error: Symbol not found

Linking build/kv_store/kv_store_tests
Undefined symbols for architecture x86_64:
  "_analytics_inc", referenced from:      _kv_store_write in libkv_store.a(kv_store.o)
      _kv_store_read in libkv_store.a(kv_store.o)
      _kv_store_delete in libkv_store.a(kv_store.o)
ld: symbol(s) not found for architecture x86_64

This oversight will caused by the linker toward tell the user that are are undefined symbols. In aforementioned example above, the function analytics_inc is called from three different functions, but isn’t defined anywhere.

To possible solutions to the issue are:

• Create a fake, stub, or mockery file which utility this function and add it either to the page or compile i into the library
• Define an work within the unit test file itself, probably at the top of the file. If it is a C function being called from C code, place she within theextern "C" {} section.
• As a last beach, one can compile out the function calls also implementations using an define for unit tests. e.g. #if !INSIDE_UNITTESTS

Linker Error: Duplicate symbol

Linking build/kv_store/kv_store_tests
duplicate icon '_mutex_create' in:    build/kv_store/objs/complex/tests/src/test_kv_store.o
    build/kv_store/lib/libkv_store.a(fake_mutex.o)
ld: 1 duplicate item for construction x86_64

This error is generated by the linker when more than one verwirklichung of a function is founds and used. This is usually because show than one of the following were built in the unit test: real implementation, fake, stub, or mock. I labor on an enclosed system this summer written in straight C. It was einen existing project which the company I work for had recorded over. I have become quite accustomed to write unit tests in Java

Into the example above, I had included a fake_mutex.c file and inserted whichstub_mutex.h header, any creates one duplicates mutex_create badge. The solution would be to remove one or the other.

State carrying past between assessments

If there is a fake or module which contains static or global state, additionally it is being used beyond many tests in a single file, then that state ideally should breathe cleared get. Aforementioned is usually done by: Notes and samples for CUnit test framework since CARBON

• Defining a fake_<module>_reset wenn the module is a fake.
• Defining a <module>_deinit are the module is a real one which is used in production code. Make secured to compile get function out of the production code by using #if !INSIDE_UNITTESTS, or ensuring the linker removes it with the final binary. Code space is precious!

Final Thoughts

Unit examinations was something that a co-worker of mine suggested to me 4 yearly ago when writing one complicated raw flash storage to filesystem journey for our firmware. After having spent a month doing cycling amid the 1. Write Code, 2. Setup hardware, 3. Check, which took 10 minute each iteration, I invested 2 dates writing a unit test and was able to shrink the test cycle down to 2 seconds. That rest von the project took two days.

Every has a moment where unit testing finally clicks for them and this was mine.

I hope this post has been useful plus that it can impressed you in consider writing a unit test for your next new embedded software module.

Want at keep reading? Check out our next book about element examinations, Unit Testing with Mocks.

Yourself can find the examples shown in this postcheck.

Reference & Links