Simple test framework in Bash

Bash allows for basic lookup table (associative arrays) to be created. While this aspect is not very well known, it is powerful enough to build some simple test framework.

The context

Suppose we have a Linux based (embedded Linux) host talking with some peripherals thru a set of predefined commands. For example, a Linux powered host talking to a DSP audio codec via ALSA commands. Each command needs to be tested exhaustively for non-regression.

As the developer implements the commands both on the host and the DSP, we also define tests scripts to validate them. For example, to validate a playback kind of function, we just send an audio file via an “aplay” command and expect the sound to be played correctly. More commands implemented means more tests scripts being written.

We want these tests to be run automatically and a nice report being displayed, highlighting what passed and what failed. For doing so, we can define several lists of commands: unitary tests commands, system tests, etc … and go along these lists to execute the associated tests.

This can be easily perform with the “lookup table” functions of bash.

Bash table (arrays) functions

Associative arrays are necessarily created with the “declare” command. The “declare” command is as follows (extract from the bash man)

declare -A -g <table_name>=(['key']='value-string' ['key']='value-string' ['key']='value-string' ...)

Here I use the -A variant (uppercase A, as lowercase has a different meaning) for associative array, -g meaning global scope. To access various elements or properties of this array we can use (beware, the syntax is ugly …)

Accessing one specific element: ${table_name[‘key_string’]}
All the keys of the table: ${!table_name[@]}
Accessing the looked-up value (i.e. position 2): table_name[“${2}”]

Now we are armed with this magic command (and ugly syntax), let us define:

A Look up table for all the test commands to be executed
A look up table to store the results
Some test commands handlers

Lookup table for the tests list and the tests results

declare -gA fct_table=(['Test 1']="do_test1"
 ['Test 2']="do_test2"
 ['Test 3']="do_test3"
 ['Test 4']="do_empty_handler"
 ['Test 5']="do_empty_handler"
 ['Test6']="do_empty_handler"
 )

declare -gA res_table=(['Test 1']="N/A"
 ['Test 2']="N/A"
 ['Test 3']="N/A"
 ['Test 4']="N/A"
 ['Test 5']="N/A"
 ['Test 6']="N/A"
 )

After each test, the corresponding result placeholder will be updated with a meaningful string. The default is “N/A” to cover not yet implemented tests.

Test commands handlers

We define a trivial empty handler for non-implemented tests, and real handlers for all the others. Of course, each handler has to be filled with meaningful commands 🙂

function do_empty_handler {
   echo "Test Not Implemented: $1"
   return 2
 }

function do_test1 {
   echo "Test 1"
   ret=0
   return $ret
 }

function do_test2 {
   echo "Test 2"
   ret=0
   return $ret
 }

function do_test3 {
   echo "Test 1"
   ret=0
   return $ret
 }

Functions to update and print the results

Here we use specific ASCII sequences to trigger a nice colored output. Green for passed, Bold Red for fail and black for N/A. This is only to get an eye-catching display

PASS_STR="\e[1;32mPASS\e[0m"
FAIL_STR="\e[1;31;4mFAIL\e[0m"
NA_STR="\e[1;34;4mN/A\e[0m"

# $1 the result value
# $2 the test index
# $3 the table
function __update_result_table {
  local -n __table
  __table=$3

  if [[ $1 -eq 0 ]]; then
    echo ">>>>> SUCCESS"
    __table["${2}"]=$PASS_STR
  elif [[ $1 -eq 1 ]]; then
    echo ">>>>> FAIL"
    __table["${2}"]=$FAIL_STR
  else
    echo ">>>>> N/A"
    __table["${2}"]=$NA_STR
  fi
 return 0
 }

function print_result {
 # printing the results
 echo TEST RESULTS:
 echo ============
 for i in "${!res_table[@]}"; do
 printf "Test: $i Result: ${res_table[$i]}\n"
 done
 }

Worker functions

Here we just go thru the table, execute the tests and print the results. This is very simple processing and it can obviously be enriched at will.

function validate_system {
  for i in "${!fct_table[@]}"; do
    echo "Test : $i"

    ${fct_table[$i]}
    res=$?

    __update_result_table $res "$i" res_table
  done
 }

# Main entry point
validate_system
print_result

Conclusion

This simple test framework was heavily used for a commercial product development. It is very light and easily extensible, yet powerful enough to be integrated in a bigger automation as a first “smoke test”.