gcc 4.7.2
c89
Hello,
I am wondering does any one know of any tutorials or text books that cover using makefile to create some simple unit testing for my c programs.
I would like to run some automated testing that will create a test suite and add this to my Makefile.
Just want some ideas on how to get started.
Many thanks for any suggestions
Yes indeed, less than 30 lines of makefile you can build a generic unit test engine.
Note that I wrote the following for testing gawk and lisp scripts but it can be easily customized for c. Actually, IMHO, the whole thing is a nice example of the power of shell scripting.
To begin, you place all your tests is executable files in some $Testdir. In this example, all the tests have file names 001 002, etc (with no extension).
Next, you need some set up stuff:
Here=$(PWD)
Testdir=$(Here)/eg
ready: $(Testdir) $(Tmp)
$(Tmp) :
@ - [ ! -d $(Tmp) ] && mkdir $(Tmp)
Then you'll need to collect all the tests into a list called $Tests
Tests:=$(shell cd $(Testdir); ls | egrep '^[0-9]+$$' | sort -n )
(Note the use of :=. The is a slight optimization that builds $Tests once, and uses it many times.)
Each file $(X) in my list of $Tests can be executed in two ways. Firstly, you can just run it. Secondly, you can run it and cache the results in $(X).want.
run : ready $(Testdir)/$(X)
@echo $X 2>&1
@cat $(Testdir)/$(X) | $(Run)
cache : ready
@$(MAKE) run | tee $(Testdir)/$X.want
@echo new test result cached to $(Testdir)/$X.want
I cache a test outcome once the test is ready to go and is producing the right output.
The actual execution is defined by a magic variable called $(Run). This is something you have to write specifically for the language being tested. For the record, I'm testing Gawk scripts so my $(Run) is just as follows (and you can change it to whatever you need).
Run= gawk -f mycode.awk
Once that is done, then to run one test, I just compare what I get after running $(X) to the cached copy:
test : ready $(Testdir)/$(X).want
@$(MAKE) run > $(Tmp)/$X.got
@if diff -s $(Tmp)/$X.got $(Testdir)/$X.want > /dev/null; \
then echo PASSED $X ; \
else echo FAILED $X, got $(Tmp)/$X.got; \
fi
This is how I run all my tests:
tests:; @$(foreach x, $(Tests), $(MAKE) X=$x test;)
You can also do a batch cache of all the current outputs (warning: do not do this unless your tests are currently generating the right output):
cache :
@$(foreach x, $(Tests), $(MAKE) X=$x cache;)
Finally, if you want, you can also get final score of the PASSEDs and FAILEDs:
score :
@$(MAKE) tests | cut -d\ -f 1 | egrep '(PASSED|FAILED)' | sort | uniq -c
That's it: as promised- a generic unit tool in Make in under 30 lines. Share and enjoy.
Generating function snippets is easy with the help of gccxml and nm. The following bash script generate_snippets.sh can be called with one command line argument to generate a test function snippet for each function defined in a source file:
#!/bin/bash
#
# Generate stub functions for one file
#
# # Initialize
FILE=$1
[ ! -e "$FILE" ] && echo "file doesn't exist" && exit -1
# # Compile
OBJECT=$(mktemp)
gcc -c $FILE -o $OBJECT
# # Get Functions
# ## Get all symbols in the text sections
Y=$(mktemp)
nm $OBJECT | grep " T " | awk '{print $3;}' | sort > $Y
# ## Get functions defined in the file (including #includes)
# get all functions defined in the compilation unit of $FILE, excluding included
# dynamically linked functions
XML=$(mktemp)
X=$(mktemp)
gccxml $FILE -fxml=$XML
grep "<Function" $XML | sed 's/^.*name="\([^"]*\).*/\1/g' | sort > $X
# ## get the common lines
# This is done to get those functions which are defined in the source file and end up in
# the compiled object file.
COMMON=$(comm $Y $X -1 -2)
# # Create stubs
for func in $COMMON;
do
cat <<_
// Test stub for $func. Returns 1 if it fails.
char test_$func() {
return 1;
}
_
done
# # Clean up
rm $OBJECT $XML $X $Y
The script is not yet perfect. You should probably include a test to only generate test functions for those functions which aren't tested yet. As this is done analogous to finding the common names between $X and $Y, I leave this as an exercise. When this is implemented, it makes sense to run this script from a Makefile. See the other answer for pointers on that.
Consider the C file hello.c:
#include <stdio.h>
int foo();
int bar();
int main() {
printf("hello, world\n");
return 0;
}
int foo() {
printf("nothing\n");
return 1;
}
int bar() {
printf("still nothing\n");
return 1;
}
Running the script above with this file as input yields the following output:
// Test stub for bar. Returns 1 if it fails.
char test_bar() {
return 1;
}
// Test stub for foo. Returns 1 if it fails.
char test_foo() {
return 1;
}
// Test stub for main. Returns 1 if it fails.
char test_main() {
return 1;
}
Just put those snippets into the appropriate file and fill them with logic as needed. After that, compile the test suite and run it.
printf("not implemented")) is straight forward: create an xml representationoutput.xmlof the source coude usinggccxml, usenmto extract the symbol names of the text code section, get the corresponding symbols fromoutput.xmlto ensure that they are functions, and write the stubs to a file. Adding a test target to the makefile is no different than adding other targets.