1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
|
Running the tests
-----------------
- All tests reside in the tests/ directory tree.
- Running the tests is as simple as:
$ ctest
- Alternatively, the test scripts can be run directly:
$ cd tests
$ ./run_format_tests.py
- To run the only the tests associated with language X ("cpp", "java", etc.):
$ cd tests
$ ./run_format_tests.py X
Adding new tests
----------------
- A test is defined by adding a line in one of the tests/*.test
files: you must pick the .test file for the proper programming
language, i.e. pick the X.test file which defines tests which use
inputs in language X.
- A test has a (unique) number (a.k.a. test name) -- it does not have
to be a sequential number, but it might be handy to treat it like
that.
- The X.test entry (line) has this format:
<test.name> <config.file> <input.file>
where the fields are separated by an arbitrary non-zero amount of
whitespace.
- As each test references a config file and an input file (the latter
written in programming language X), these are assumed to exist in
the paths tests/config/<config.file> and tests/<input.file>
respectively. It is _strongly_ advised to put input files in their
own subdirectory, so a 'C' source formatting test entry might look
like this (in 'c.test'):
99902 ger-full-cmt-reflow.cfg c/cmt-not-a-boxed-cmt.c
where 'ger-full-cmt-reflow.cfg' is stored at
tests/config/ger-full-cmt-reflow.cfg and 'c/cmt-not-a-boxed-cmt.c'
will be fetched from tests/c/cmt-not-a-boxed-cmt.c when you run
the tests.
Features
--------
- You can re-use config files and input files for multiple tests. So
one might observe test sets like these:
# clark's style - blank lines before and after flow control,
indented comments:
20100 clark.cfg c/i2c-core.c
20101 clark.cfg c/comment-indent.c
20102 clark.cfg c/indent.cpp
20103 clark.cfg c/output.cpp
and
21051 return-1.cfg c/nl_return_expr.c
21052 return-2.cfg c/nl_return_expr.c
21053 return-3.cfg c/nl_return_expr.c
21054 return-4.cfg c/nl_return_expr.c
- Reference output files (the reference against which the uncrustify
test run output is compared) are to be stored in the path
tests/expected/<X>/<NNN-TTTTT.TTT> where <X> is the 'language
directory' part of the input file, e.g. 'c' for input file
'c/nl_return_expr.c', <NNN> is the test name (number), e.g. '21051',
and <TTTTT.TTT> is the file name part of the input test filespec,
e.g. 'nl_return_expr.c' for input file spec 'c/nl_return_expr.c'.
This means that for test
21051 return-1.cfg c/nl_return_expr.c
the accompanying reference output is:
tests/expected/c/21051-nl_return_expr.c
- When starting out with a new test, you don't need to have a
'reference output' yet: the test will simply be reported as a
'fail'ed test until you do.
- Tip: the easiest way to produce 'reference output' is to copy the
test output (from tests/results/...etc... ) to
tests/expected/...etc... once you've ascertained that those tests
produce the desired (correct) output. The helper script
tests/fixtest.sh will copy the results file to the output folder:
$ fixtest.sh 30014 00110 # copy tests 30014 and 00110
- Tip: the helper script tests/difftest.sh runs 'diff' on the result
and output versions to point out what changed.
- Tip: There are also a few options to run_format_tests.py that can
help. Most interesting is '-d' will run 'diff' if the test fails.
- Tip: If some errors occur with Windows, set the macro variable
NO_MACRO_VARARG to 1 to test some more pointer under Linux.
|
