Add cc-filter line-based filtering script (#221)

* Add perl program `cc-filter`, a general purpose by-line text filter with built-in default rules for filtering text containing C++ types and expressions.
* Add documentation for the tool to the scripts `README.md` file.
* Add demonstration table `filters/massif-strip-cxx` for using `cc-filter` with valgrind massif output.

scripts/README.md

@@ -1,4 +1,4 @@
-#tsplot
+# tsplot
 
 The `tsplot` script is a wrapper around matplotlib for displaying a collection of
 time series plots.
@@ -77,7 +77,8 @@ of the timeseries data.
 Use the `-o` or `--output` option to save the plot as an image, instead of
 displaying it interactively.
 
-#profstats
+
+# profstats
 
 `profstats` collects the profiling data output from multiple MPI ranks and performs
 a simple statistical summary.
@@ -90,10 +91,222 @@ are reported instead.
 
 Output is in CSV format.
 
-#PassiveCable.jl
+
+# cc-filter
+
+`cc-filter` is a general purpose line-by-line text processor, with some
+built-in rules for simplifying output comprising templated C++ identifiers.
+
+Full documentation for `cc-filter` can be obtained by running it with the
+`--man` option. The information below has been transcribed from this output.
+
+In the `filter` subdirectory there is a sample table `massif-strip-cxx`
+that will remove the C++ content from the valgrind massif tool output.
+This can be then be used without running the default rules with the following:
+```
+cc-filter -n -t filters/massif-strip-cxx
+```
+
+## Options
+
+#### **-n**, **--no-default-rules**
+
+Omit the built-in rules from the default list.
+
+#### **-N**, **--no-built-ins**
+
+Omit all rule, group, and macro definitions from the default table.
+
+#### **-r**, **--rule=RULE**
+
+Apply the rule or group of rules **RULE**.
+
+#### **-x**, **--exclude=RULE**
+
+Skip the application of the rule or group of rules **RULE**.
+
+#### **-t**, **--table=FILE**
+
+Add the macro, rule and table definitions in **FILE**.
+
+#### **-d**, **--define=DEF**
+
+Add an explicit definition.
+
+#### **-l**, **--list\[=CAT\]**
+
+By default, list the applicable rules and definitions. If **CAT** is
+`expand`, expand any macros in the definitions. If **CAT** is
+`group`, list the group definitions. If **CAT** is `macro`, list the
+macro definitions.
+
+#### **-h**, **--help**
+
+Print help summary and exit.
+
+#### **--man**
+
+Print the full documentation as a man page.
+
+## Description
+
+Rules are applied sequentially to each line of the input files in turn.
+The rules are taken from the built-in list, and from any rules defined
+in tables supplied by the `--table` option. If the table file is not an
+absolute path, it is looked for first in the current directory, and then
+relative to the directory in which `cc-filter` resides.
+
+The default list of rules comprises all the rules specified in the
+built-in list any supplied table, however no default list is used if a
+rules are specifically requested with the `--rule` option. The built-in
+rules are omitted from the default list if the `--no-default` option is
+given. Rules can be explicitly omitted with the `--exclude` option.
+
+Each line has any terminal newline stripped before processing, and then
+is subjected to each rule's action in turn via `$_`. If a rule introduces a
+newline character, the string is not split for processing by subsequent rules.
+(This is a limitation that may be addressed in the future.) If a rule
+sets `$_` to `undef`, the line is skipped and processing starts anew with
+the next input line.
+
+Tables can include groups of rules for ease of inclusion or omission
+with the `--rule` or `--exclude` options.
+
+For details on the table format and example tables, refer to the full
+documentation provided by `cc-filter --man`.
+
+## Example usage
+
+The rules applied by default can be listed with `cc-filter --list`, e.g.
+
+```
+$ cc-filter --list
+cxx:rm-allocator	s/(?:,\s*)?%cxx:qualified%?allocator%cxx:template-args%//g
+cxx:rm-delete	s/(?:,\s*)?%cxx:qualified%?default_delete%cxx:template-args%//g
+cxx:rm-std	s/%cxx:std-ns%//g
+cxx:rm-std	s/%cxx:gnu-internal-ns%//g
+cxx:rm-template-space	s/%cxx:template-args%/$1=~s| *([<>]) *|\1|rg/eg
+cxx:unsigned-int	s/\bunsigned\s+int\b/unsigned/g
+cxx:strip-qualified	s/%cxx:qualified%//g
+cxx:strip-args	s/(%cxx:identifier%%cxx:template-args%?)%cxx:paren-args%/$1(...)/g
+```
+
+These actions are as follows:
+
+* Remove `allocator<...>` entries from template argument lists (`cxx:rm-allocator`).
+
+* Remove `default_delete<...>` entries from template argument lists (`cxx:rm-delete`).
+
+* Remove `std::` qualifiers (`cxx:rm-std`).
+
+* Remove `__gnu_cxx::` qualifiers (`cxx:rm-std`).
+
+* Collapse spaces between template brackets (`cxx:rm-template-space`).
+
+* Replace occurances of `unsigned int` with `unsigned` (`cxx:unsigned-int`).
+
+* Strip all class or namespace qualifers (`cxx:strip-qualified`).
+
+* Replace argument lists of (regularly named) functions with `(...)` (`cxx:strip-args`).
+
+The rules are grouped, however, so the more invasive transformations can be
+straightforwardly enabled or disabled. The defined groups are listed with
+`cc-filter --list=group`:
+
+```
+cxx:tidy	cxx:rm-template-space cxx:unsigned-int
+cxx:strip-all	cxx:strip-qualified cxx:strip-args
+cxx:std-simplify	cxx:rm-allocator cxx:rm-delete cxx:rm-std
+```
+
+`cc-filter --rule ccx:tidy` would perform only the space and `unsigned` transformations.
+`cc-filter --exclude ccx:strip-all` would leave arguments and non-standard namespace and class
+qualifications intact while applying the other transformations.
+
+One can see in the rule list the use of some in-built macros, such as `%cxx:template-args%`. These
+macro definitions can be listed with `cc-filter --list=macro`:
+
+```
+$ cc-filter --list=macro
+cxx:std-ns	(?:(::)?\bstd::)
+cxx:identifier	(\b[_\pL][_\pL\p{Nd}]*)
+cxx:gnu-internal-ns	(?:(::)?\b__gnu_cxx::)
+cxx:template-args	(<(?:(?>[^<>]+)|(?-1))*>)
+cxx:qualified	(?:(::)?\b(\w+::)+)
+cxx:paren-args	(\((?:(?>[^()]+)|(?-1))*\))
+```
+
+Rule definitions with macros expanded can be displayed with `--list=expand`, e.g.
+
+```
+$ cc-filter --rule cxx:rm-std --list=expand
+cxx:rm-std	s/(?:(::)?\bstd::)//g
+cxx:rm-std	s/(?:(::)?\b__gnu_cxx::)//g
+```
+
+### Built-in rules in action
+
+Consider the following error message generated by g++ (some of the middle lines and the full paths to gcc headers have been elided):
+
+```
+In file included from /.../g++/vector:62:0,
+                 from badvec.cc:1:
+/.../g++/bits/stl_construct.h: In instantiation of 'void std::_Construct(_T1*, _Args&& ...) [with _T1 = long_namespace::bad; _Args = {const long_namespace::bad&}]':
+/.../g++/bits/stl_uninitialized.h:75:18:   required from 'static _ForwardIterator std::__uninitialized_copy<_TrivialValueTypes>::__uninit_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = const long_namespace::bad*; _ForwardIterator = long_namespace::bad*; bool _TrivialValueTypes = false]'
+[...]
+/.../g++/bits/stl_vector.h:379:2:   required from 'std::vector<_Tp, _Alloc>::vector(std::initializer_list<_Tp>, const allocator_type&) [with _Tp = long_namespace::bad; _Alloc = std::allocator<long_namespace::bad>; std::vector<_Tp, _Alloc>::allocator_type = std::allocator<long_namespace::bad>]'
+badvec.cc:10:48:   required from here
+/.../g++/bits/stl_construct.h:75:7: error: use of deleted function 'long_namespace::bad::bad(const long_namespace::bad&)'
+     { ::new(static_cast<void*>(__p)) _T1(std::forward<_Args>(__args)...); }
+       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+badvec.cc:6:5: note: declared here
+     bad(const bad&) = delete;
+     ^~~
+```
+
+Running this message through `cc-filter`:
+
+
+```
+In file included from /.../g++/vector:62:0,
+                 from badvec.cc:1:
+/.../g++/bits/stl_construct.h: In instantiation of 'void _Construct(...) [with _T1 = bad; _Args = {const bad&}]':
+/.../g++/bits/stl_uninitialized.h:75:18:   required from 'static _ForwardIterator __uninitialized_copy<_TrivialValueTypes>::__uninit_copy(...) [with _InputIterator = const bad*; _ForwardIterator = bad*; bool _TrivialValueTypes = false]'
+[...]
+/.../g++/bits/stl_vector.h:379:2:   required from 'vector<_Tp, _Alloc>::vector(...) [with _Tp = bad; _Alloc = ; vector<_Tp, _Alloc>::allocator_type = ]'
+badvec.cc:10:48:   required from here
+/.../g++/bits/stl_construct.h:75:7: error: use of deleted function 'bad(...)'
+     { ::new(...) _T1(...); }
+       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+badvec.cc:6:5: note: declared here
+     bad(...) = delete;
+     ^~~
+```
+
+If we wanted to retain the function call arguments, run with `cc-filter --exclude ccx:strip-args`:
+
+
+```
+In file included from /.../g++/vector:62:0,
+                 from badvec.cc:1:
+/.../g++/bits/stl_construct.h: In instantiation of 'void _Construct(_T1*, _Args&& ...) [with _T1 = bad; _Args = {const bad&}]':
+/.../g++/bits/stl_uninitialized.h:75:18:   required from 'static _ForwardIterator __uninitialized_copy<_TrivialValueTypes>::__uninit_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = const bad*; _ForwardIterator = bad*; bool _TrivialValueTypes = false]'
+[...]
+/.../g++/bits/stl_vector.h:379:2:   required from 'vector<_Tp, _Alloc>::vector(initializer_list<_Tp>, const allocator_type&) [with _Tp = bad; _Alloc = ; vector<_Tp, _Alloc>::allocator_type = ]'
+badvec.cc:10:48:   required from here
+/.../g++/bits/stl_construct.h:75:7: error: use of deleted function 'bad(const bad&)'
+     { ::new(static_cast<void*>(__p)) _T1(forward<_Args>(__args)...); }
+       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+badvec.cc:6:5: note: declared here
+     bad(const bad&) = delete;
+     ^~~
+```
+
+
+# PassiveCable.jl
 
 Compute analytic solutions to the simple passive cylindrical dendrite cable
-model with step current injection at one end from t=0.
+model with step current injection at one end from _t_ = 0.
 
 This is used to generate validation data for the first Rallpack test.