Converting BASH scripts to use XML Configuration files
prowl [-s] <bash.script>
The source for this software can be obtained from
The supporting paper can be obtained from
The prowl program was developed by heavily hacking the open
source code for bash-4.2. All changes to
the original source code are
bracketed by #ifdef
IJD, making it easy to both identify changes and to compare this code to the
original bash-4.2 code.
Powl exploits all of the pre-existing bash functionality to
read and parse an arbitrary bash shell script, ignore comments, and to source
other referenced bash shell scripts, invoke functions, etc.
In particular it assigns values to variables
just as bash would, expanding as necessary other variables as part of this
It differs from bash-4.2 in not invoking either external
programs or built in bash commands (with the exception of the built in source
command) thus ignoring bash statements such as return and exit, and in
following all control paths within the bash script exactly one.
It thus executes both the
if and the else block within each if statement, and every possible clause
specified within a case statement. It
executes while and until blocks exactly once irrespective of the result
returned by the corresponding test. The
intent is to ensure that every logical line of a bash script is at least
visited once by prowl.
During the execution of prowl the bash variables are
internally annotated with flags that indicate the potential usage made of each
variable, and thus the potential relevance of deeming the variable externally
configurable with a user specified value.
These flags (currently) are:
USED (Some logical use is made of this variable. IE. at some point the bash script looks up this variables value)
ASSIGNED (This variable is assigned a value using the assignment command)
CHANGED (This variable assumes multiple distinct values when examined using prowl)
EXIT (This variable is used with a return or an exit statement)
ENV (This variable is initially duplicated from an external run time environment variable)
SOURCED (This variable is only used in one or more of the sourced shell scripts)
DERIVED (This variable is assigned a value computed in part from other variables)
LOOP (This variable is assigned a value within a looping construct)
FUNC (This variable is assigned a value within a function)
CONFIG (This variable is assigned using prowler)
BADEARLY (Variable is used before being assigned a value by prowler)
BADLATE (Variable is assigned a new value after being set by prowler)
Other flags might easily be added as future requirements dictate.
At end of run those variables marked as USED by prowl are emitted (on standard
output) in an XML format to a prototypical configuration file.
The above flags are expressed as cumulative XML
attributes within the output XML file as follows:
LOOP then loop='y'
Not ASSIGNED then tested='y'
This variable may never have been involved in
an assignment command as consequence of having been using as the argument of a
for statement, or may potentially be a configuration value that is tested to
see if it exists, but is not assigned any default value when it does not.
CHANGED then changed='y'
Within the execution across all possible
paths within prowl this variable assumed multiple distinct values.
Typically such variables would not be
candidates to be considered a configuration parameter since configuration
parameters must assume a constant value.
However, a sloppy coding style within scripts is to add alternative
values for a configuration parameter and not to comment out earlier now
incorrect values for this parameter.
EXIT then exit='y'
This variable was used as the return code of
a bash return or exit statement. Such
variables would not normally be deemed configuration parameters since their
primary function is to assign meaningful names to internal return code values
ENV then env='y'
This variable was assigned its initial value
from the corresponding named external environment variable.
Typically such variables would only be
candidates for being treated as configuration parameters
if one was already using the run time environment within which a script was
executed to determine how a script should be configured at execution time.
SOURCED then sourced='y'
This variable was referenced in scripts
sourced by the initial bash script, but not used in this main bash script.
If the main script is performing most of the
real work, it may be that the variable is not being used, and can be removed
from further consideration. However, it
is always possible that the core scripting code is itself contained within
sourced bash scripts.
The list of sourced scripts in which this variable is referenced if any
DERIVED then derived='y'
Somewhere within the bash script this variable
is assigned a value that is derived (in part) from the current value of other
bash variables. Such variables are
typically strictly internal to the bash script and so should not be considered
to be configuration parameters. However,
it is possible that a script becomes more general by permitting
such derived values to be assigned initial values determined by a configuration
CONFIG then config='y'
Somewhere within the bash script this
variable is assigned a value within a for, while or until loop construct.
CONFIG and !TESTED then unused='y'
This variable is assigned a value by prowler
BADEARLY then BADEARLY='y'
This configuration variable is never used
BADLATE then BADLATE='y'
This configuration variable is tested before being set by prowler
Thus the most obvious
configuration parameters within the output XML file format will be those that
have no qualifying attributes, while those least likely to be legitimate
configuration parameters will those having any of the attributes
env='y', derived='y' or exit='y'.
Those remaining which have attributes need to
be independently examined within the bash script to determine the
appropriateness of permitting end user configuration of their initial values.
This configuration variable is assigned a value after being set by prowler
If the -s option is specified
then the output is shortened to show only those variables most likely to be
deemed configuration variables. Specifically variables that having any of the
following flags are not then printed:
It is recommended however that
the -s option is generally avoided since it is safer to manually remove
potentially invalid configuration variables, than to never see variables that
may still potentially be considered configuration variables.
The XML File Format
The XML file format involves 4 layers of hierarchy:
The outermost root entity is labelled <prowl> and confirms the XML file is a prowl
The next entity level identifies the
application layer, and may be repeated to support multiple distinct
applications within a single XML file if this is deemed desirable.
This entity has an optimal attribute version,
naming the default configuration for the application if none is provided.
The next entry level is repeated with
differing tags matching arbitrary version/configuration options associated with
the outer named application. Distinct
configuration versions of an application will typically describe the same
parameters but with differing values.
Within each application's version are parameters
identified by entity tag name, and parameter values identified by the content
enclosed with such tags. Parameters may
have attributes as described above. They
occur in ascending name order.
prowler <XML file> <application> [<version>] [<name>=<value>]*
Prowler scans the specified XML file, identifies the section of
the configuration file matching the application, and if specified the version
of configuration values required within this application.
If not specified the default version as
specified in the file, or the first if not specified
is scanned. Each parameter and
associated value is echoed on standard output, in a format suitable for
inclusion within bash.
When generating the XML file prowl escapes special HTML characters
such as '&','<' and '>'.
This escaping is reversed by prowler, and quotes within values
themselves escaped appropriately.
Assignments specified on the prowler command line for named variables present in
the XML configuration file override the XML assignments. To pass script arguments to prowler, either present them to prowler when and where appropriate or use
the bash $@ macro to copy all of the arguments presented to the script to
To assign the correct configuration parameters and their values
within an arbitrary bash script simply back quote the above invocation of
prowler within bash and precede this expansion of the prowler output into the
bash script through the interpretation of the back quotes, with the bash built
in command eval,
thus causing the variable assignments to be applied within the bash script.
eval `prowler <XML file> <application> <version>`
This should retrieve the XML parameters and their values for the
named version of the application within the XML file, and assign them within
the bash script. The env
command will then print them all out.
Because python is more strongly typed than bash, care is now taken to ensure
that integer and float values are not quoted in the output from prowler. If
all values returned by prowler should be quoted they can subsequently be cast
to strings by using the str(...) python cast mechanism.
The XML configuration files offer a number of useful additional
capabilities. Since the variables are
sorted one can easily:
Compare distinct configuration files for similarities and differences
Compare variable usage to documented usage
Use knowledge about variable usage to improve scripts
the XML configuration files to allow C programs to also obtain parameterization
through the same mechanism as scripts
Document variables within the XML file in XML comments
Log the values generated by prowler
For more information on PROWL please contact us at