Parseargs Tutorial

Parseargs is a command line option parser for shell scripts.

This tutorial explains the features of Parseargs using examples.

The first section POSIX Shells treats all functionality that works in a POSIX compliant shell. Also zsh is not POSIX compliant (and never wanted to be), the mentioned features should run with it.

The second section Extended Shells handles those few features that need functionality beyond the POSIX standard. Currently there is only on feature, that requires the support of array variables. The shells bash, ksh and zsh provide this. These shells can be enabled by using the option -s / --shell with the shell name.

When ksh is mentioned in this document, we are talking about ksh92 or newer. Parseargs was never tested with ksh88.

1. POSIX Shells

1.1. Simple Usage

The following script should support two options and arguments. The options are:

-l to enable "long" output.
-o <filename> to define a file to write to

Script example.sh

1
2
3
4
5
6
7
8
9 #!/bin/sh

eval "$(parseargs -n example.sh -o 'l#long_output,o=outfile' -- "$@")" || exit 1

if [ -n "$long_output" ]; then
    echo "Long output is enabled"
fi
echo "Output file: '$outfile'"
echo "Arguments: $*"

Before testing lets look at the options and arguments given to Parseargs.

-n example.sh

This tells Parseargs the name of the calling script. The name is used as prefix for error messages.

-o 'l#long_output,o=outfile'

This defines two options, separated by a comma.

l#long_output: This definition defines a simple flag. The # in the middle is the marker for a flag. The part before defines the option character. Here l defines the option -l. The part after the # is the variable that should be set, when -l was found on the command line. The variable gets the value "true" assigned.
o=outfile: This definition defines a assignment option. A assignment option needs an additional argument. The = in the middle is the marker for a assignment. Again the part before defines the option. Here -o. The part after the marker is the name of the variable. This variable gets the option argument assigned.

--

The double dash separates the Parseargs options from the shell script options.

"$@"

This is replaced by the script options. Always use exactly this notation. A simple $@ (without the quotes) or $* might fail. See your shell documentation.

The option character may be any ASCII character, except for - (minus), whitespace or control characters.

The characters #, %, +, :, =, \ and , have to be escaped with a backslash.

The trailing || exit 1 is just a safety net, in case something goes wrong during eval.

Let’s test the script:

$ ./example.sh -l -o out.file in.file other.file
Long output is enabled
Output file: 'out.file'
Arguments: in.file other.file

No space needed between -o and the file name:

$ ./example.sh -l -oout.file in.file other.file
Long output is enabled
Output file: 'out.file'
Arguments: in.file other.file

And -l and -o can be combined into one:

$ ./example.sh -loout.file in.file other.file
Long output is enabled
Output file: 'out.file'
Arguments: in.file other.file

Options and argument order is not relevant:

$ ./example.sh -l in.file other.file -oout.file
Long output is enabled
Output file: 'out.file'
Arguments: in.file other.file

Using a unknown option:

$ ./example.sh -X
example.sh: Unknown option: '-X'

Duplicate options are not allowed:

$ ./example.sh -o out.file -o other-out.file  in.file
example.sh: Duplicate option: '-o'

$ ./example.sh -l -l -o out.file  in.file
example.sh: Duplicate option: '-l'

How does this work?

To understand how Parseargs works, it can simply be called from the command line.

$ parseargs -n example.sh -o 'l#long_output,o=outfile' -- -l -o out.file in.file other.file
long_output='true';
outfile='out.file';
set -- 'in.file' 'other.file'

First the potentially used variables are defined. As -l is given, the variable assignment long_output='true' is generated. And due to -o out.file the assignment outfile='out.file is added. Finally with set — … the positional parameter ($1, $2 …) are assigned.

Here is what happens, when an unknown option is found:

$ parseargs -n example.sh -o 'l#long_output,o=outfile' -- -X
example.sh: Unknown option: -X
exit 1

The error message "example.sh: Unknown option: -X" is written to STDERR.

Or here the argument for the option -o is missing:

$ parseargs -n example.sh -o 'l#long_output,o=outfile' -- -o
example.sh: Missing argument for: -o
exit 1

Note that the error messages are printed to STDERR by parseargs. Only exit 1; is printed to STDOUT and hence evaluated by eval when used in a script.

Just play around with Parseargs. Use arguments with spaces or special character.

1.2. Long Options

A lot of programs support additional long forms of options. Like -l and --long. Parseargs also supports this:

Script long-opt.sh

 1
 2
 3
 4
 5
 6
 7
 8
 9
10 #!/bin/sh

eval "$(parseargs -n long-opt.sh -o 'l:long#long_output,o:out-file=outfile' -- "$@")" || exit 1

if [ -n "$long_output" ]; then
    echo "Long output is enabled"
fi
echo "Output file: '$outfile'"
echo "Arguments: $*"

Now we have two colon-separated options before the type marker (#, =). If a option is a single character, it defines a short option (l → -l). With multiple characters it is a long option, that has two leading dashes (long → --long).

Long options may contain any ASCII character, except for =, whitespace or control characters. The - (minus) is not allowed as the first character.

The characters #, %, +, :, \ and , have to be escaped with a backslash.

Now our example script enables long output by either using -l or --long and the output file can be set with -o out.file or --out-file out.file or even --out-file=out.file.

Again some tests:

$ ./long-opt.sh --long --out-file out.file in.file other.file
Long output is enabled
Output file: 'out.file'
Arguments: in.file other.file

$ ./long-opt.sh --long --out-file=out.file in.file other.file
Long output is enabled
Output file: 'out.file'
Arguments: in.file other.file

Duplicate option detection still works:

$ ./long-opt.sh --long -l
long-opt.sh: Duplicate option: '-l/--long'

Long Options and Optional Arguments

With long options an optional argument is supported for flag options. This optional argument is directly appended to the option with a =. The values true and yes are interpreted as boolean true and false and no as false. The check is case insensitive.

So, to reuse the example above:

$ ./long-opt.sh --long=true --out-file=out.file in.file
Long output is enabled
Output file: 'out.file'
Arguments: in.file

$ ./long-opt.sh --long=yes --out-file=out.file in.file
Long output is enabled
Output file: 'out.file'
Arguments: in.file

$ ./long-opt.sh --long=false --out-file=out.file in.file
Output file: 'out.file'
Arguments: in.file

$ ./long-opt.sh --long=no --out-file=out.file in.file
Output file: 'out.file'
Arguments: in.file

$ ./long-opt.sh --long=anything --out-file=out.file in.file
long-opt.sh: Invalid boolean value: 'anything'

By the Way ….

It is possible to define multiple short and long options.

1 eval "$(parseargs -n long-opt.sh -o 'l:long:D:detailed#long_output,...' -- "$@")" || exit 1

Now -l, --long, -D and --detailed all would enable long output. I don’t know how useful this is, but it is possible.

1.3. Counting Options

Tools sometimes have an option to increase verbosity of the output. Example from the ssh man page:

-v      Verbose mode.  Causes ssh to print debugging messages about its
        progress.  This is helpful in debugging connection, authentica‐
        tion, and configuration problems.  Multiple -v options increase
        the verbosity.  The maximum is 3.

Parseargs has an own option type to support this. A "Counting Option" is defined using the marker +.

The following script only supports the options -v and --verbose.

Script verbosity.sh

1
2
3
4
5 #!/bin/sh

eval "$(parseargs -n verbosity.sh -o 'v:verbose+verbosity' -- "$@")" || exit 1

echo "Verbosity: $verbosity"

$ ./verbosity.sh
Verbosity: 0

$ ./verbosity.sh -v
Verbosity: 1

$ ./verbosity.sh -vvv
Verbosity: 3

$ ./verbosity.sh -vvvvvvv
Verbosity: 7

The long option form additionally supports a optional argument:

$ ./verbosity.sh --verbose
Verbosity: 1

$ ./verbosity.sh --verbose -v
Verbosity: 2

$ ./verbosity.sh --verbose=5
Verbosity: 5

$ ./verbosity.sh --verbose=full
verbosity.sh: Not a valid count value: "full"

The long form with optional argument sets the verbosity, it does not increase it by the given number.

$ ./verbosity.sh -vv --verbose=5
Verbosity: 5

1.4. Mode Switch Options

A Mode Switch Options are not a new option type, but a extension of a simple flag. Mode switches use one variable with different options and assign different values to the variable.

A simple example would be whether something should be copied or moved. In that case the option -c would request to copy and -m would request move.

The definition of such options look like normal flags, but have a equal sign and a value appended.

Script mode-switch.sh

1
2
3
4
5 #!/bin/sh

eval "$(parseargs -n mode-switch.sh -o 'c:copy#mode=copy,m:move#mode=move' -- "$@")" || exit 1

echo "Mode: $mode"

And here some tests:

$ ./mode-switch.sh -c
Mode: copy

$ ./mode-switch.sh -m
Mode: move

$ ./mode-switch.sh -cm
mode-switch.sh: Options are mutual exclusive: -c/--copy, -m/--move

1.5. Required Options

Sometimes a option might be required. Parseargs supports this with a asterisk before the variable name.

Script required.sh

1
2
3
4
5 #!/bin/sh

eval "$(parseargs -n required.sh -o 'o=*out_file' -- "$@")" || exit 1

echo "Output file: $out_file"

And now a test:

$ ./required.sh -o output.file
Output file: output.file

$ ./required.sh
required.sh: Required option not found: -o

1.6. Showing Help and Version

First up: Parseargs itself does not support creating help texts. But Parseargs can call a existing shell function that prints some help text.

With the Parseargs option -h / --help-opt, the script option --help is supported. If the option is given, the shell function show_help is called and the script is terminated with exit code 0.

To display the script version, the Parseargs option -v / --version-opt enables support for the script option --version. If the option is given, the shell function show_version is called and the script is terminated with exit code 0.

The script must define the named show_* functions. If they are not defined, a error message is displayed and the script is terminated.

Script help.sh

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16 #!/bin/sh

show_help()
{
    echo "Usage: example.sh OPTIONS <input-file...>"
    echo "  -l, --long           enable detailed output"
    echo "  -o, --out-file FILE  file to write result"
}

show_version()
{
    echo "help.sh 1.0"
}


eval "$(parseargs -n help.sh.sh -hv -o 'l:long#detailed,o:out-file=outfile' -- "$@")" || exit 1

Displaying help and version:

$ ./help.sh --help
Usage: example.sh OPTIONS <input-file...>
  -l, --long           enable detailed output
  -o, --out-file FILE  file to write result

$ ./help.sh --version
help.sh 1.0

To better understand how this works, see the sections Using Callbacks and Singleton Options.

1.7. Using Callbacks

Till now we used Parseargs to assign variables for the options found on the command line, but it is also able to work with shell functions.

When using functions, Parseargs also generates code to test for the existence of the function. Assuming a function set_out_file should be used, it is always checked whether this function exists.

1
2
3
4
5 # default
if ! LC_ALL=C command -V set_out_file 2>/dev/null | head -n1 | grep function >/dev/null; then echo >&2 "ERROR: Function 'set_out_file' does not exist."; exit 127; fi;

# with --shell bash, ksh or zsh
if ! typeset -f set_out_file >/dev/null 2>&1; then echo >&2 "ERROR: Function 'set_out_file' does not exist."; exit 127; fi;

This code will exit the calling script if the function does not exist. The check is always done, whether the function is needed in the actually generated code or not.

When calling the callback the exit status of the function must be zero else the calling script is terminated with function exit code. The code for this looks like this:

1 set_out_file 'output.file' || exit $?

1.7.1. Callbacks for Options

Instead of assigning variables for options, it is also possible to call a function. By adding () to the name, it defines the function to call.

Script option-cb.sh

1
2
3
4
5
6
7
8
9 #!/bin/sh

set_long() { echo "set_long($1)"; }
set_outfile() { echo "set_outfile($1)"; }
set_verbosity() { echo "set_verbosity($1)"; }

eval "$(parseargs -n option-cb.sh -o 'l:long#set_long(),o=set_outfile(),v+set_verbosity()' -- "$@")" || exit 1

echo "Arguments: $*"

Testing:

$ ./option-cb.sh -v -l -o out.file -vv input
set_verbosity(1)
set_long(true)
set_outfile(out.file)
set_verbosity(3)
Arguments: input

$ ./option-cb.sh --long=false input
set_long()
Arguments: input

For counting options, the callback might be called multiple times with the current count value.
For flags it is called with a value 'true'. If the option explicitly is set to false using --option=false, the callback is called with an empty string.
For assignment options the callback is called with the option argument.

Using a callback disables checks within Parseargs.

The duplicate usage of options is not checked and also the duplicate usage of mode-switch options are not detected.

With callbacks you have more control and possibilities, but also more responsibilities.

1.7.2. Callback for Arguments

In the previous sections we have seen callbacks for options, this is also possible for program arguments.

The callback for program arguments is defined with the Parseargs option -a or --arg-callback.

Script args-cb.sh

1
2
3
4
5 #!/bin/sh

set_argument() { echo "set_argument($1)"; }

eval "$(parseargs -n args-cb.sh -a set_argument -o '' -- "$@")" || exit 1

When the argument callback is used, the positional parameters are always empty. So $1 etc are unset.

1.7.3. Callback on Error

Parseargs allows the defition of an error callback. This defines a function that is called before Parseargs emits exit 1 to terminate the calling script.

The following example doesn’t support any options and insults you when you give one.

Script error-cb.sh

1
2
3
4
5
6
7 #!/bin/sh

error_callback() { echo "You did something stupid!"; }

eval "$(parseargs -n error-cb.sh -e error_callback -o '' -- "$@")" || exit 1

echo "OK"

$ ./error-cb.sh
OK

$ ./error-cb.sh -x
error-cb.sh: Unknown option: -x
You did something stupid!
$

1.8. A Script without Options

Parseargs if even useful in scripts that don’t support any options. In that use case it would output an error message when a option is given.

Script no-opt.sh

1
2
3
4
5 #!/bin/sh

eval "$(parseargs -n no-opt.sh -p -- "$@")"

echo "Arguments: $*"

In this script we also use the option -p / --posix, then Parseargs stops looking for options as soon as the first program argument is found.

$ ./no-opt.sh first second
Arguments: first second

$ ./no-opt.sh -X first second
no-opt.sh: Unknown option: -X

$ ./no-opt.sh  first second -X
Arguments: first second -X

1.9. The Option Argument Separator '--'

POSIX defines the -- as a separator between options and program arguments.

Reusing our first script example.sh:

$ ./example.sh -o out.file -X
example.sh: Unknown option: -X

$ ./example.sh -o out.file -- -X
Output file: 'out.file'
Arguments: -X

A second -- is handled as a normal argument:

$ ./example.sh  -o out.file -- -X -- test
Output file: 'out.file'
Arguments: -X -- test

1.10. Initializing Variables

With the option -i / --init-vars the variables can be initialized with their default values. Note, that variables of counting options are always initialized to 0. This is useful, when the script runs with set -u to treat unset variables as error.

Note that this is for variables only. Callbacks are not called.

$ parseargs -n example.sh -o 'l#long,o=outfile,v+verbosity' --init-vars -- -o out.file -l
long='';
outfile='';
verbosity=0;
outfile='out.file';
long='true';
set --

1.11. Singleton Options

A Singleton Option is an option that supersede all other options or arguments on the command line. The typical use for this is a help option. In fact the support for --help is implemented using this.

A option is marked as a by putting a ? in front of the function name (variables are rarely used here). The following is used to support --help

help#?show_help()

The ? tells Parseargs that this is a singleton option and that

only this option should be processed.
all other content of the command line should be dropped. (The content before the --help must still be valid.)
the calling script should be terminated with exit code 0 if the target is a callback.

So this can be easily used to implement additional help options. Maybe --help-storage#?show_help_storage().

This feature can also be used for other things. The following is from a script that is used as an unpacker for arbitrary archives.

1 eval "$(parseargs -hin ERROR -o "l#mode=listMode,L#mode=topLevelList,D#mode=basenameDir,S#mode=singleDir,T#?checkPrograms(),d=tgtDir,v+verbose" -- "$@")" || exit 1

The interesting part is T#?checkPrograms(). This defines the option -T as a singleton option, that calls the function checkPrograms. In the actual script the -T triggers a test whether the needed tools are available and hence which archive types are supported. After this test is completed, the script is terminated. No other actions is performed, independent of other options given on the command line.

2. Extended Shells

This section describes functionalities, that need additional capabilities beyond those defined by POSIX.

2.1. Separating Arguments behind a `--`

Only supported with shells bash, ksh or zsh.

Sometimes it is useful to handle the arguments before and after a -- differently.

The following command monitors the file tutorial.adoc and executes the command make html as soon as a change is detected.

$ when-changed tutorial.adoc -- make html

Here the arguments before the -- are different than the arguments behind it.

Parseargs is able to collect the arguments behind the -- in a shell array and leave the arguments before it as positional parameter ($1…). The option -r / --remainder is used to define the name of the array.

The Parseargs call for this would look like this:

1 eval "$(parseargs -s bash --remainder cmd -h -- "$@")" || exit 1

Assuming the call above, the name tutorial.adoc would be available as $1 and the words make and html as ${cmd[0]} and $cmd[1]}.

Note that in zsh it would be ${cmd[1]} and $cmd[2]}, as arrays in zsh are 1-based.

3. Details

3.1. Support for Different Shells

Parseargs supports generating code for different shells. The following shells are supported:

--shell=sh (the default): With this setting, code for a POSIX compliant shell is generated. This should work with any POSIX compliant shell and with zsh. The option -r / --remainder is not supported.
--shell=bash, --shell=ksh and --shell=zsh: This shells support all features of Parseargs. The code generated is (as of today) identical, except for array initialization, which is different in ksh.

3.2. Parseargs and ShellCheck

ShellCheck is a static code analysis tool for shell scripts. If you don’t use it yet, you really should.

As Parseargs is creating and assigning new variables at runtime, ShellCheck can’t know about them and will complain. For our first example the following output would be created:

$ shellcheck ./example.sh

In example.sh line 8:
echo "Output file: '$outfile'"
                    ^------^ SC2154 (warning): outfile is referenced but not assigned.

For more information:
  https://www.shellcheck.net/wiki/SC2154 -- outfile is referenced but not ass...

The best solution is to initialize default values before calling Parseargs.

Like:

1
2
3
4
5 #!/bin/sh

long_output=
outfile=
eval "$(parseargs -n example.sh -o 'l#long_output,o=outfile' -- "$@")" || exit 1

3.3. Parseargs and Invalid UTF-8

As of today, Parseargs can only handle arguments that are valid UTF-8.

If a invalid UTF-8 character is found, Parseargs will display an error message and exit the calling script.