Shellscripting: Wildcards, Logging and Debugging

Software Engineering · 2018

This is part four of my post series about Shellscripting, you can check out the previous posts here:

Last time I promised a few words about wildcards, character classes, and about logging and debugging your scripts. So here we are.

Wildcards

A wildcard is a character (or a small pattern) that the shell expands into a list of matching filenames before the command ever runs. This is the part that trips people up at first, so it’s worth saying clearly:

The shell expands wildcards, not the command. By the time ls or rm runs, it never sees the *. It only sees the list of files the shell already matched.

The two you’ll use constantly are * and ?.

* matches zero or more characters.
? matches exactly one character.

Say we have these files:

$ ls
file1.txt  file2.txt  file10.txt  notes.md  image.png

Then:

$ ls *.txt
file1.txt  file2.txt  file10.txt

$ ls file?.txt
file1.txt  file2.txt
# file10.txt is skipped on purpose:
# ? matches exactly one character, and "10" is two

A couple of things that are good to know early, because they cause confusing bugs later:

By default * does not match hidden files (the ones starting with a dot, like .bashrc). You have to ask for those explicitly, e.g. .* or by enabling dotglob.
If a pattern matches nothing, bash leaves it untouched and passes the literal string to the command. So ls *.md in a folder with no .md files actually runs ls with the literal argument *.md, which then errors out. You can change this with shopt -s nullglob, which makes a non-matching pattern expand to nothing instead.

Character classes

Sometimes * and ? are too blunt and you want to match a specific set of characters in one position. That’s what bracket expressions are for.

$ ls [fn]*
file1.txt  file2.txt  file10.txt  notes.md
# everything starting with f OR n

Inside the brackets you can also use ranges:

$ ls file[1-9].txt
file1.txt  file2.txt
# file10.txt doesn't match: [1-9] is a single position,
# but "10" needs two

And you can negate the set with ! (bash also accepts ^):

$ ls [!f]*
image.png  notes.md
# everything that does NOT start with f

On top of plain ranges, there are POSIX character classes, which read a bit nicer and don’t depend on your locale’s alphabet ordering. They go inside the brackets, so you end up with the slightly funny looking double brackets:

$ ls *[[:digit:]].txt
file1.txt  file2.txt  file10.txt
# any name that ends in a digit before .txt

Some of the handy ones:

[[:alpha:]] letters
[[:digit:]] digits
[[:alnum:]] letters and digits
[[:space:]] whitespace
[[:upper:]] and [[:lower:]] for case

A very common source of bugs: shell wildcards are not regular expressions. In a glob, * means “any string” and ? means “any single character”. In a regex, * means “zero or more of the previous thing” and . means “any character”. So the glob *.txt is the regex .*\.txt. Tools like grep, sed, and [[ =~ ]] work with regex; filename matching works with globs. Mixing them up will quietly match the wrong files.

There’s also brace expansion, which looks similar but is a different thing. {a,b,c} and {1..5} generate strings whether or not any matching files exist:

$ echo file{1,2,3}.txt
file1.txt file2.txt file3.txt

$ mkdir -p project/{src,test,docs}

And if you ever need more powerful patterns (matching “one or more of”, “none of”, and so on), bash has extended globbing behind shopt -s extglob. That’s a rabbit hole for another day.

Logging

Once a script does anything beyond the trivial, you’ll want it to tell you what it’s doing, especially when it runs unattended from cron or CI and there’s nobody watching the terminal.

The simplest improvement over a bare echo is to send your diagnostic messages to standard error instead of standard output. That keeps your logs separate from the script’s actual output, so something like result=$(./script.sh) doesn’t accidentally capture your log lines.

echo "Starting backup" >&2

From there, a small logging function pays for itself fast. Timestamps and a level make logs you can actually grep through later:

#!/bin/bash

log() {
    local level="$1"
    shift
    echo "$(date '+%Y-%m-%d %H:%M:%S') [${level}] $*" >&2
}

log INFO  "Starting backup"
log ERROR "Disk is full"

2026-06-23 14:05:01 [INFO] Starting backup
2026-06-23 14:05:01 [ERROR] Disk is full

If you want everything the script prints from a certain point on to go to a file, you can redirect once at the top with exec:

#!/bin/bash
# Everything after this line goes to the log file, both stdout and stderr
exec >> /var/log/myscript.log 2>&1

If you want it on screen and in a file at the same time, pipe through tee:

./backup.sh 2>&1 | tee -a backup.log

And if you’re on a system with syslog or journald, logger will drop your message straight into the system log, which is handy for scripts that should show up alongside everything else the machine logs:

logger -t myscript "Backup finished successfully"

Debugging

Sooner or later a script does something baffling and you need to see what it’s actually doing, not what you think it’s doing. Shell gives you a few good tools for that.

The first one is execution tracing. Run the script with bash -x and the shell prints each command, with its variables already expanded, right before it runs it:

$ bash -x script.sh
+ HOST=google.com
+ ping -c 1 google.com

You don’t have to trace the whole thing. Wrap just the suspicious section with set -x to turn tracing on and set +x to turn it back off:

set -x          # trace on
risky_function
set +x          # trace off

The trace lines start with + by default. You can make them far more useful by customizing PS4 to include the script name and line number:

export PS4='+ ${BASH_SOURCE}:${LINENO}: '
set -x

Now each traced line tells you exactly where it came from.

The next set of tools doesn’t find bugs so much as stop your script from sailing past them. These three options, usually set together at the top of a script, save an enormous amount of pain:

#!/bin/bash
set -euo pipefail

set -e exits the moment a command fails, instead of carrying on and doing more damage with bad state.
set -u treats the use of an unset variable as an error. This is the one that catches the classic rm -rf "$DIR/" disaster when $DIR is empty because of a typo.
set -o pipefail makes a pipeline fail if any command in it fails, not just the last one. Without it, false | true is considered a success.

You can also have the script tell you where it died using a trap on the ERR signal:

trap 'echo "Error on line $LINENO" >&2' ERR

And if you just want to check a script for syntax errors without running it (a poor man’s dry run), use -n:

bash -n script.sh

It reads the whole script and reports syntax problems, but executes nothing.

The single best tool I can recommend here isn’t built into bash at all: ShellCheck. It’s a linter for shell scripts, and it catches an embarrassing number of bugs before you ever run them, including unquoted variables, the unset-variable traps above, and a hundred subtle quoting issues. Run shellcheck script.sh (or paste into the website), and wire it into your editor and CI if you write shell regularly.

That rounds out the little tour of shell scripting I set out to write. Between this and the earlier posts, you’ve got enough to write real, maintainable scripts: conditionals, functions, exit statuses, wildcards, and a way to see what’s going on when things break. As always, there’s a lot more depth to each of these than a blog post can hold, so treat this as a map rather than the territory.

Thanks for reading!