Expansion and Globbing

Before bash runs a command, it rewrites the line through a fixed sequence of expansions: brace expansion, then tilde, then parameter and variable expansion, command substitution and arithmetic expansion (these three happen together, left to right), then word splitting, and finally pathname expansion — globbing. The program you are calling never sees *.log or $HOME or {a,b}; it sees the already-expanded list of words the shell handed it. By the time execve() runs, the original text is gone.

That order is the single most useful thing to internalize, because almost every "weird" shell result is just expansion happening in a stage you did not expect — or not happening because something was quoted. Word splitting runs after variables expand and before globbing, which is exactly why an unquoted variable holding a filename with spaces turns into multiple arguments and then gets glob-matched against the directory. Quote it and the whole chain collapses to a single literal word.

Pathname Expansion (Globbing)

Globbing is the shell matching unquoted patterns against existing filenames. Three operators do the work: * matches any string including the empty string (but not a leading dot and not /), ? matches exactly one character, and [...] matches one character from a set or range, such as [0-9] or [!a-z] for negation. The shell expands the pattern into a sorted list of matching paths and passes that list to the command — the command itself does no matching.

This is why ls *.log and rm *.log behave identically with respect to which files are chosen: ls and rm both receive the same already-expanded argument list. It is also why a leading dot is protected — * deliberately skips dotfiles, so cp * /backup will not copy .bashrc or .ssh. To include hidden files you need an explicit pattern or the dotglob option, covered below.

# the shell expands these; the command receives the file list
ls *.log              # every .log in the current dir
ls log-?.txt          # log-1.txt, log-a.txt — exactly one char
ls access-[0-9].log    # access-0.log .. access-9.log
ls *.[ch]             # .c and .h files
# prove it: echo runs no matching of its own
echo *.log             # prints the same expanded list ls would get

Brace Expansion

Brace expansion looks like globbing but is fundamentally different: it generates text, and it does not touch the filesystem at all. {a,b,c} expands to three words; {1..10} is a sequence; {01..10} zero-pads; {a..z} walks the alphabet; {1..10..2} takes a step. Whether the resulting names exist is irrelevant — brace expansion happens first, before any file is consulted.

That distinction has a practical consequence. mkdir {src,test,docs} creates three directories that did not exist a moment ago, because brace expansion produces the names regardless of the filesystem. A glob could never do that — * can only ever return things that already exist. The classic atomic backup, cp config.yaml{,.bak}, expands to cp config.yaml config.yaml.bak for the same reason.

# brace expansion generates words — files need not exist
mkdir -p project/{src,test,docs}
cp nginx.conf{,.bak}        # -> cp nginx.conf nginx.conf.bak
touch log-{01..05}.txt       # log-01.txt .. log-05.txt
echo {a..e}                 # a b c d e
# brace first, then glob the generated words against real files
ls /var/log/{syslog,auth.log}*

Tilde, Parameter, and Command Substitution

The remaining expansions fill in values rather than match files. Tilde expansion turns ~ into $HOME and ~user into that user's home directory — but only when the tilde is unquoted and at the start of a word, which is why "~/file" in a script is a common surprise that leaves a literal tilde. Parameter expansion is $VAR or ${VAR}, and the brace form unlocks defaults and slicing: ${VAR:-default} substitutes a fallback, ${VAR:?msg} aborts with an error if unset, and ${PATH##*:} strips a prefix.

Command substitution, $(command), runs the command and replaces the construct with its stdout, with trailing newlines stripped. The older backtick form `command` does the same but does not nest cleanly, so prefer $(...). Arithmetic expansion, $((expr)), evaluates integer math. All three feed their result back into the line and — if unquoted — into word splitting and globbing, which is the source of most substitution bugs.

# parameter expansion with defaults and string ops
echo "${DEPLOY_ENV:-staging}"     # fallback if unset/empty
port="${PORT:?PORT must be set}"  # abort if unset
echo "${file##*/}"             # basename via prefix strip
# command and arithmetic substitution — quote the result
now="$(date +%F)"
count="$(ls -1 *.log | wc -l)"
echo "$(( count * 2 ))"

Word Splitting and IFS

After variable and command substitution, bash splits the result into words on the characters in IFS — by default space, tab, and newline. This step runs only on unquoted expansions. So files=$(ls) followed by for f in $files breaks on every whitespace character, which destroys any filename containing a space and is the reason "parse ls" is a perennial bug. The fix is to quote: "$var" is never word-split.

Arrays are the correct tool when you genuinely have a list. "${arr[@]}" expands to one word per element with no splitting inside elements, preserving spaces in each filename. For iterating files, a glob in a for loop — for f in *.log — is safe because pathname expansion produces properly separated words without going through whitespace splitting. Setting IFS=$'\n' changes the split character, but quoting and arrays are the answer that holds up against spaces and newlines in filenames.

# unquoted expansion is word-split AND globbed — usually a bug
name="my report.txt"
rm $name              # tries: rm my report.txt  (two args!)
rm "$name"            # correct: one argument
# safe iteration: glob directly, quote the variable
for f in *.log; do
    gzip "$f"
done

nullglob, globstar, and dotglob

By default, a glob that matches nothing is passed through literally. ls *.xml in a directory with no XML files runs ls with the literal argument *.xml, producing a confusing "No such file" error — or worse, a loop body that runs once with a nonexistent path. The nullglob shell option changes this so a non-matching pattern expands to nothing, which is what you almost always want inside a script. failglob goes further and raises an error instead.

globstar enables ** for recursive matching: **/*.log matches log files at any depth below the current directory. It is off by default and must be enabled per shell with shopt -s globstar. dotglob makes * include dotfiles, which is occasionally wanted and frequently dangerous — rm * with dotglob set will also delete .gitignore and .env. These are bash-specific shopt options, distinct from the POSIX set options.

# shopt options change globbing behavior (bash-specific)
shopt -s nullglob        # non-matching glob -> empty, not literal
shopt -s globstar        # enable ** for recursion
shopt -s dotglob         # * also matches dotfiles (careful)

# with globstar on, descend the whole tree
for f in **/*.log; do gzip "$f"; done

# disable globbing entirely for a risky line
set -f                   # noglob: * stays literal