backend / linux bash / 03_quoting_and_expansion.md

Quoting and Expansion

6 interview angles 5 min read source

Quoting and Expansion

The single biggest source of bash bugs: subtle differences between single quotes, double quotes, no quotes, and $(...) substitution. The rules are simple once you internalize them; ignoring them produces “works on my machine” disasters.

The four levels

echo hello world          # no quotes — word-splits
echo "hello world"        # double — interpolates $vars and $(...)
echo 'hello world'        # single — literal everything
echo `date`               # backticks — command substitution (legacy)
echo $(date)              # $(...) — modern command substitution

Quote rules — what each protects from

Quoting Word splitting Glob expansion $var $(...) \ escape
no quotes NO (splits) NO (expands) YES YES YES
"double" yes (preserves) yes (literal) YES (interpolates) YES YES
'single' yes yes NO (literal) NO (literal) NO (literal)

Mnemonic: double does (interpolate), single doesn’t.

name="alice"

echo $name           # alice
echo "$name"         # alice
echo '$name'         # $name (literal)
echo "\$name"        # $name (escaped)

Word splitting — the killer bug

Without quotes, the shell splits on IFS (default: space, tab, newline) and expands globs:

files="report q1.txt"
cat $files           # tries: cat report q1.txt    → two files
cat "$files"         # tries: cat "report q1.txt"  → one file

For filenames with spaces, $files looks like two filenames. Always quote unless you specifically want splitting.

The same bug with command output:

files=$(ls *.txt)        # works only if no filenames have spaces
for f in $files; ...     # word-splits on space

Right way:

for f in *.txt; do       # glob doesn't split
    echo "$f"
done

Or with find + read:

find . -name "*.txt" -print0 | while IFS= read -r -d '' f; do
    echo "$f"
done

-print0 and read -d '' use NUL bytes as separators, which can’t appear in filenames.

Globbing

The shell expands *, ?, [abc] patterns to matching filenames:

Pattern Matches
* any chars except / (and dotfiles by default)
? any single char
[abc] a, b, or c
[a-z] lowercase letter
[!abc] NOT a, b, or c
** any depth, only with shopt -s globstar

If no files match, by default the pattern stays literal:

ls *.foo                 # if no .foo files, literally tries: ls *.foo → "*.foo: no such file"

Fix with shopt -s nullglob (no matches → empty list):

shopt -s nullglob
for f in *.foo; do echo "$f"; done    # loop runs 0 times if no matches

Brace expansion

Distinct from globbing — happens before:

echo file_{a,b,c}.txt        # file_a.txt file_b.txt file_c.txt
echo {1..5}                  # 1 2 3 4 5
echo {a..e}                  # a b c d e
mkdir -p project/{src,tests,docs}
cp file.txt{,.bak}            # cp file.txt file.txt.bak (handy backup pattern)

Brace expansion happens regardless of whether the resulting names exist (no glob check).

Tilde expansion

cd ~                         # $HOME
cd ~/projects                # $HOME/projects
cd ~alice                    # alice's home directory

Only at the start of a word. ~ mid-word is literal.

Command substitution

today=$(date +%Y-%m-%d)
files=$(ls -1 | wc -l)
hash=$(git rev-parse --short HEAD)
echo "Build $hash on $today: $files files"

$(...) runs the command, captures stdout (without trailing newline), substitutes it inline. Trailing newlines are stripped — convenient for $(date) but surprising if you actually want them.

Backticks (`cmd`) do the same but:

  • Don’t nest cleanly: `cmd1 `cmd2 `` is ambiguous.
  • Look weird and clash with markdown.

Use $(...).

Arithmetic expansion

echo $((2 + 3))              # 5
i=10
echo $((i * 2))              # 20  — no $ needed inside (())
echo $((i++))                # 10 (post-increment)

For floats, fall back to bc, awk, or Python:

echo "scale=2; 1/3" | bc     # 0.33
python3 -c "print(1/3)"

Process substitution

diff <(sort file1) <(sort file2)

<(...) makes the command’s output appear as a temporary filename — useful when a tool wants a file and you only have a command’s output. Bash-only (not POSIX sh).

$variable vs ${variable}

name="alice"
echo "$name_alpha"           # bash thinks variable name is "name_alpha"
echo "${name}_alpha"         # alice_alpha

${var} braces are required when followed by a character that could be part of a variable name. Habit: always use ${var} for clarity.

Heredocs

Multi-line literal strings:

cat <<EOF
Line 1
Line 2: $name        # interpolates
EOF

cat <<'EOF'
Line 1
Line 2: $name        # literal $name (single-quoted heredoc)
EOF

cat <<-EOF           # leading TABS stripped
    indented but tabs eaten
    EOF

Useful for embedding multi-line content (config files, SQL, JSON) in scripts:

psql -c "$(cat <<SQL
    SELECT count(*) FROM users WHERE active = true;
SQL
)"

“Strong” vs “weak” quoting cheat sheet

file="my report.txt"

cat $file              # cat my report.txt → two args
cat "$file"            # cat "my report.txt"
cat '$file'            # literally tries to open file "$file"
cat "\"$file\""        # but odd — passes the quotes through

cmd=$(echo "$file")    # "my report.txt"
cmd="$(echo $file)"    # word-splitting still happens inside $(...)
cmd="$(echo "$file")"  # quote inside $() too

The lesson: $(...) is its own quoting context — quote variables inside it just like outside.

Common interview confusions

  • “Single and double quotes are interchangeable.” — totally different. Single is literal; double interpolates. Test with name="alice"; echo "$name" '$name'.
  • echo $var is fine.” — works in trivial cases. Breaks when var has spaces, asterisks, or other shell-special characters. Always "$var".
  • “Backticks and $() do the same thing.” — semantically yes, but $() nests cleanly and reads better.
  • “Variables don’t need braces.”${var} is required when followed by chars that could be part of a name. Habit: always use braces.

Interview angle

  • “What’s the difference between \"$var\" and '$var'?” — double quotes interpolate $var to its value; single quotes treat it as a literal string $var.
  • “Why quote \"$var\"?” — without quotes, the shell word-splits on whitespace and expands globs. Variables containing filenames with spaces or * break unquoted commands.
  • “What’s word splitting?” — when an unquoted variable expands, the result is split on IFS characters (space, tab, newline) into multiple words. Often unwanted.
  • $(...) vs backticks?” — same effect (run command, substitute output), but $() nests properly and is the modern form.
  • “What does \"${var}\" give you that \"$var\" doesn’t?” — explicit name boundary, lets you concatenate: \"${var}_suffix\" works; \"$var_suffix\" looks for variable var_suffix.
  • “Process substitution — what is it?”<(cmd) syntax that makes a command’s output appear as a file path. Use to feed two command outputs to a tool that wants files: diff <(sort a) <(sort b).