Field Splitting, IFS, and the Difference Between "$@" and $*

A shell has no type for a list of strings. Everything is a string, and every command receives an argument vector assembled by textual operations on those strings. That is why a variable holding a filename with a space in it works in one context and silently becomes two filenames in another, and why the distinction between "$@" and $* is not stylistic.

Expansion happens in a fixed order

POSIX defines the sequence, and the sequence is the explanation for most quoting surprises. Tilde expansion runs first, then parameter expansion, command substitution, and arithmetic expansion. These three are performed together, left to right. The results are then subject to field splitting, then pathname expansion, then quote removal.

The ordering means that characters produced by a variable’s value are not re-examined for syntax. A variable containing ; rm -rf / cannot inject a command separator, because parsing happened before expansion. But the same value is subject to field splitting and globbing unless it was quoted, so it can become many arguments, and a * inside it can expand to the contents of the working directory. Two different classes of metacharacter, two different stages, and only one of them is neutralised by quoting alone.

The practical rule that follows: quoting suppresses field splitting and pathname expansion, and it does nothing about the parse, which is already finished. That is why quoting is safe everywhere and necessary almost everywhere.

What IFS actually delimits

Field splitting uses the value of IFS, defaulting to space, tab, and newline. The rules differ depending on whether a character is whitespace or not, and the difference is not cosmetic.

IFS whitespace characters are collapsed: a run of them delimits one boundary, and leading and trailing runs produce no empty fields. Non-whitespace IFS characters each delimit a field, and consecutive occurrences do produce empty fields. Setting IFS=: and splitting a::b yields three fields, the middle one empty; splitting a b with the default IFS yields two.

value='alpha::beta'
IFS=: ; set -- $value
printf '%s\n' "$#" "$1" "$2" "$3"   # 3, alpha, empty, beta

The example also shows why changing IFS for one operation is hazardous: the variable is global to the shell, it affects every subsequent unquoted expansion, and read consults it too. A subshell or an explicit IFS=: read -r a b c limits the damage. Note that read with a non-default IFS strips leading and trailing IFS whitespace from each field, and that the last variable receives the remainder of the line including any delimiters, a behaviour that makes IFS=, read -r a b on 1,2,3 assign 2,3 to b.

The two array idioms are not interchangeable

"$@" expands to one field per positional parameter, each quoted, with no splitting and no globbing. "$*" expands to a single field in which the parameters are joined by the first character of IFS, which is a space by default. Unquoted $@ and $* are the same thing: both split and glob.

set -- 'a b' 'c*d'
printf '[%s]\n' "$@"     # [a b] then [c*d]
printf '[%s]\n' "$*"     # [a b c*d]
printf '[%s]\n' $@       # [a] [b] [c*d], or more if c*d matches files

The third form is the classic defect. It produces two fields from the first parameter and, if any file in the working directory matches c*d, replaces the second parameter with those filenames. A wrapper script that forwards its arguments with $@ therefore corrupts any argument containing a space and any argument containing a glob character. The fix is one pair of double quotes, and it is the single highest-value habit in shell programming.

"$@" has one special case worth knowing: when there are no positional parameters, it expands to nothing at all, not to one empty field. That is why cmd "$@" passes no arguments rather than an empty string, and why the same construct with "$*" passes one empty argument. A wrapper that needs to distinguish “no arguments” from “one empty argument” depends on this difference.

Where splitting silently changes meaning

The most common pattern that looks correct and is not is iterating over command output:

for f in $(find . -name '*.log'); do
    wc -l "$f"
done

Two separate faults. The substitution’s output is split on whitespace, so a path containing a space becomes two iterations. And the result is globbed, so a path containing * or ? is replaced by whatever matches. The inner "$f" is correctly quoted, which gives a false sense of safety: the damage happened before the loop body ran.

The same shape appears in for f in $(ls), in accumulating filenames into a single scalar variable, and in any place where a list is transported as a string. Shells provide arrays for exactly this reason, and the portable alternative is to have the producer emit NUL-delimited records and consume them with read -d '' in bash or xargs -0 where available. Newline-delimited records fail on filenames containing newlines, which are legal on every Unix filesystem. A wrapper that forwards a list to another tool has the same problem from the other end, since the receiving option parser cannot tell a split word from two intended ones.

The reverse hazard is treating a scalar as if it were a list. A variable set to a b c and then iterated unquoted works until one element needs quoting, at which point there is no way to represent it. The data structure has to be chosen before the value is assigned, not after.

Arrays replace the string list, where they exist

Bash, ksh, and zsh provide real arrays, and the quoting rules carry over exactly: "${arr[@]}" expands to one field per element, while "${arr[*]}" joins with the first character of IFS, and ${arr[@]} unquoted splits and globs. The parallel with "$@" is not a coincidence; positional parameters are an array with a special name, and the same expansion machinery handles both.

Arrays remove the representation problem rather than working around it. Assigning a glob to an array stores each match as an element, so a later "${arr[@]}" passes them through unchanged regardless of what characters they contain. Appending with arr+=("$new") cannot be confused by spaces in the new element. The failure mode that remains is using the wrong expansion form, which is the same failure mode as $@ and has the same fix.

None of this is portable. POSIX defines no array syntax, so a script that must run under a minimal sh has to choose between a delimited string with careful quoting and a loop that processes items as they are produced, never storing the list. The second option is usually better: find ... -exec cmd {} + or a while read loop over NUL-delimited records avoids materialising the list at all. Globbing can be suppressed wholesale with set -f, which turns off pathname expansion for the rest of the shell; that is occasionally the right tool when a value must be passed through literally, but it is a global setting and it will surprise any function called afterwards.

Quoting forms and their exact scope

Single quotes preserve everything literally; no expansion of any kind occurs inside them, and a single quote cannot appear inside a single-quoted string at all. Double quotes allow parameter expansion, command substitution, and arithmetic expansion, but suppress field splitting and pathname expansion on the results. Inside double quotes, a backslash is special only before a dollar sign, a backtick, a double quote, a backslash, and a newline; before any other character it is a literal backslash.

That rule trips up anyone who expects a two-character sequence of backslash and n inside double quotes to become a newline. It does not; the escape is not interpreted there, and only printf or the C-style quoting form does the translation. The C-style form, written with a leading dollar sign and single quotes, is a bash and ksh extension that POSIX Issue 7 does not define, and the locale-translation form with a leading dollar sign and double quotes is likewise an extension. A script that must run under a shell implementing that revision cannot use either, and the portable way to embed a tab or a newline is to build it with printf or to place the literal character in the source.

Here-documents deserve their own note: an unquoted delimiter expands the body, a quoted delimiter does not. An unquoted here-document expands variables and command substitutions; a delimiter in quotes passes the text through unchanged, which is what is wanted for embedding a script inside a script.

The ordering rule that resolves the confusion

When a construct is ambiguous, the expansion order resolves it. Splitting happens once, after all expansions, on the unquoted results; it does not happen recursively on fields produced by splitting, and it does not happen to text that was quoted at the source. Globbing happens after splitting, on each field, so a field that came from splitting cannot glob into something that then splits again.

That single pass explains why "$@" is correct in every position where a list is meant, why an unquoted variable is a list of words rather than a single string, and why quoting a command substitution preserves its output as one argument including internal newlines while the unquoted form flattens it. The same ordering underlies the way argument vectors are built, since the shell’s output at the end of expansion is exactly the array handed to execve, and it explains why the number of arguments a program receives can differ from the number of words that appear to have been written.