A reduction combines many values into one using an operation such as sum, minimum, maximum or logical AND.

A sequential sum of $n$ values forms a chain of $n-1$ additions. If addition is treated as associative, the same work can be arranged as a balanced tree:

[a b c d e f g h]
 \ / \ / \ / \ /
 a+b c+d e+f g+h
   \ /     \ /
   ...     ...
       \ /
      total

With $n$ inputs, a balanced reduction performs $O(n)$ total work but has only $O(\log n)$ span, exposing substantial parallelism.

Associativity is what permits regrouping:

$$a\circ(b\circ c)=(a\circ b)\circ c.$$

Commutativity is helpful for arbitrary reordering but is not required when the implementation preserves operand order within the reduction tree.

Floating-point addition is not exactly associative because rounding depends on grouping. A parallel sum can therefore differ slightly from a sequential left-to-right sum even when both are numerically reasonable.

Efficient reductions usually accumulate local partial results first and combine those partials hierarchically, avoiding one highly contended shared accumulator.