Learning path

Full curriculum

Full curriculum

Arrows go from each prerequisite to the units that depend on it. Hover or focus a unit to highlight its path.

Unit content

Source-channel separation theorem

Source coding and channel coding solve opposite problems. Source coding removes predictable redundancy from the message; channel coding adds controlled redundancy to protect transmission.

The source-channel separation theorem states that, under the standard asymptotic assumptions for an independent source and a memoryless channel, these two tasks can be designed separately without losing fundamental efficiency.

If the source's irreducible information rate after ideal lossless compression is below the channel capacity, reliable communication is possible in principle.

A conceptual architecture is therefore

source → compression → channel code → channel → decoder → decompression

The theorem is asymptotic. Finite delay, finite block length, complexity, changing channels and application-specific distortion can make joint source-channel designs useful in practice.

Its importance is architectural: it explains why communication systems can often treat compression and transmission reliability as modular layers with a clean information-rate interface between them.