The theory of communication

The theory of communication (Shannon, 1948) is the immediate predecessor of Christensen’s theory of knowledge. It provides the basis for optimizing the designs of telecommunications systems, including television systems, telephone systems and computer data communications systems.

A “telecommunications system” is a device that moves a message such as “Mary had a little lamb…” from one place to another. A message is a sequence of symbols. Each symbol is drawn from a fixed alphabet.

For the purpose of efficiency in the utilization of equipment, the message passes through a device called an “encoder” before being transmitted. The encoder translates the un-encoded transmitted message into a message whose symbols are drawn from a different alphabet; the latter is called the “encoded, transmitted message.” As the encoded, transmitted message passes down the transmission line, it is corrupted by noise, switching some of the symbols of the message. Thus, the encoded, received message differs from the encoded transmitted message; for example, the received message is “111000110…” when the transmitted message was “011000110…” A device called a “decoder” translates the encoded received message into the un-encoded received message. Because the encoded received message differs from the encoded transmitted message, the un-encoded received message may differ from the un-encoded received message. For example, the un-encoded received message may be “Zary had a little lamb…” when the un-encoded transmitted message was “Mary had a little lamb…”

In the theory of communication, the role of the decoder is viewed as making a sequence of inferences. Each such inference is to an unobserved state-space whose elements are the alphabet of the un-encoded transmitted message.

The procedure by which a decoder makes inferences is an example of a model. In building this model, the builder faces the problem of selecting which inference is correct from among many alternatives. Under the theory of communication, this problem is solved by a principle of reasoning. This principle is to identify as correct that alternative which minimizes Shannon’s measure of the various alternatives; Shannon’s measure of an alternative is the missing information in it for a deductive conclusion.

In computing the missing information, one must assign values to probabilities. Each such assignment makes an inference.

There are an infinite number of alternate inferences. Which of these inferences is correct? The developer of the theory of communications, Claude Shannon, doesn’t say. In this sense, the theory of communication is incomplete, as it is left to us by Shannon. Ultra-optimization completes Shannon’s theory via its second principle of reasoning.

Home