This chapter provides a brief introduction to the Fundamental Problem of Communication, as formulated by Shannon, and evolved over the years into various generalities, including the authors’ views on Duality of a Source to a Channel. Suggestions for further research are described, with emphasis on the importance of this duality in nonanticipative or real-time information transmission in both communication and communication for control, of delay-sensitive applications.

Building on the material of previous chapters, a duality of sources and channels, both with memory and feedback is investigated, based on nonanticipative transmission, with respect to excess distortion probability. The methodology is based on (i) identifying the information structures of capacity achieving channel distributions, and encoder functions and (ii) identifying the information structures of optimal reproduction distributions of the nonanticipative rate–distortion function (RDF). An example is presented of a source with memory and a channel with memory, operating optimally, in real-time.

Shannon's coding capacity of memoryless additive Gaussian noise (AGN) channels with noiseless feedback, is known to be achieved by the Elias [1] coding scheme of communicating the mean square-error (MSE) of a Gaussian RV X ~ N(0, σ'), from past channel outputs, and decoding it using a MSE decoder. Further, it is known that among all encoders, and decoders that minimize the MSE, then the Elias encoder and decoder are globally optimal. In this paper we derive analogous results, for communicating unstable Gaussian Markov processes over unstable multiple-input multiple-output (MIMO) Gaussian recursive models (GRM) with memory, often called infinite impulse response (IIR) models, subject to an average cost of quadratic form. However, unlike memoryless AGNs, certain conditions are required, for such generalizations. Further, to show global optimality, we need to invoke Gorbunov and Pinsker [2] nonanticipatory € entropy, instead of the classical rate distortion function of the source. Another important observation is that we need a two-parameter coding scheme, instead of the one-parameter coding scheme of memoryless AGN channels.