Question

Exponential noise channels. ponentially distributed noise with mean u. Assume that we have Yi = Xi + Zi, where Zi is iid. ex- 9.4 mean constraint on the signal (i.e., EX A). Show that the capacity of such a channel is C = log(1 +

Answer 1

Just as for the Gaussian channel, we can write

1(X:X) C= max

h(Y)-h(y|X) C= max

h(Y)-h(Z| X) C= max

h(y)-h(Z) C= max

h(Y)-(1+1n,1) C= max

Now Y= X+ Z, and

$EY=EX+EZ\leq \lambda +\mu$

Given a mean constraint,the entropy is maximized by the exponential distribution, and therefore

EX-A+μ

Unlike normal distributions, though, the sum of two exponentially distributed variables is not exponential, so we cannot set X to be an exponential distribution to achieve the right distribution of Y.Instead, we can use characteristic function to find the distribution of X.

The characteristic function of an exponential distribution-

$\psi (t)=\int \frac{1}{\mu }e^{-\frac{x}{\mu }}e^{-itx}dx=\frac{1}{1-i\mu t}$

The distribution of X that when added to Z will give an exponential distribution for Y is the ratio of the characteristic functions

$\psi_{X} (t)=\frac{1-i\mu t}{1-i(\lambda +\mu )t}$

Which can have seen to correspond to mixture of a point mass and an exponential distribution.

If

$X = \left\{\begin{matrix} 0, & with probability \frac{\mu }{\lambda +\mu }\\ X_{e},& with probability \frac{\lambda }{\lambda +\mu } \end{matrix}\right.$

Where X_e has an exponential distribution with parameter $\mu +\lambda$ , we can verify that the characteristic function of X is correct.

Using the value of entropy for exponential distributions, we get

$C=h(Y)-h(Z)=1+ln(\lambda +\mu )-(1+ln\mu )=ln\left ( 1+\frac{\lambda }{\mu } \right )$