ECE/TCOM-5583
Syllabus
Links
Homeworks
HW1
HW2
HW3
HW4
HW5
Presentation
Presentation order
Final project
Project abstract
Project report
Other
My Home

HW2 (Due 9/19)

  1. Continue with tropical island question in HW1. Let's denote \(W_i\) as the weather condition (either sunny or rainy) and \(T_i\) as the temperature on day \(i\).

    1. What is \(H(T_0)\)?

    2. What is \(H(T_0|W_0)\)?

    3. What is \(H(T_1|W_0)\)?

    4. What is \(I(T_1;T_0|W_1)\)?

  2. Let \(X_1\) and \(X_2\) be two discrete random variables whose outcomes do not overlap. And let \(X=X_1\) with prob \(p\) and \(X=X_2\) with prob \((1-p)\). Show that \(H(X) = p H(X_1) + (1-p) H(X_2) + H(p)\), where \(H(p) \triangleq -p \log_2(p) - (1-p) \log_2 (1-p)\).

  3. (Typical sequences of coin-flips) Consider sequences of \(10,000\) coin-flips with the probability of head equal to \(0.7\). Show that a sequence with \(7,000\) heads and \(3,000\) tails is typical.