Processing math: 100%

HW2 (Due 10/4)

  1. Continue with Question 2 in HW1

    1. What is I(D1;D2)?

    2. What is I(D1;S)?

    3. What is I(D1;D2|S)?

    4. What is I(D1;D2|S=3)?

  2. Let X1 and X2 be two discrete random variables whose outcomes do not overlap. And let X=X1 with prob p and X=X2 with prob (1p). Show that H(X)=pH(X1)+(1p)H(X2)+H(p)

  3. A group of 1,000 students have the average height of 170 cm and the standard deviation of 15 cm. Approximatley how many bits are needed to store all the height data with the precision of 0.1 cm assuming the height is normally distributed?

  4. (Typical sequences of coin-flips) Consider sequences of N coin-flips with the probability of head equal to p. Denote nH(x) as the number of heads in a sequence x. Assume p<0.5. Show that a sequence x is ϵ-typical (xANϵ(X)) if and only if pϵlog1ppnH(x)Np+ϵlog1pp

    1. Let p=0.2, ϵ=0.01, and N=100, compute Pr(XNANϵ(X))

    2. Repeat (a) with N change to 10,000.

    3. Repeat (a) and (b) with p change to 0.45.