5654 Syllabus Outline

• Review of infinite sums, probability and conditional expectation properties (Chapter 1)
• Random walk, a central special Markov chain example. Recurrence and transience. Hitting probabilities, gambler's ruin. (Chapter 2)
• General discrete time and space Markov chains. Path probabilities and transition probabilities. n-step transition probabilities. Invariant distributions, convergence to equilibrium. (Chapter 2)
• Filtering for discrete time and space Markov chains. (Chapter 3)
• L_2 -space, completeness, projection. The general mathematical notion of conditional expectation. Conditional expectation for Gaussian random variables. (Chapter 4).
• Filtering for discrete-time continuous-space Markov processes. Gaussian case: the Kalman filter. Linear filtering (Chapter 5)
• Some elements of stochastic calculus, Brownian motion, continuous time Kalman filter (Chapter 6)
• Elements of stationary processes, filtering based on the spectrum (first three sections of Chapter 7).