Material


Stochastic systems analysis and simulation is divided in four blocks plus two introductory lectures. This page indexes the material prepared for different blocks

Introduction


This introductory block previews the class's contents and exemplifies topics by studying a simple stochastic system. Material available for this block is the following:

Probability review


This block is a speedy review of Probability theory. We start reviewing the axiomatic definition of probability and introduce the concept of random variable. We then introduced commonly used distributions, the concept of expected value and joint distributions. We finish introducing different concepts of limit and limit theorems like the law of large numbers and the central limit theorem. We also devote a class to talk about conditional probability since this is something we will be using extensively in the rest of the class. Material available follows.

  • First set of slides covering definition of probability, random variables, commonly used distributions, expected value and joint distributions.
  • Second set of slides covering Markov and Chebyshev's inequalities, definitions of limits in probability, the law of large numbers and the central limit theorem. Also conver conditional probability and exemplify its use to compute probabilities and expected values.

Markov chains


  • Slides containing first 7 lectures on Markov chains. The first two lectures cover definitions, notations and the introduction of Chapman-Kolmogorov equations for the evolution of probabilities. We discuss then two simple examples, gambler's ruin and buffers in communication networks. We then follow with the introduction of the different classes of sates that may compose a Markov chain, and the concept of limiting distributions for irreducible aperiodic Markov chains. We also discuss ergodicity to some detail. The last class in this block expands the discussion of buffers in communication networks.
  • Slides for two classes covering ranking of nodes in graphs.

Continuous time Markov chains


Gaussian, Markov and Stationary random processes


  • Gaussian processes. This set of slides defines Markov processes, Gaussian processes and stationary processes. It then studies Gaussian processes to some detail putting emphasis on Brownian motion and white noise. We also study biased Brownian motion and geometric Brownian motion to facilitate the future discussion on arbitrages, stock and option pricing.
  • Arbitrages and pricing of stocks and options.
  • Stationary processes.