All readings will be listed here, organized by topic. Dates indicate the class by which you should have read the indicated material.   Some readings may be announced here before they are announced in class, in case you want to read ahead.   Sometimes, particularly if I am traveling, you may find some references posted at the library class reserves before they are linked here.
If a reading does not have a date, you don't have to worry about it yet.

Fundamentals of Probability Theory and Stochastic Processes

Assigned Readings

• Karlin & Taylor, A First Course in Stochastic Processes, Sec. 1.1A-F (PDF) (09/04/08)
• Kloeden & Platen, Numerical Solution of Stochastic Differential Equations, Sec. 1.1-1.3 (PDF) (09/04/08)

Finite-State, Discrete-Time Markov Chains

Assigned Readings

• Lawler, Introduction to Stochastic Processes, Ch. 1 (PDF) (09/18/08)

Optional Readings

• Karlin & Taylor, A First Course in Stochastic Processes, Secs. 2.1-2.3 (PDF)
  • examples of Markov chain models
• Resnick, Adventures in Stochastic Processes, Secs. 2.1-2.3 (PDF)
  • Discussion of stochastic update rule and more Markov chain model examples
• Resnick, Adventures in Stochastic Processes, Secs. 2.12-2.15 (PDF)
  • Complete probabilistic proof of existence and uniqueness of stationary distribution, and law of large numbers for Markov chains. Also some further discussion of recurrence/transience classification techniques and computational techniques for stationary distribution.
• Karlin & Taylor, A First Course in Stochastic Processes, Secs. 3.1-3.2 (PDF)
  • Analytical (rather than probabilistic) method of proving limit theorems for Markov chains
• Karlin & Taylor, A First Course in Stochastic Processes, App. 2 (PDF)
  • Perron-Frobenius theory for positive matrices
• Haken, Synergetics, Secs. 4.6-4.8  (PDF)
  • Stationary distributions:  Detailed balance, microreversibility, and Kirchoff's method of solution
• Resnick, Adventures in Stochastic Processes, Secs. 2.5-2.6 (PDF)
  • Dissection principle for Markov chains and some theory on recurrence and transience when the n-step probability transition densities can be computed explicitly.
• Guttorp, Stochastic Modeling of Scientific Data, Sec. 2.1
  • Maximum likelihood method for choosing parameters in Markov chain

Countable State, Discrete-Time Markov Chains

Assigned Readings

• Lawler, Introduction to Stochastic Processes, Ch. 2 (PDF) (10/16/08)
• Karlin & Taylor, A First Course in Stochastic Processes, Ch. 2 and 3 (PDF) (10/16/08)

Countable State, Continuous-Time Markov Chains

Assigned Readings

• Lawler, Introduction to Stochastic Processes, Ch. 3 (PDF) (10/30/08)
• Karlin & Taylor, A First Course in Stochastic Processes, Ch. 4 (PDF) (10/30/08)
• Andersson and Britton, Stochastic Epidemic Models and Their Statistical Analysis, Ch. 2 (PDF) (12/04/08)

 

Optional Readings

• Feller, An Introduction to Probability Theory and its Applications, Volume I, Ch. XII.5 (PDF)
  • Application of branching process theory to queues
• Karlin & Taylor, A First Course in Stochastic Processes, Sec. 6.8 (PDF)
  • Markov times for continuous-time stochastic processes
• Reichl, A Modern Course in Statistical Physics, Ch. 5 (PDF)
  • A concise discussion of Markov chains and stochastic differential equations from a physicist's perspective.  A nice collection of concepts, but there are some misleading statements!

Renewal Processes

Assigned Readings

• Karlin & Taylor, A First Course in Stochastic Processes, Secs. 5.1-5.6 (PDF) (11/24/08)
• Karlin & Taylor, A First Course in Stochastic Processes, Secs. 5.8 B&C  (PDF) (11/24/08)
• Resnick, Adventures in Stochastic Processes, Secs. 3.3 & 3.4 (PDF) (12/04/08)

Optional Readings

• Karlin & Taylor, A First Course in Stochastic Processes, Sec. 5.7C (PDF)
  • Renewal processes with two phases per renewal period
    
• Karlin & Taylor, A First Course in Stochastic Processes, Sec. 5.7G (PDF)
  • Application of renewal theory to calculating bankruptcy risk for insurance company
• Karlin & Taylor, A First Course in Stochastic Processes, Sec. 5.9 (PDF)
  • Limit theorems for superposition of rare renewal processes being well approximated by Poisson point process
    

Martingales

Assigned Readings

• Karlin & Taylor, A First Course in Stochastic Processes, Sec. 6.1-6.4 (PDF)
• Lawler,  Introduction to Stochastic Processes, Secs. 5.1-5.3 (PDF)