Textbook in PDF format

These lecture notes have been developed for the course Stochastic Processes at Department of Mathematical Sciences, University of Copenhagen during the teaching years 2010-2016. The material covers aspects of the theory for time-homogeneous Markov chains in discrete and continuous time on finite or countable state spaces.
The back bone of this work is the collection of examples and exercises in Chapters 2 and 3. It is my hope that all mathematical results and tools required to solve the exercises are contained in Chapters 2 and 3 and in Appendix B. The manuscript was never intended to provide complete mathematical proofs of all the main results since these may be found elsewhere in the literature. Further, inclusion of (even more) long and technical proofs would remove focus from the main goal which is to learn how to solve problems. However, any lecturer using these lecture notes should spend part of the lectures on (sketches of) proofs in order to illustrate how to work with Markov chains in a formally correct way. This may include adding a number of formal arguments not present in the lecture notes. Some exercises in Appendix C are formulated as step-by-step instructions on how to construct formal proofs of selected theoretical results. It is definitely advisable (and probably necessary) to consult other textbooks on Markov chains in order to be able to solve all of the exercises in Appendix C. I advise students to postpone these exercises until they feel familiar with the exercises in Chapters 2 and 3