Textbook in PDF format

This is a thoroughly revised and expanded third edition of a successful university textbook that provides a broad introduction to key areas of stochastic modelling. The previous edition was developed from lecture notes for two one-semester courses for third-year science and actuarial students at the University of Melbourne. This book reviews the basics of probability theory and presents topics on Markov chains, Markov decision processes, jump Markov processes, elements of queueing theory, basic renewal theory, elements of time series and simulation. It also features elements of stochastic calculus and introductory mathematical finance. This makes the book suitable for a larger variety of university courses presenting the fundamentals of modern stochastic modelling. To make the text covering a lot of material more appealing and accessible to the reader, instead of rigorous proofs we often give only sketches of the arguments, with indications as to why a particular result holds and also how it is related to other results, and illustrate them by examples. It is in this aspect that the present, third edition differs from the second one: the included background material and argument sketches have been extended, made more graphical and informative. The whole text was reviewed and streamlined wherever possible to make the book more attractive and useful for readers. Where appropriate, the book includes references to more specialised texts on respective topics that contain both complete proofs and more advanced material.
Preface to the First Edition.
Preface to the Second Edition.
Preface to the Third Edition.
About the Author.
Introduction.
Basics of Probability Theory.
Markov Chains.
Markov Decision Processes.
The Exponential Distribution and Poisson Process.
Jump Markov Processes.
Elements of Queueing Theory.
Elements of Renewal Theory.
Elements of Time Series.
Elements of Simulation.
Martingales and Stochastic Calculus.
Diffusion Processes.
Elements of Mathematical Finance.
Answers and Solutions to Problems.
Greek Alphabet.
Notations.
Abbreviations.
Index