Modeling and Simulation of Transition Probabilities

Dante J. Marcozzi

University of Nevada, Las Vegas. Mathematics Department.


Citation: Marcozzi, Dante J. “Modeling and Simulation of Transition Probabilities.” Nevada State Undergraduate Research Journal. V2:I1 Spring-2016. (2016).

How to Cite: 

Author last name, Author first name first initial. “The name of the article in parenthesis.” Nevada State Undergraduate Research Journal. V(volume):I(issue) Semester (Fall or Spring)-Year. (2016).[Insert DOI here].


Transition probabilities are defined as the probabilities of changing state in a system. We introduce a mean reverting and non-negative transition process where the probabilities of leaving a given state must sum to a whole. We develop and implement a C++ Monte-Carlo program and present numerical examples, which can be used to study the model parameters.


A transitional probability is the probability of transitioning from one state to another at a particular instant of time. Applications of systems with transitional states are widespread. As an example from mathematical finance, we note that credit risk can be modeled by transition probabilities (cf. ref. [1]). A notable feature of many such applications is the changing nature of the transitional probabilities themselves, which evolve over time. To this end, we have developed a mathematical model that allows each transitional probability of a system to evolve over time, while still reverting to a predetermined mean value at a given rate. By developing approximation procedures based on the Monte-Carlo method (i.e. Euler approximation), we are able to visualize the relative effects of the parameters in order to fine-tune our model.

The act of changing state in a system is defined as a transition; the probabilities of state changing are called transition probabilities (cf. ref. [4], [7]). With this in mind, we consider m states, where the transition probability Pi from a given state to a state i satisfies the following conditions:

  1.   0 ≤ Pi ≤ 1       for each i=1,2,…,m
  2.   P1 + P2 + ⋯ + Pm = 1

Condition (2) means that the probabilities of leaving one fixed, given state moving to another must sum to unity. Our objective is to obtain a mean reverting process for the transition probabilities that satisfies (1)-(2).

Two meaningful examples that guide our development are:

Example 1. An Ornstein-Uhlenbeck process [6] describes the velocity dX(t) of a very heavy Brownian particle under the influence of friction at time t, which over time tends to drift towards its long-term mean X̄(a so-called meanreverting process)

  1.   dX(t) = α(X̄ – X(t))dt + σdW(t)

where α > 0 is the rate of mean reversion, σ > 0
is the dispersion, and W(t) is a Wiener process, which is a continuous-time stochastic process (also known as standard Brownian motion).

Example 2. The Cox-Ingersoll-Ross model (CIR) [3] describes the short-term interest rate X(t) at time t as given by:

  1.   dX(t) = α(X̄ – X(t))dt + σ√X(t)dW(t)

where the dispersion σ√X(t) avoids the possibility of the interest rate X(t) being negative.

Model of Transition Probabilities:

Motivated by the Ornstein-Uhlenbeck process and the CIR model, we define the following system of stochastic differential equations (SDE): given the current state, the transitional probabilities to states i=1,2,…,m-1 are defined by:

  1. a)   dPi(t) = αi(P̄i – Pi(t))dt + σiP(sub i)(t)Βi(P1,P2,…,Pm-1)dWi(t),

for 0 < tT, such that

  1. b)   pi(0) = pi,

where the initial probability of being in state i is 0 ≤ pi ≤ 1, the dispersion is σi > 0, the rate of reversion to the mean αi > 0,dWi(t) is a Wiener process, and

  1.   Βj (P1, P2, … , Pm-1) = √1 – (P(sub 1) + P(sub 2) + … + P(sub m-1)-1   ; j < m – 1,  j = m – 1

ensures the transition probabilities are each less than one. Finally, for j=m, we set

  1.   Pm(t) = 1 – (P1(t) + P2(t) + … + Pm – 1(t)),

which assures that the sum of the transition probabilities is one as required by (2).

For modeling transitional probabilities, it is essential that (1) and (2) be enforced. Motivated by examples (1) and (2), our contribution is the inclusion of Βj which imposes the constraints (1) and (2) upon the transitional probability dynamics (5a). That is, we ensure that
0 ≤ Pi ≤ 1. This is seen as follows:

For i = 1, 2, …, m-1, the transition probability Pi must be non-negative for the term √P(sub i)(t) in (5a)
to be defined. Likewise, the transition probability Pi is prevented from being greater than one by the square root term in (6) upon noting that Pi ≥ 0, for i = 1, 2, … m – 1. For i = m, the Βi term requires:

1 – (P1 + P2 + … + Pm-1) ≥ 0 or P1 + P2 + … + Pm-1 ≤ 1   such that Pm ≥ 0 by (7).

Also, Pm ≤ 1 since Pi ≥ 0 for i = 1, 2, … m-1. It follows then that conditions (1) and (2) are satisfied for the stochastic process (5)-(7).

Numerical Procedure:

The Euler approximation of the system of stochastic differential equations (5)-(7) is presented on the time grid, 0 < t1 < t2 < … < tN, such that the Wiener process dWi(tn) is replaced by εniΔt, where εni is a random element of the normal distribution with mean 0, variance 1 and Δt = tn+1 – tn, for m=1, 2, …, N-1. For the states i = 1, 2, …, m-1, letting Pni = Pi(tn), the discrete analogue of (5) is given by:

  1. a)   Pin+1 = Pinαi(P̄i – Pin)Δt + σiP(sub i)(sup n)βin(P1n, P2n, …, Pm-1ninΔt

for n=1,2,…,N-1, such that

  1. b)   Pi0 = pi
    βjn(P1n, P2n, …, Pm-1n) = √1 – (P(sub 1, sup n) + P(sub 2, sup n) + … + P(sub m-1, sup n)-1   ;j < m-1, j = m-1


  1.   Pmn = 1 – (P1n + P2n + … + Pm-1n)

(cf. ref. [5]).

Numerical Experiments and Results:

Numerical experiments were made to examine the effects of the various parameters appearing in equation (5). For tractability, we considered here four transition states (m=4) and a time horizon T=1. The two sets of model data of concern for the transitional probabilities Pi included the rates of reversion to the mean αi and the dispersions σi, for the transition states i = 1, 2, 3. In the experiments presented, we hold the dispersion at a constant level and vary the rates of reversion to the mean between the transition states. Specifically, we assigned α1 = 20, α2 = 10, α3 = 50 and σ1 = σ2 = σ3 = 0.05. For the mean reverting levels, we chose: P̄1 = 0.5, P̄2 = 0.3, P̄3 = 0.15; the processes was started at the mean reverting levels.

For the numerical approximation of section 3, we took N=100 (∆𝑡 = 0.01) computed by (8)-(9). Our random sample εin in (8a) was taken from a normal distribution with a mean of zero and a variance of one. A C++ code was developed in order to implement the Euler
approximation using the <random> library. As an indication of the model’s output, we present in Figures 1 and 2 (figures shown after references) trial results in two complementary ways: (i) each transition probability considered individually over the course of three simulations and (ii) the entire set of transition probabilities collectively resulting from a single numerical experiment.

In Figure 1, we showed three realizations (ω1, ω2, ω3) and the mean for each transitional probability (P1, P2, P3, P3). We leave the dispersions σi unchanged, however, the rate of reversion to the mean αi varied for each of the probabilities with P2 having the weakest rate and P3 having the strongest. The effect of the rate of reversion to the mean is seen by noting the relative choppiness in the sample path of Figure 1c and smoothness of Figure 1b as benchmarked against Figures 1a and 1d.

In Figure 2, we plotted the first and second realizations for each transition probability. In particular, we note that at any instance of time, the four probabilities satisfy the prior conditions (1) and (2) such that each probability remains bounded between 0 and 1 and the sum of the transitional probabilities is 1. In both figures we see the randomness inherent to the model resulting from the selection of εin in (8a).

Discussion and Conclusion:

A transitional probability is the probability of transitioning from one state to another at a particular instant of time. We have developed a mathematical model that allows each transitional probability of a system to evolve over time, while still reverting to a predetermined mean value at a given rate. We have also presented approximation procedures of the stochastic system based on the Monte-Carlo method (i.e. Euler approximation) and were able to visualize the relative effects of the various model parameters. In particular, the system of section 2 was employed in [2] to represent transitions between credit states in order to model default risk in financial contracts.


I would like to thank Dr. S. Choi, Department of Finance, University of Nevada Las Vegas and Dr. M.D. Marcozzi, Department of Mathematical Sciences, University of Nevada Las Vegas, for suggesting this project and their help in implementing various techniques used throughout the paper, their input, and their overall support.