Markov and Hidden Markov Models

Transition Matrix
When Xt is discrete, the conditional distribution P(Xt | Xt-1 ) can be written as a matrix A where Aij = P (Xt = j | Xt-1 = i) is the probability of going from state i to state j. and each row sums to 1. (stochastic matrix)