Determining the Transition Matrix

Back when I was introduced to Markov chains in Robert Messer's Linear Algebra, the prime real-world examples of what Markov chains might model were interacting populations in a biosphere. Oddly though, there was no mention of how one might determine the transition matrix based on observations of actual populations. Nor can I seem to find a simple explanation of this online. So here is my own calculation.

We wish to determine the Markov Chain transition matrix based on an observed set of transitions where the dynamics are unknown. The update function is
where p is the probability distribution, q is the old probability distribution, and M is the transition matrix. In order to find the optimal transition matrix, we minimize the variance
where d is the dimensionality and E is the size of the statistical ensemble over time and space to which p and q belong. The index of the ensemble e is suppressed. We find the differential
Setting the differential to zero we get
which, assuming M_aj is known for all j ≠ b gives the solution
Our strategy then is to iteratively increase the size of the ensemble of observations and compute the transition matrix for each iteration. For j ≠ b we use the value M_aj from the previous iteration. For the initial matrix there are many possible choices. We will use

To be continued...