SCOPE 34 - Practitioner's Handbook on the Modelling of Dynamic Change in Ecosystems

6.1 MARKOV PROCESSES

Many ecological processes exhibit a great deal of variability, but are nevertheless influenced, if not controlled, by events which have gone before. A Russian mathematician, Markov, who lived early in the 20th century defined one such class of processes, in which the probability of the process being in a given state at a particular time is related to the immediately preceding state of that process. A Markov chain is, therefore, a sequence or chain of discrete states in time or space with fixed probabilities for the transition from one state to a given state in the next step in the chain. In its simplest form, a Markov chain may be regarded as a series of transitions between different states, such that the probabilities associated with each transition depend only on the immediately preceding state, and not on how the process arrived at that state. Such a chain contains a finite number of states, and the probabilities associated with the transitions between the states do not change with time, i.e. they are stationary. A first-order Markov chain takes account of only a single step in the process, but the definition may be extended so that the probabilities associated with each transition depend on events earlier than the immediately preceding one. Furthermore, the Markov chain may exhibit multiple dependence relationships, so that the probabilities associated with each transition depend jointly on more than one previous event.

The mathematical model of the Markov chain occupies an intermediate position in the spectrum of dynamic models, ranging from the classical, deterministic models at one extreme to stochastic models with independent events at the other. In the Markov model a random component is present, so that the state of the system at any point of time or space is not wholly dependent on the previous event or events, but there is, nevertheless, a structure of successive events which defines the process. Many ecological processes can be shown to exhibit the Markov property, i.e. a dependence of the probabilities associated with each transition on the immediately preceding state (or states). Some of these processes can be described by the simplest form of Markov chain (first-order chains with stationary transition probabilities), but many require more complex relationships involving multi-dependence chains with non-stationary probabilities.            

Figure 10. Classification of Markov processes

A simple classification of Markov processes distinguishes between processes in discrete and continuous time, and between states which are either discrete or continuous, as illustrated in Figure 10.

In this account of Markov models, most attention will be given to discrete-time, discrete-state processes, but continuous-time Markov processes are dealt with briefly later.

6.2 TRANSITION MATRICES

The matrix of transition probabilities provides a compact and unique description of the behaviour of a Markov chain. Each element in the matrix represents the probability of the transition from a particular state (represented by the row of the matrix) to the next state (representing the column of the matrix). Assuming a fixed number of possible states, the transition to and from every state can be described by a single matrix. A Markov transition matrix with three states, S₁, S₂, and S₃, is given (Figure 11), where each P_ij represents the probability of the transition from state S_i to state S_j. Because all events from anyone state must either remain in the same state or move to one of the others, the sum of the probabilities in each row is exactly 1.0. There is, however, no necessity for the sum of the probabilities in the columns to equal any fixed value. 

Figure 11. Markov transition matrix

Figure 12. Hypothetical transition matrix for vegetation succession

As an example, consider the hypothetical transition matrix of Figure 12, where the transitions are assumed to take place over a long period of time, say 20 years. Only three states are included in this example, so that 3 x 3 transition matrix is sufficient to describe the process completely, where S₁ represents post-fire vegetation, S₂ represents scrub vegetation, and S₃ represents wood-land vegetation. This matrix defines the only possible transition from post-fire vegetation as one to scrub vegetation. There is absolute certainty (P₁₂ = 1), therefore, that post-fire vegetation will be succeeded by scrub vegetation ( P_l3 = 0). Where scrub vegetation already exists, however, there is a zero probability of a transition to post-fire vegetation, a probability of 0.8 of remaining as scrub, and a probability of 0.2 of a transition to woodland. When a woodland vegetation has been established, there is a probability of 0.9 of the woodland vegetation remaining, and a probability of 0.1 of returning to post-fire vegetation, but a zero probability of returning to scrub. The system can only reach the scrub vegetation state after passing through the post-fire vegetation state.

An alternative method of representing the transition probabilities of a Markov matrix is the Markov transition diagram shown in Figure 13. The arrows from each state indicate the possible states to which a transition may be made, and the values beside each arrow show the probability of the transition being made.

Figure 13. Markov transition diagram

It is not argued that the transition probabilities in this example are representative of any particular ecosystem. For any real application, the probabilities would have to be based on observed frequencies, derived from long-term monitoring of changes taking place in post-fire vegetation. It can be shown that some actual sequences in vegetation succession exhibit the Markov property. Where this property can be demonstrated, the transition matrix is a useful analytical tool, providing a simple way of describing in probabilistic terms a succession of events in time or space. An alternative use of the transition matrix, however, is as a regulating mechanism in the construction of simulation models of the dynamics of change in ecosystems, and examples of this type of application are given below.

6.3 CALCULATION OF TRANSITION PROBABILITIES

The construction of Markov models requires three broad groups of information:

1. some classification that, to a reasonable degree, separates successional states in time or space into definable categories (the multivariate models of the next chapter are frequently useful in establishing such states);

2. data to determine the transfer probabilities or rates at which states change from one category of this classification to another with time;

3. data describing the initial conditions at some particular time, usually following a well-documented perturbation.

Transition probabilities are commonly based on frequency distributions or tabulations of the number of transitions from each state to each other state in the system under consideration. The frequencies are converted to estimates of the probabilities by dividing each row by its total. Alternatively, the transition probabilities may be computed from a series of integers representing different states by the simple BASIC algorithm given in the Appendix. Input to the program consists of a sequence of integers in which the value of each integer denotes a state. Thus, if there are six states in the sequence, the integers 1 to 6 might be used.

A test for the Markov property has been suggested by Anderson and Goodman (1957) as:

where P_ij = probability in cell i, j of the transition probability matrix
P_j = marginal probabilities for the j^th column

n_ij = transition frequency total in cell i, j of the original count of the observed transitions
m = total number of states

The test distinguishes between the two alternative hypotheses:

H₀ : the successive events are independent of each other
H₁ : the successive events are not independent

If the null hypothesis ( H₀ ) is rejected, the successive events may form a first-order Markov chain. The value -2 log_e l is distributed asymptotically as x² with (m -1)² degrees of freedom. A BASIC algorithm for this test is given in the Appendix.

It is convenient to assume that Markov transition matrices are the result of processes that are stationary in time or space, i.e. that the transition probabilities do not vary with either time or space. If a very long sequence of observations is available, or if several sequences of observations from different locations are available, separate transition matrices can be calculated for each sub-interval or location. For a stationary process, these matrices should at least be similar, and Anderson and Goodman (1957) also suggested a test for stationarity. In a stationary Markov chain, P_ij is the probability of a transition from state i at time t-1 to state j at time t. In a non-stationary Markov chain, the transition probabilities vary with time (or space). Thus, P_ij(t) is the probability of a transition from state i to j, and is a function of time (or space). The null hypothesis to be tested is that P_ij(t) = P_ij for all t = 1, 2, ..., T. In other words, the test is to verify that the transition probabilities calculated from each sub-interval of time (or space) are equal to the pooled transition probability matrix obtained by estimation over the whole sequence. The alternative hypothesis is that the Markov process is non- stationary, i.e. 

P_ij(t) ¹ P_ij.

The suggested test is:

where m = number of states
           T = number of sub-intervals
      n_ij(t) = frequency count for the transition from state i to state j in the t^th sub-interval.

The value -2 log_el is distributed as x² with (T- 1)(m(m -1)) degrees of freedom. If the null hypothesis of stationarity is not to be rejected, the calculated value of x² must be less than the tabulated value at some preselected level of significance for the appropriate number of degrees of freedom. A BASIC algorithm for the test is given in the Appendix. 

Figure 14. Three-state Markov chain with n time steps

6.4 POWERS OF MARKOV TRANSITION MATRICES

Although we have so far only considered a single step in a Markov chain, the probabilities of multiple step transitions can be calculated readily by multiplying the transition matrix by itself an appropriate number of times. If the system begins in state P_i, the probability that it will be in state P_j after n steps may be denoted by P_ij ⁽ⁿ⁾. Note that this notationx does not indicate the nth power of the element P_ij, but indicates instead the probability of passing from state i to state j in n time steps. Figure 14, therefore, represents a three-state Markov chain after n steps.

It can easily be shown that the values of the probabilities for a transition matrix after two successive steps are given exactly by multiplying the one-step transition matrix by itself, or P⁽²⁾ = PP. Similarly, a three-step transition may be written as P⁽³⁾ = P⁽²⁾ P, and, in general, for the nth step, we may write P⁽ⁿ⁾ = P^(n-1) P. An algorithm for the successive powering of transition matrices is given in the Appendix.

6.5 CLASSIFICATION OF STATES AND MARKOV CHAINS

Before exploring the kinds of analysis made possible by the use of Markov chains it is important to identify the components of a simple classification of Markov chains. In particular, it is necessary to distinguish between two principal kinds of states, transient and closed. A composite Markov chain may be composed of several transient states and one or more closed states.

A closed set contains one or more states that have the property of confining a Markov process once it enters any of the states of the set. If the closed set contains more than one state in which communication between states is possible, the Markov process can move from state to state within the set, but it can never leave the set. If more than one closed state is present, the Markov process will eventually be confined within one of the closed sets, and will now not be able to reach any other closed sets. If only one state is present in a closed set, that state is an absorbing state, and the Markov chain is an absorbing Markov chain.

A transient set contains only states that are temporary and, where combined with a closed set, the transient set leads the process towards the closed set. However, not all Markov chains have a transient set or sets. A matrix containing more than one closed set, but no transient set, necessarily consists of two or more unrelated Markov chains that have been considered together needlessly. Without the transient set, there can be no communication between the closed sets, and the chains represented by the different sets can be studied separately. A Markov process without an absorbing state represents a process that is constantly in transition, and is known as an ergodic Markov chain.

Figure 15. Diagrammatic representation of transient, closed and absorbed states

Figure 15 gives a diagrammatic representation of transient, closed and absorbing states. State A is an absorbing state. Once the process enters this state, it does not leave it. Similarly, the states D_l, D₂, and D₃ represent a closed set. Having entered D_l, the process can move to D₂ or D₃, but cannot make a transition to any other state. In contrast, the states B_l, B₂ and B₃ represent a transient set, linking the absorbing state A to the closed set D.

By rearranging the order of the states, or by algebraic analysis, it is always possible to partition a transition matrix so as to demonstrate the existence of transient sets of states, closed sets of states, or absorbing states, as in 

where s = number of transient states
r- s = number of closed states
Q = square sub-matrix, containing s _* s elements, for transitions between transient states
R = rectangular sub-matrix containing s _* (r -s) elements, for transitions from transient set to closed set or sets
S = square sub-matrix containing (r- s) _* (r -s) elements, forming a class of closed sets from which there is no exit after entry
O = sub-matrix, containing s _* (r -s) elements, all of which are zero and therefore represent no transition

Figure 16. Analysis of transition matrices

Figure 16 Where a transition matrix can be partitioned in this way, the several components can be investigated separately, thus simplifying the task of studying and modelling the ecosystem. Analysis of the matrix of transition probabilities also simplifies the calculation of the average times needed to move from one state to another, and of the average length of stay in a particular state once it has been entered. Where closed or absorbing states exist, the probability of absorption and the average time to absorption can also be calculated readily. Examples of these calculations are shown for a simple model below.

6.6 ANALYSIS OF AN ERGODIC MARKOV CHAIN

Raised mires frequently show interesting successional changes as a result of increased drainage, and Figure 17 gives the estimated probabilities for the transitions between four possible states of a raised mire over a period of 20 years. State 1 represents the wettest facies dominated by Sphagnum, with Calluna, Erica tetralix and Eriophorum as the major vascular plant components. State 2 represents a drier facies, with a Calluna-Cladonia association and seedlings of Betula and Pinus sylvestris, the more mature woodland of State 3 having a typical Vaccinium myrtillus community with hypnaceous mosses. State 4 represents disturbance due to grazing by large herbivores of the drier facies, leading to the establishment of a Molinia-Pteridium-dominated association.

Thus, areas which start as typical bog vegetation have a probability of 0.65 of remaining as bog vegetation at the end of the 20-year time-step, and probabilities of 0.29 and 0.06 respectively of becoming Calluna-dominated and woodland. Areas which start as Calluna-dominated have roughly equal probabilities of remaining in the same state, returning to bog vegetation because of fluctuations in the water table, or of becoming woodland: they have a small (0.07) probability of being subjected to sporadic grazing. Woodland areas have a 0.69 probability of remaining as woodland, a probability of 0.28 of returning to Calluna because of deaths of trees, and, again, a small (0.03) probability of being subjected to sporadic grazing. The grazed areas have an equal probability of being subjected to further grazing and of returning to a Calluna-dominated vegetation, and a small (0.20) probability of becoming woodland because of the growth of ungrazed seedlings.

Figure 17. Transitional probabilities for successional changes in a raised mire (time step = 20 years)

None of the states, therefore, are absorbing or members of a closed set, and the matrix represents a transition from the bog vegetation to woodland, with an imposed disturbance due to grazing. However, although there can be a return from the Calluna-dominated vegetation to bog vegetation because of fluctuations in the water table, there is no immediate return from woodland to bog vegetation. Where there are no absorbing states, the Markov process is known as an ergodic Markov chain, and the full implications of this model can be exploited by an analysis of the transition matrix.

Using the algorithm of the Appendix, it is possible to calculate the transition probabilities after two, three, four, ..., time steps. Thus, for the transition matrix of Figure 17, the corresponding probabilities after two time steps are:    

and after four time steps 

Alternatively, the Appendix gives a BASIC program which will calculate directly the transition probabilities after a given number of time steps.

If a matrix of transition probabilities is powered successively until a state is reached at which each row of the matrix is the same as every other row, forming a fixed probability vector, the resulting matrix is called a regular transition matrix. The matrix then gives the limit at which the probabilities of passing from one state to another are independent of the starting state, and the fixed probability vector expresses the equilibrium proportions of the various states. In the example above, the vector of probabilities is: 

[0.2177  0.2539   0.3822  0.1462]

If, therefore, the transition probabilities have been correctly estimated and remain stationary, the raised mire will eventually reach a state of equilibrium in which approximately 22% of the mire is bog, and approximately 25%, 38% and 15% are Calluna, woodland and grazed communities, respectively.

Although the limiting or equilibrium probabilities can be calculated by successive powering of the transition matrix, a quicker method is given in the BASIC algorithm of the Appendix. The same program calculates the mean first passage times defined as the average length of time required to move from any one state to one of the other states. Figure 18 gives the mean first passage times for the transition matrix of Figure17. Each element of the mean passage time matrix needs to be multiplied by the time step of 20 years. Thus, the average length of time a Calluna-dominated area takes to become bog is 9.566*20 = 191 years. Similarly, the average length of time needed for woodland to become Calluna is 4.107*20 = 82 years, and the other times can be calculated as required.

Figure 18. Mean first passage times for raised mire system

Alternatively, if an area is chosen at random, the average length of time needed for that area to reach any of the defined states is given by the mean first passage times in equilibrium, also calculated by the program of the Appendix. For the raised bog system, the mean first passage times in equilibrium are given by the vector:

[10.385  3.676  3.627  25.351]

Remembering again that each time step represents 20 years, the mean first passage time for a randomly chosen area to become bog is 10.385*20 = 208 years, while the corresponding mean first passage times for Calluna, woodland and grazed communities are 74, 73 and 507 years, respectively.

6.7 ANALYSIS OF AN ABSORBING MARKOV CHAIN

In the analysis of an absorbing Markov chain, in contrast to that of an ergodic chain, the main emphasis is necessarily focused on the average time for the system to reach the absorbing state, and on the probability that the system will reach this state from any given starting state. Figure 19 gives a version of the transition probabilities for the raised mire system, modified so that the woodland state becomes an absorbing state. It is then convenient to rearrange the matrix so that the absorbing state is partitioned from the set of transition states.

Figure 19. Modified transition probabilities for successional changes in a raised mire (time step = 20 years)

Figure 20. Fundamental matrix for raised mire transition matrix modified to create an absorbing state

The appropriate analysis for an absorbing Markov chain is given by the algorithm of the Appendix. This algorithm first calculates the so-called fundamental matrix which summarizes the number of time steps for which the system will be in the state indicated by the column before being absorbed, given that the present state is that of the row. The fundamental matrix for the modified raised mire system is given in Figure 20 and indicates, for example, that 2.21 is the number of time steps that the system will, on average, be in the Calluna state, having started as bog, before being absorbed into woodland. Where there is only one absorbing state, the probability of reaching that state is necessarily equal to 1.0, but, where there are two or more absorbing states, it is important to determine the probability of reaching each of these states from each of the possible starting states, and the program also calculates these probabilities. Finally, the same program calculates the number of time steps, on average, that the system will take to reach the absorbing state from each of the possible starting states. For the modified transition matrix of Figure 19, these numbers of time steps are summarized in the following vector:

[7.22  5.27  5.18]

6.8 DEPENDENCE, ORDER AND STEP LENGTH IN MARKOV CHAINS

It is customary to classify Markov chains in terms of their dependence, order and step length. The Markov chains illustrated so far in this chapter have been dependent only on the immediately preceding state, and are therefore defined as being single-dependence chains. They are also first-order chains because this preceding state is immediately preceding. The transition involves a single step of unit length, equivalent to the length of the time step.

It is, however, possible to define Markov chains with dependency relationships that involve more than one preceding step, and a double-dependence chain, for example, is one that is dependent on two preceding states. If these two states are the two immediately preceding states, the chain is a second-order chain, and either or both of these steps may be of greater than unit length. An algorithm for analysing and testing for the double-dependence Markov property is given by Harbaugh and Bonham-Carter (1970). Input to the algorithm includes a sequence of states for which a transition-frequency matrix is calculated and subsequently transformed into a transition probability matrix. In addition, the algorithm calculates a maximum likelihood criterion test statistic which is similar to that for first-order Markov chains. The null hypothesis under test is that the chain is singly dependent against the alternative that it is doubly dependent. The program calculates this test statistic for each value of the second memory step length.

6.9 DISCRETE-STATE CONTINUOUS TIME MODELS

In the usual discrete-time Markov chain, the system advances in a series of discrete time steps, with transitions occurring at each of the steps. In a continuous-time Markov model, a matrix of transition rates q_ij is employed. These transition rates not only indicate the probability of one state succeeding another, but also establish the expected waiting time in each state.

In the continuous-time model, attention is focused less on the probability of a state change, P_ij (where i = j), than on the rate of transfer, q_ij, from state i to state j. The transition rates may be obtained by direct observation, or alternatively they can be derived by algebraic transformation of the P_ij matrix. The following equations relate transition probabilities to transition rates:

where M_i = sum of the off-diagonal elements in the i^th row of the q_ij matrix 

and k is the number of states.
P_i = sum of the off-diagonal elements in the i^th row of the P_ij matrix  

where P_ij is the diagonal element.

Rearranging the first of these equations:

The reverse transformation from the q_ij matrix to the P_ij matrix can be made by using the same equations. In this case, the values of q_ij are known, and the values of M_i are calculated by summing each row of the qij matrix. The values of P_ij are then determined directly from the equations. Harbaugh and Bonham-Carter (1970), from whom this section is taken, also give a FORTRAN algorithm for accomplishing these transformations in either direction.

6.10 GENERATING SEQUENCES OF EVENTS FROM MARKOV CHAINS

In addition to the analysis of ergodic and absorbing Markov chains described above, it is often of interest to generate sequences of events from a matrix of transition probabilities or rates. By their very nature, involving probabilities, the outcome of such sequences will vary from simulation to simulation, and this variation may be exploited to show the essential variability of the system, possibly derived from genetic or environmental causes.

The simplest of these sequence generation procedures is that depending on a simple discrete event, discrete-time transition matrix. At each time step, the program refers to the given transition matrix and determines the state to which the system moves by generating a random number. This computation can be repeated as often as is necessary to determine what may be regarded as the 'final' state. Where a two-dimensional system is being simulated, the result may be a computer-generated map.

The principles of simulating double dependence Markov chains are identical to those used in generating single dependence chains. Input to the program includes the number of states, the length of the second step in whole-number multiples of the unit time step, the number of transitions to be generated, and the transition probabilities. Several two-dimensional double dependence transition probability matrices may be read and stored in a multi-dimensional array.

6.11 MARKOV-TYPE MODELS, BASED ON DIFFERENTIAL EQUATIONS

As an alternative to the use of conventional Markov chains, it may be convenient to employ the commonly used and well-documented techniques of feedback and control systems, by using ordinary differential equations to describe the trajectories leading from one state to another. Shugart et al. (1973) give an example of this kind of approach in the modelling of forest succession over large regions. With this procedure, it is necessary to define a set of possible cover states, or stand types, in the same way as for the state-by-state approach of the Markov model.

The transition probabilities used in the Markov model and the rate constants used in a system of differential equations can be calculated from the same data set. Both types of model assume that the replacement of one type of stand by another is dependent only on the present conditions, and that replacement patterns remain constant with time and are not affected by spatial heterogeneity. The approach is therefore essentially Markovian in character, and the results from a conventional Markov model and from the system described by differential equations, also derived from the same data set, should be identical.

The choice between the two approaches may often depend on the objectives of the study, as well as on the knowledge and mathematical expertise of the user, but, when a straightforward Markov approach can be used, it is usually to be preferred. Not only do the possibilities of further algebraic analysis lead to a better appreciation of the stochastic nature of many ecological processes, but, by the calculation of such parameters as the mean passage time, time to absorption, and the degree of stability and convergence within the defined states, additional information of direct ecological and management value is provided.

Extensions of the Markov model suggest the use of higher-order chains, where the next state is dependent not only on the present state, but also on one or more previous states. Processes may also be regarded as semi-Markovian, the time spent in a given state (the direct passage time) being variable and dependent on the present and the next state. Certainly, some biolgoical processes might be more accurately represented in this way. Although these modifications of the Markov model illustrate some of the flexibility that is conceptually possible within a basically Markovian approach, the collection of data and the development of appropriate transition matrices become much more difficult.

Data collection for the calculation of transition probabilities and for the construction of transition matrices is a major task, ideally requiring detailed documentation of changes in systems over extended periods of time, and of responses to various types of perturbation. Nevertheless, data of this type do exist, both from historical and experimental records. In other situations, it may be sufficient to use hypothetical transition probabilities so as to show the consequences of particular rates of transition for the future development of a system.

In contrast, when a differential equation approach is used, the analytical techniques developed to explore the control and feedback aspects of such models are directly applicable, and themselves provide useful insights into ecosystem management.

It is, perhaps, worth emphasizing the relationship between the Markov models described in this chapter and the Leslie matrix models described in Chapter 5. The Leslie matrix describes the transition probabilities from one time to another, the state of the population at time t + 1 being dependent on the state of the population at time t. However, the probabilities in the Leslie matrix do not sum to unity in either the rows or the columns, and the Markov theory is not therefore applicable to these matrices, despite their apparent similarity.

6.12 ADVANTAGES AND DISADVANTAGES OF MARKOV-TYPE MODELS

The advantages of Markov-type models may be summarized as follows.

1. Markov models are relatively easy to derive (or infer) from successional data.

2. The Markov model does not require deep insight into the mechanisms of dynamic change, but it can help to indicate areas where such insight would be valuable and hence act as both a guide and stimulant to further research.

3. The basic transition matrix summarizes the essential parameters of dynamic change in a way that is achieved by very few other types of model.

4. The results of the analysis of Markov models are readily adaptable to graphical presentation, and, in this form, are frequently more readily presented to, and understood by, resource managers and decision- makers.

5. The computational requirements of Markov models are modest, and can easily be met by small computers, or, for small numbers of states, by simple calculators.

The disadvantages of Markov models are as follows.

1. The lack of dependence on functional mechanisms reduces their appeal to the functionally orientated ecologist.

2. Departure from the simple assumptions of stationary, first-order Markov chains while, conceptually possible, makes for disproportionate degrees of difficulty in analysis and computation.

3. In some areas, the data available will be insufficient to estimate reliable probability or transfer rates, especially for rare transitions. For example, it may not be possible to observe sufficient transitions from a given transient set of states to a closed state where this transition is dependent on a rare climatic event, even though the value of this parameter is of vital importance in the dynamics of the community.

4. Like all other successional models, the validation of Markov models depends on predictions of system behaviour over time, and is therefore frequently difficult, and may even be impossible, for really long periods of time.

These difficulties warrant a high degree of caution in the uncritical use of Markov models in ecology. Of the disadvantages listed, the second is perhaps of the greatest significance because it is doubtful whether most ecological successions do, in fact, have homogeneous transition matrices that are constant in time. A constant transition probability implies that the future behaviour of a system is determined only by its present state and is independent of the way in which this state has developed. It also implies that the replacement of, for example, an individual of species j by one of species k does not vary over the time units used in calculating the transition probabilities.

Both of these assumptions run counter to many widely accepted biological ideas concerning evolution, adaptive strategies and the occurrence of episodic events such as abnormal weather, natural catastrophes and epidemics. However, the accumulating evidence that the initial floristic composition, after a perturbation, dominates the subsequent successional patterns supports the view that past history may not be of primary importance in determining successional sequences following the perturbation, even though it is of a major factor in influencing the suites of species that exist at a site and are therefore available to participate in the succession. Horn (1975) has suggested that the episodic event, shifting the course succession from one pathway to another , may well be included in the Markov model by switching from one transition matrix to another.

6.13 CASE STUDIES OF MARKOV MODELS

Although the mathematical basis of the Markov model has been well known to mathematicians for many years, there have been relatively few practical applications of the model to studies of ecosystem dynamics. Whether this lack of application is due to the inaccessibility of the theory to non-mathematicians or to the practical difficulties of collecting data from which to estimate the transition probabilities is not clear, and perhaps both of these reasons have contributed to the slowness of practical use of the technique in ecology.

One of the early applications of Markov models to ecology was by Williams et al. (1969) in their study of rainforest communities. In these studies, ten sites were cleared in tropical forest, and the presence and absence of species on twelve occasions over a period of approximately seven years were recorded, the records being subsequently classified into seven states. By counting the frequency with which one state followed another, the transition probabilities of a Markov process were estimated. Stephens and Waggoner (1970) and Waggoner and Stephens (1970) used a similar approach to predict the composition of a mixed hardwood forest in the eastern United States from the enumeration of natural transitions over a 40-year period. Some of the best-known examples, however, of the use of Markov models to characterize forest succession were given by Horn (1974, 1975), in which the succession of species was estimated by simple approximations of the tree-by-tree replacement within the stand. Extrapolation of the Markov model gave a forest composition which was very similar to areas of forest which were known to be very old, and which were believed to be largely untouched by man.

Other studies of the use of Markov models in forests include those of Peden et al. (1973), Cassell and Moser (1974), and Lembersky and Johnson (1975). More recently, Binkley (1980) as applied stationary Markov models to three kinds of problems arising in forestry, including individual tree level models of forest succession, plot canopy gap level models of forest succession, and stand level analyses of specific forest management problems. Buongiorno and Michie (1980) have also developed a Markov model of a selection forest, where the parameters of the model represent the stochastic transition of trees between diameter classes and the recruitment of new trees into the stand. The parameters of this model were estimated from north-central United States region hardwoods, and the model was used to predict long-term growth of undisturbed and managed stands. Subsequently, a linear programming method was used to determine sustained use of management regimes which would maximize the net present value of periodic harvests. The method allows for the joint determination of optimum harvests, residual stock, diameter distribution and cutting cycles.

Applications to other types of ecological system are even rarer than those of forests, but Debussche et al. (1977) describe an agricultural application to the southern end of the French Massif Central. This model considered the rural exodus and decrease of sheep grazing, and the best utilization of the grazing resources. The mapping of vegetation types, study of the dynamic changes from one type to another and the definition of key species were used as a basis for the formulation of the model. Vandeveer and Drummond (1978) have also used Markov processes for estimating land use changes, particularly where a major impact such as a reservoir is imposed upon an existing system.

Other biological applications of Markov processes include that by Rao and Kshirsagar (1978), who made a study of the population dynamics of predator/ prey systems in which the attack cycle of a predator is assumed to consist of four different activities, namely search, pursuit, handle and eat, and digestion. A semi-Markovian model was proposed to obtain the number of prey devoured by a predator during the activity of a day. Usher and Parr (1977) suggest that two kinds of succession could be recognized in a decomposer communities, one related to habitat change (plant species, micro-environment), and the other to the decomposition stages of the decaying food resource. Data are given for the decomposition of wood by termites in West Africa, and a succession of both plants and soil arthropods on the developing chalk soil in Britain, and it is suggested that both processes are Markovian in character. Usher (1979), in an analysis of published studies, suggests that ecological succession can usually be considered as a non-random process, and that complex non-random or Markovian processes are likely to characterize all ecological successions. The elements of the transition probability matrix are, however, unlikely to be constant, but are functions of either the abundance, or the rate of change in abundance, of a recipient class. Glaz (1979) provides explicit formulae for the absorption probabilities, means and variance of first absorption times in terms of birth and death rates in finite homogeneous birth and death processes.

Tavare (1979) has proposed a simple and intuitive method of deriving some properties of finite homogeneous continuous-time Markov chains in population genetics. The chains have absorbing barriers, and the method involves the representation of such a process in terms of a discrete Markov chain, and a series of waiting times which reconstruct the original timescale.

Although not strictly applied to ecological problems, the text by Bartholomew and Forbes (1979) gives practical applications of Markov and Markov-type methods in manpower planning, and many of the techniques which are suggested are of practical value in the modelling of biological populations. The general-purpose programme included for showing the development of a population from a given starting point is of particular value. Finally, Cook (in press) gives an interim report on an attempt to estimate the transition probabilities of plant-by-plant replacement from aggregate data, i.e. the percentage of sites in each state at regular time intervals, and a least-squares method is proposed for estimating these transition probabilities, but the method is not, as yet, entirely successful.

Geologists have used Markov models to provide a mechanism for computer simulation of a wide variety of geological processes, and Krumbein (1967) emphasizes the value of first-order Markov chains because of their intuitive appeal, particularly in stratigraphic analysis. The text by Harbaugh and Bonham-Carter (1970) contains a chapter giving a general account of Marko models, with applications to geology and stratigraphy. This chapter contains several useful FORTRAN programs for critical computations. Norris (1971) gives a comparison of different methods in the transition matrix approach to the numerical classification of soil profiles. Bhattacharya et al. (1976) suggests a Markovian stochastic basis for the transport of water through unsaturated soil, and Lloyd (1977) provides a theoretical proposal for the modelling of reservoirs by a first-order Markov process. No actual example is given of the application of these processes, however, and the major part of the paper is devoted to the problem of obtaining the probability distributions of recurrence times and first passage times. Smeach and Jernigan (1977) present further aspects of a Markovian model sampling policy for water quality monitoring.

Finally, Anthony and Taylor (1977) explore the use of Markov models in forecasting air pollution levels. The analysis of historical data concerning variations in air pollution indices suggests a pattern which might usefully be described by a transition probability matrix. By utilizing what is essentially a Markov-based framework, the model suggests a number of interesting and important questions and facilitates their solutions. In their presentation, the authors focus primarily on the variations in a single component of air pollution, i.e. suspended particulate matter. For implementation of the model it would be necessary to handle other components in the same manner.