ON A STABILITY OF
STOCHASTIC PROCESSES
by
Per-
1 A background
The notion of stochastic process is considered difficult
and there is in the literature a variety of definitions or representations of
that concept, all of which are supposed to be formally equivalent but do have
different flavor to intuition. To get some idea of what we are up to, let us
ignore all formal details and consider the situation depicted by Fig. 1: At the time a particle is released at A
and moves to the right because, say, there are a barrier to the left of the
Y-axis and a transition probability (for motion to the right) acting on the
particle. The particle’s position at the time
is recorded. The experiment is repeated many
times and at each experiment the location
is observed and recorded. After, say, 100 such
experiments the recorded positions of the particle form a realization of a
probability distribution indicate by (a) in Fig.
1. Assume that we also record each particle’s location
at
as well: this gives a distribution indicated
by (b) in Fig. 1.
Fig. 1 A
random walk as illustration of the notion of stochastic time process. Let a
particle be released at A at the time
.
Because of some mechanism acting on
it moves to the right and its position is
observed at the times
and
.
The experiment is repeated many times and the recorded positions form two realizations
of probability distributions. In general one should expect the standard
deviation at
to be larger than that at
.
However, the difference is often less than one would expect, because stochastic
time processes in nature usually contain a deterministic part that gives a kind
of stability to the processes. It is this deterministic part that, when
sufficiently dominating, allows for deterministic data analysis. Often there is
a co-variance between the two data clusters (it decreases when the
time-distance between the two clusters increases), and this means that data
from such processes contain less information than ones intuition expects (e.g.
no matter how complex the underlying physics is, kinetic data look usually nice
and smooth).
There is a wealth of questions that this scenario gives
rise to, some of which are of deep mathematical significance. Here I shall
however limit the attention to one basic thing, namely that we consider a
variable
the particles location on the x-axis at the
time t
that for
each fixed
or
it is an ordinary stochastic variable (namely
the particles position at the times
and
respectively), which among various things implicates
that there is a probability distribution with the density
for
such that
is the Dirac delta function
;
that is, at the time
the location of the particle is
with probability one (i.e. at the time
the particle is exactly
at the location point A
in Fig. 1).
Let us now consider the set .
That is, we have now realizations of the one and the
same (Sic!) particle’s location at two consecutive time points. This is
about the simplest possible example there is of a realization of a stochastic
time process, and a typical question in that context is about the covariance between
and
;
for instance, if the two time points are not very far apart one would expect
that if
is well to the left of the mean value
it is likely that also
is located to the left of
.
One should thus expect that consecutive observations of a time process are usually not mutually independent. This is one of the fundamental difficulties with this kind of processes, namely that data from observations of such processes usually do not contain as much information that one intuitively would expect.
There is one aspect of this particular example that I
want to emphasize. Thus, there is an uncertainty (variance) in and it seems intuitively reasonable to assume
that this uncertainty will, so to speak, add to the uncertainty of
or, differently expressed, one would expect
the standard deviation of
to be larger than that of
.
Actually, if we follow the particle for a longer period of time
it would appear natural to expect that the standard
deviation to increases with time. But usually this does not happen
there
is a kind of stability present, and if one should
ask why this is so there is usually no answer (it is somewhat similar to the
common situation with the force of gravitation: physically speaking one has
usually not known what that force is, it has only been possible to note that it
is there). Commonly this “stability” has been represented by considering the
stochastic process as a sum of one deterministic component and one stochastic
component (Fig. 2 in the next section). And often the deterministic
part is represented by the probability density
being a deterministic function of classic
form. Personally I like to express this as: some kind of determinism is
necessary for the stochastic process to have the stability we often can observe but cannot explain. In
fact, this stability is necessary (but of course not sufficient) for the
mathematics of stochastic processes to be practically useful. Therefore,
practically I use to search for forms of determinism in observed stochastic
processes.
One of the expressions of the stability one meets in nature is the possibility of such things as mass production of shoes and costumes (obviously the variation in human size is not totally wild); but this being so, things like complexion and fingerprint reveal the other side of the coin, namely that there is also an underlying variance of quite some significance. And it is imperative that the language we construct allows for such a dichotomy (determinism together with large randomness). Well, occasionally the observable stability one meets in nature is not totally impossible to “understand”: various feedback mechanisms imply a stability that occasionally is referred to as homeostasis, and the high buffering capacity of blood is not only well known but also reasonably understood in terms of classic chemistry. An example of how nature seems to induce stability by counteracting deviations from the “normal” is the notion of regression in classic medical genetics, as exemplified by how tall parents tend to receive children of normal length, and analogous for short parents: also their children tend to be of normal length (the kids grow up taller than their parents).
2 The Faraday rule
When interpreting his experiments in electrolysis Michael Faraday
applied a rule that stated that the data analysis should be done in a language
(grammar often in the form of postulates
and a vocabulary representing concepts) that
was such that the experiments could be discussed without making necessary any
commitments to hypotheses about the unknown and the uncontrollable. For
instance, initially he wanted to name the two electrodes (in an electrolytic
cell) east and west but a philologist whom he consulted pointed out that this
implied a hypothesis about a connection with magnetism and this violated
Faraday’s own principle (this was before the work of Maxwell); the eventual
result of this terminology-fuss was that the two names anode and cathode were
coined (Williams, 1971).
Personally I like to formulate Faraday’s rule as:
The language must be such that ease of use is obtained through concepts that do not ignore but are independent of the factors that cause difficulties.
This principle has been widely applied during the years. For instance, the meaning of “rate constant” in classic chemistry does not ignore but is independent of the complexities of chemical reactions that are dealt with in quantum chemistry. And the significance of “mean-value” in statistics is by and large independent of the look of the underlying distribution.
It is in the light of this rule I often like to contemplate the usefulness of the theory of stochastic processes, namely that one often can in exact terms speak of the probability of a physical event without getting involved in the nature of the underlying physical mechanisms. Usually such mechanisms are crushingly “complex” and an analysis in those terms necessarily becomes speculative and violates Faraday’s rule (in other words, the whole thing becomes messy). That is, the various probability concepts can be given physical significance independently of the underlying mechanisms in terms of more classic physics. This is one of the features of probability that makes this notion so powerful.
However, to use the theory of stochastic processes for this reason carries a cost, namely that the very idea of stochastic process means that one introduces elements of randomness (that mask, as it were, the investigator’s ignorance about the underlying physics). And it is here that the stability matter discussed in the preceding section becomes important, because without this stability the theory’s usefulness would be lessened at design and interpretation of experiments.
In order to handle the situation it is important to note
that it is very much a matter of selecting appropriate concepts. Yes, it
becomes a linguistic problem as it were, and the primary task is to visualize
the notion of stochastic process ,
and the empirical dichotomy “stability together with variance”. And as already
hinted in the preceding section, to this end I like the representation
Fig. 2 A
rather general representation of a stochastic process. It is looked upon as a
sum of one deterministic
part and one purely nondeterministic part
,
and it supplies the reader with a somewhat acceptable “feel” for how the two
properties contribute to observable features of the Nature’s behavior.
where is the deterministic part (a time function of
classic sense) and
is the purely nondeterministic part (cf. Cramér and Leadbetter, 1969); often it is convenient
to write the mean value
,
so that
.
In these terms
and using standard probability algebra
it is not difficult to see how the randomness balances the deterministic feature and, above all, the
converse. That is, because of the stability discussed in the preceding section
it becomes urgent to recognize the determinism that obviously does exist in
nature; the necessity of such an existence follows from the representation
depicted in Fig. 2, together with the empirical facts discussed in the preceding
section. It is of course a matter of description: any attempt to “explain” the
situation would be contra productive. Again, it is very much a linguistic matter, namely how to describe and conceive
the dichotomy that Mother Nature supplies us with.
It should be noted here that the treatment of random walk in standard textbooks usually shows how the variance increases with time (i.e. with increase of the distance the particle moves). The problem here is that often this variance is under some form of control, and Fig. 2 suggests one possibility to represent this phenomenon, which suggests that the influence of the deterministic element should increase with time. The formal expression of “how” is outside the present discussion; perhaps one might say that the search for deterministic factors constitutes a significant part of the work on the topic. In that context it is somewhat interesting to observe that as early as 1944 Erwin Schrödinger predicted the existence of a stabilizing physical factor like a crystal transferred from generation to generation. This was in order to “understand” the biologic stability as the one I have entertained in Section 1 above, illustrated by man’s limited variation in size. That was thus years before the discovery of the DNA molecule.
REFERENCES
Cramér, H. and Leadbetter, M.R. (1967): Stationary and Related Stochastic Processes.
John Wiley & Sons, Inc.
Schrödinger,
E. (1944): What is life?
Williams, L.P. (1971): The
Selected Correspondence of Michael Faraday. Volume 1.