A difference forgets a drift

An earlier essay here ended with a proposal, stated in two sentences and offered as a bargain:

If a plant’s disturbances are inherited, the divergence sequence carries a slow wander as well as a comb: block means over a hundred organs vary several times more than independent errors would allow. If they are merely shared between touching organs, the sequence has teeth at the contacts and no wander.

It was priced as the cheap half. A comb costs about nine hundred organs at half a degree of protractor precision; a variance ratio is not a spectral feature and needs no extra measurements, so a stem long enough for the comb is more than long enough for both. Two statistics off one sequence, and they disagree about the hypothesis.

The proposal is wrong, and the arithmetic that shows it is one line that was never written down.

The line that was never written down #

A disturbance acts on positions. Organ i sits at i divergences plus an error eᵢ, and the divergence between consecutive organs is therefore

δ + (eᵢ − eᵢ₋₁)

A divergence is a difference. And differencing a series is precisely the operation that removes its low-frequency power: a drift moves eᵢ and eᵢ₋₁ by nearly the same amount, and the difference between them does not see it.

The wander is real. It is in the disturbance. The divergence sequence — the one thing about a plant a botanist can actually record — is the one place it is not.

The two columns #

Block means over a hundred, as a multiple of what independent draws would give:

The disturbance with the largest wander of any shape this collection has built leaves none in the sequence. A disturbance with a memory in time keeps most of its own. The two columns are not two versions of one measurement; they reorder the disturbances almost completely.

That reordering is the whole finding, and it is why the proposal was not merely imprecise. It named the transport as the shape that would show a wander, and the transport is the shape that shows none.

Why the transport in particular #

The reason is more specific than “differencing removes drift”, and it is worth having because it says what a wander is evidence of.

Write the statistic properly. For a differenced stationary stream the variance of a block mean of B values is 2σ²(1 − ρ(B))/B², so the quantity that is one for independent errors is

W(B) = B²·Var(block means)/Var(divergences) = (1 − ρ(B))/(1 − ρ(1))

where ρ is the autocorrelation of the disturbance. Measured by two routes that share no arithmetic — one a variance of block means of the angles, the other a lagged product of the deviates — the two agree everywhere to inside the sampling error of the noisier of them, across seven disturbances and eight block sizes.

So W(B) is the autocorrelation function, evaluated at the block size instead of at a lag. It is not a second statistic at all.

What the formula lets W do, and what it forbids #

Written as (1 − ρ(B)) ÷ (1 − ρ(1)), the statistic stops being a black box and its whole behaviour follows in three lines.

W is above one exactly when ρ(B) is below ρ(1) — when the disturbance is less correlated at the block size than at one step. A memory decays, so every block size is less correlated than lag one, and W climbs. That is the wander.

W is below one exactly when ρ(B) is above ρ(1) — when the disturbance is more correlated at the block size than at one step. A contact transport has almost nothing at lag one and a great deal at eight and thirteen, so W dips there. That is the hole.

And W has a ceiling. Its largest possible value is 1 ÷ (1 − ρ(1)), reached when the correlation at the block size has fallen to nothing. For the memory at ρ = 0.9 that ceiling is ten — and the measured value in the divergences is 10.25, which is the ceiling to within two and a half per cent.

So the memory is not merely showing a wander; it is showing the largest wander its own lag-one correlation permits, and the number that looked like a measurement of drift is a measurement of ρ(1).

That is the sharpest form of the essay’s finding. The two columns do not reorder the disturbances because differencing is capricious; they reorder them because the second column is a function of ρ(1) and ρ(B) and the first is a function of ρ(B) alone.

The replacement is the same statistic read the other way #

Which means the specification did not lose a statistic and gain another. It lost a reading and gained a different reading of the same function.

The wander and the hole are W above one and W below one. Both are computed from the same block means of the same sequence at the same cost, and the choice between them is which block sizes to look at: any of them for a drift, and the two contact offsets for a transport.

That is worth stating plainly because the statistic was replaced with a cheaper one would overstate what happened. Nothing new was built. What changed is knowing which side of one to look at, and that the block sizes to look at are numbers a count of the shoot supplies before the sequence is taken.

And that settles the transport. A contact transport puts its correlation in a residue class — at eight, thirteen, twenty-one, and at their sums and differences, and at nothing else. Its autocorrelation at a general large lag is near zero, so W is near one, and no amount of coupling changes that. Nothing whose correlation lives in a residue class can produce a wander, which is a statement about arithmetic rather than about this particular model.

The number the identity supplies #

An identity is worth more when it predicts something with no free parameters, and this one does. For a disturbance that remembers the last error with coefficient ρ, the autocorrelation is ρ^B, so W climbs to 1/(1 − ρ) and stops.

Measured against the closed form at five coefficients: 1.38 against 1.43, 1.90 against 2.00, 3.15 against 3.33, 9.51 against 10.00, and 30.94 against 33.33. That is 0.93 to 0.97 of the closed form at every coefficient — a uniform low bias, which is what a variance of block means does in a finite run, rather than a disagreement about the shape.

What the statistic is, in detail #

It is worth writing the recipe out, because a variance ratio has more than one plausible normalisation and the wrong one hides the whole result.

Take the divergence sequence. Cut it into consecutive blocks of B, take the mean of each block, and take the variance of those means. Divide by the variance of the sequence itself, and multiply by B squared.

The square is the part that has to be right. For an ordinary series — one that is not a difference of anything — block means have variance falling as 1/B, and the natural normalisation is a single factor of B. A divergence sequence’s block means fall as 1/B² instead, because the block mean telescopes: the mean of eᵢ − eᵢ₋₁ over a block is the last error minus the first, divided by B — a difference of two values rather than a sum of B of them.

Normalised with a single B, every disturbance here produces a tidy curve falling as 1/B and nothing is distinguishable from anything. Normalised with B², independent errors give a flat line at one and everything else is a departure from it. The two normalisations differ by a factor that is the same for every series, so neither is wrong; only one of them puts the interesting structure where an eye can see it.

What the two observables actually separate #

The proposal had the right instinct and the wrong pairing. There are two observables, they are almost independent, and they separate two different kinds of inheritance:

• A comb says the disturbance is correlated at the contact offsets and at combinations of them. It is produced by an error passed between touching organs and passed on again, and — as this collection has already had to concede — it is produced by nothing else that has been tried.
• A wander says the disturbance is correlated at every lag, with a tail that decays slowly. It is produced by a memory in time: an apex whose conditions drift, a measurement whose zero moves, a shoot growing through a changing season.

A sequence with a comb and no wander is a shoot whose errors are shared between neighbours and not carried through time. A sequence with a wander and no comb is a shoot in a drifting environment with independent local errors. A sequence with both is a shoot with both, and there is no reason it should not have.

What is left of the wander, and it is not nothing #

Two things survive, and both are sharper than what they replace.

A hole at the comb’s own lags. An inherited disturbance drives W down where the block size is one of the offsets it couples at: 0.22 at a block of eight and 0.21 at thirteen, against 1.00 and 0.99 for independent errors. That is a signature with the right sign for a one-sided test, and it is free, since the lags are the ones the comb already names. It is not a wander; it is the comb, read through a different window.

The identity itself is a check. Two routes to one number, agreeing across seven disturbances, is the sort of thing this collection puts weight on — and here it is doing more than reassurance. It says that anybody who computes a variance ratio on a divergence sequence and reports it as new information has computed the autocorrelation, and should say so.

What it costs an experiment, and what it buys #

The proposal’s accounting has to be redone, and it comes out better in one place and worse in another.

Worse: there is no free second statistic. A divergence sequence yields one autocorrelation function, and the comb and the wander are two windows onto it. An experimenter who measures a sequence long enough for a comb has not thereby measured a second thing that discriminates the same hypotheses.

Better: the hole at the contact offsets is a genuinely cheap addition, and it is cheaper than the comb it comes from. A comb is read by scoring a whole set of lags against a sampling band and requires the pair to be known or recovered; the hole is a single ratio at a stated block size, and the block sizes to use are the two parastichy numbers, which a counting of the shoot already supplies. At a coupling of 0.7 it is 0.22 and 0.21 against a null of one.

And better again in a way the proposal did not anticipate: the wander is now an observable for a hypothesis nothing here could previously address. Whether a meristem’s errors drift — whether the placement conditions of organ five hundred resemble those of organ four hundred more than those of organ one — is a question about the plant’s environment and its own development rather than about contacts, and a variance-time curve on a long enough sequence answers it.

How the mistake was made #

Worth setting out, because the shape of it is general.

The wander was measured, correctly, on the disturbance — block means of the deviates, ×5.5 at one coupling and ×55 at another, reported as evidence that the inherited disturbance has a history and the shared one does not. That measurement stands and it was the right one for the question it answered, which was about the structure of the disturbance.

The error was in the sentence that carried it across. “The divergence sequence carries a slow wander” was written as though the disturbance and the sequence were the same series with a scale factor between them. They are not, and the operation between them is exactly the one that annihilates the quantity being carried across.

Nothing about the model changed. What changed is that somebody asked the statistic to be computed on the other side of a difference operator, and it took a few minutes.

What this does not say #

It does not say the wander is not there. It is there, in the disturbance, at the sizes previously measured. What it is not is observable in the sequence a plant hands over.

It does not say the earlier measurement was wrong. The block-mean variances of the streams are recomputed here and agree: ×0.93 for white, ×49.09 for the inherited disturbance at a coupling of 0.7, ×2.55 for the shared one. Only the claim about what those numbers imply for a sequence has gone.

It does not say the comb is unaffected. The comb is a different statistic and this essay leaves it exactly as it was, including the concession that it is evidence of a re-transmitted disturbance rather than of a placement rule.

And it does not say a wander is uninformative. A divergence sequence with a wander in it says something specific and testable — that the disturbance has a memory in time — and this collection had no observable for that before. It is a different question from the one it was proposed for.

A general form of the mistake #

The failure has a name and it is worth attaching, because this collection is going to keep proposing observables.

An observable is a function of what can be measured. The disturbance is not what can be measured; the positions are, and a divergence sequence is a particular function of them. Every claim of the form “if the plant is doing X, the measurement will show Y” therefore has a step in it where a property of the hidden thing is carried across an operator to a property of the visible thing — and that step can annihilate exactly the property being carried.

The operators this collection has between hidden and visible are all of that kind. Positions are differenced to get divergences. Divergences are wrapped modulo a turn. A jugate divergence is reduced modulo a fraction of a turn. Cell areas are taken relative to a background. Each of them destroys some property of what went in, and each of them is applied without comment because it is what a measurement is.

The cheap protection is to compute the proposed statistic on the visible side before writing the sentence. It costs a few minutes and it is the difference between an observable and a sentence about one.

The check that would refuse it #

Four assertions, and the first two are the retraction.

Independent errors have to give one at every block size, within a tenth. That is the calibration, and without it every other number here is a ratio against nothing.

The inherited disturbance has to have many times an independent stream’s variance in its own block means and no low-frequency power in the divergences it produces. Both halves are required together: either on its own is consistent with a mistake in one of the two computations.

The identity has to hold, at every shape and every block size, to within the sampling error of the block-mean variance — which is a tolerance that widens with the block, because a variance over forty-six numbers is a fifth of itself uncertain and a fixed tolerance would either fail on sampling at the wide blocks or pass anything at the narrow ones.

And a memory has to leave a wander several times what independent errors give, with a curve that climbs. Without that half the essay is consistent with the statistic being unable to detect anything at all.