A difference forgets a drift
Worth reading first: The sequence has a memory · A disturbance with a memory.
An earlier essay here ended with a proposal, stated in two sentences and offered as a bargain:
If a plant’s disturbances are inherited, the divergence sequence carries a slow wander as well as a comb: block means over a hundred organs vary several times more than independent errors would allow. If they are merely shared between touching organs, the sequence has teeth at the contacts and no wander.
It was priced as the cheap half. A comb costs about nine hundred organs at half a degree of protractor precision; a variance ratio is not a spectral feature and needs no extra measurements, so a stem long enough for the comb is more than long enough for both. Two statistics off one sequence, and they disagree about the hypothesis.
The proposal is wrong, and the arithmetic that shows it is one line that was never written down.
The line that was never written down
A disturbance acts on positions. Organ i sits at i divergences plus an error eᵢ, and the divergence between consecutive organs is therefore
δ + (eᵢ − eᵢ₋₁)
A divergence is a difference. And differencing a series is precisely the operation that removes its low-frequency power: a drift moves eᵢ and eᵢ₋₁ by nearly the same amount, and the difference between them does not see it.
The wander is real. It is in the disturbance. The divergence sequence — the one thing about a plant a botanist can actually record — is the one place it is not.
The two columns
Block means over a hundred, as a multiple of what independent draws would give:
| disturbance | in the disturbance | in the divergences |
|---|---|---|
| independent | ×0.93 | 0.96 |
| a memory, ρ = 0.9 | ×16.24 | 10.25 |
| inherited at the contacts, a = 0.7 | ×49.09 | 0.83 |
| shared once, no history | ×2.55 | 0.95 |
The disturbance with the largest wander of any shape this collection has built leaves none in the sequence. A disturbance with a memory in time keeps most of its own. The two columns are not two versions of one measurement; they reorder the disturbances almost completely.
That reordering is the whole finding, and it is why the proposal was not merely imprecise. It named the transport as the shape that would show a wander, and the transport is the shape that shows none.
Why the transport in particular
The reason is more specific than “differencing removes drift”, and it is worth having because it says what a wander is evidence of.
Write the statistic properly. For a differenced stationary stream the variance of a block mean of B values is 2σ²(1 − ρ(B))/B², so the quantity that is one for independent errors is
W(B) = B²·Var(block means)/Var(divergences) = (1 − ρ(B))/(1 − ρ(1))
where ρ is the autocorrelation of the disturbance. Measured by two routes that share no arithmetic — one a variance of block means of the angles, the other a lagged product of the deviates — the two agree everywhere to inside the sampling error of the noisier of them, across seven disturbances and eight block sizes.
So W(B) is the autocorrelation function, evaluated at the block size instead of at a lag. It is not a second statistic at all.
And that settles the transport. A contact transport puts its correlation in a residue class — at eight, thirteen, twenty-one, and at their sums and differences, and at nothing else. Its autocorrelation at a general large lag is near zero, so W is near one, and no amount of coupling changes that. Nothing whose correlation lives in a residue class can produce a wander, which is a statement about arithmetic rather than about this particular model.
The number the identity supplies
An identity is worth more when it predicts something with no free parameters, and this one does. For a disturbance that remembers the last error with coefficient ρ, the autocorrelation is ρ^B, so W climbs to 1/(1 − ρ) and stops.
Measured against the closed form at five coefficients: 1.38 against 1.43, 1.90 against 2.00, 3.15 against 3.33, 9.51 against 10.00, and 30.94 against 33.33. That is 0.93 to 0.97 of the closed form at every coefficient — a uniform low bias, which is what a variance of block means does in a finite run, rather than a disagreement about the shape.
What the statistic is, in detail
It is worth writing the recipe out, because a variance ratio has more than one plausible normalisation and the wrong one hides the whole result.
Take the divergence sequence. Cut it into consecutive blocks of B, take the mean of each block, and take the variance of those means. Divide by the variance of the sequence itself, and multiply by B squared.
The square is the part that has to be right. For an ordinary series — one that is not a difference of anything — block means have variance falling as 1/B, and the natural normalisation is a single factor of B. A divergence sequence’s block means fall as 1/B² instead, because the block mean telescopes: the mean of eᵢ − eᵢ₋₁ over a block is the last error minus the first, divided by B — a difference of two values rather than a sum of B of them.
Normalised with a single B, every disturbance here produces a tidy curve falling as 1/B and nothing is distinguishable from anything. Normalised with B², independent errors give a flat line at one and everything else is a departure from it. The two normalisations differ by a factor that is the same for every series, so neither is wrong; only one of them puts the interesting structure where an eye can see it.
What the two observables actually separate
The proposal had the right instinct and the wrong pairing. There are two observables, they are almost independent, and they separate two different kinds of inheritance:
- A comb says the disturbance is correlated at the contact offsets and at combinations of them. It is produced by an error passed between touching organs and passed on again, and — as this collection has already had to concede — it is produced by nothing else that has been tried.
- A wander says the disturbance is correlated at every lag, with a tail that decays slowly. It is produced by a memory in time: an apex whose conditions drift, a measurement whose zero moves, a shoot growing through a changing season.
A sequence with a comb and no wander is a shoot whose errors are shared between neighbours and not carried through time. A sequence with a wander and no comb is a shoot in a drifting environment with independent local errors. A sequence with both is a shoot with both, and there is no reason it should not have.
What is left of the wander, and it is not nothing
Two things survive, and both are sharper than what they replace.
A hole at the comb’s own lags. An inherited disturbance drives W down where the block size is one of the offsets it couples at: 0.22 at a block of eight and 0.21 at thirteen, against 1.00 and 0.99 for independent errors. That is a signature with the right sign for a one-sided test, and it is free, since the lags are the ones the comb already names. It is not a wander; it is the comb, read through a different window.
The identity itself is a check. Two routes to one number, agreeing across seven disturbances, is the sort of thing this collection puts weight on — and here it is doing more than reassurance. It says that anybody who computes a variance ratio on a divergence sequence and reports it as new information has computed the autocorrelation, and should say so.
What it costs an experiment, and what it buys
The proposal’s accounting has to be redone, and it comes out better in one place and worse in another.
Worse: there is no free second statistic. A divergence sequence yields one autocorrelation function, and the comb and the wander are two windows onto it. An experimenter who measures a sequence long enough for a comb has not thereby measured a second thing that discriminates the same hypotheses.
Better: the hole at the contact offsets is a genuinely cheap addition, and it is cheaper than the comb it comes from. A comb is read by scoring a whole set of lags against a sampling band and requires the pair to be known or recovered; the hole is a single ratio at a stated block size, and the block sizes to use are the two parastichy numbers, which a counting of the shoot already supplies. At a coupling of 0.7 it is 0.22 and 0.21 against a null of one.
And better again in a way the proposal did not anticipate: the wander is now an observable for a hypothesis nothing here could previously address. Whether a meristem’s errors drift — whether the placement conditions of organ five hundred resemble those of organ four hundred more than those of organ one — is a question about the plant’s environment and its own development rather than about contacts, and a variance-time curve on a long enough sequence answers it.
How the mistake was made
Worth setting out, because the shape of it is general.
The wander was measured, correctly, on the disturbance — block means of the deviates, ×5.5 at one coupling and ×55 at another, reported as evidence that the inherited disturbance has a history and the shared one does not. That measurement stands and it was the right one for the question it answered, which was about the structure of the disturbance.
The error was in the sentence that carried it across. “The divergence sequence carries a slow wander” was written as though the disturbance and the sequence were the same series with a scale factor between them. They are not, and the operation between them is exactly the one that annihilates the quantity being carried across.
Nothing about the model changed. What changed is that somebody asked the statistic to be computed on the other side of a difference operator, and it took a few minutes.
What this does not say
It does not say the wander is not there. It is there, in the disturbance, at the sizes previously measured. What it is not is observable in the sequence a plant hands over.
It does not say the earlier measurement was wrong. The block-mean variances of the streams are recomputed here and agree: ×0.93 for white, ×49.09 for the inherited disturbance at a coupling of 0.7, ×2.55 for the shared one. Only the claim about what those numbers imply for a sequence has gone.
It does not say the comb is unaffected. The comb is a different statistic and this essay leaves it exactly as it was, including the concession that it is evidence of a re-transmitted disturbance rather than of a placement rule.
And it does not say a wander is uninformative. A divergence sequence with a wander in it says something specific and testable — that the disturbance has a memory in time — and this collection had no observable for that before. It is a different question from the one it was proposed for.
A general form of the mistake
The failure has a name and it is worth attaching, because this collection is going to keep proposing observables.
An observable is a function of what can be measured. The disturbance is not what can be measured; the positions are, and a divergence sequence is a particular function of them. Every claim of the form “if the plant is doing X, the measurement will show Y” therefore has a step in it where a property of the hidden thing is carried across an operator to a property of the visible thing — and that step can annihilate exactly the property being carried.
The operators this collection has between hidden and visible are all of that kind. Positions are differenced to get divergences. Divergences are wrapped modulo a turn. A jugate divergence is reduced modulo a fraction of a turn. Cell areas are taken relative to a background. Each of them destroys some property of what went in, and each of them is applied without comment because it is what a measurement is.
The cheap protection is to compute the proposed statistic on the visible side before writing the sentence. It costs a few minutes and it is the difference between an observable and a sentence about one.
The check that would refuse it
Four assertions, and the first two are the retraction.
Independent errors have to give one at every block size, within a tenth. That is the calibration, and without it every other number here is a ratio against nothing.
The inherited disturbance has to have many times an independent stream’s variance in its own block means and no low-frequency power in the divergences it produces. Both halves are required together: either on its own is consistent with a mistake in one of the two computations.
The identity has to hold, at every shape and every block size, to within the sampling error of the block-mean variance — which is a tolerance that widens with the block, because a variance over forty-six numbers is a fifth of itself uncertain and a fixed tolerance would either fail on sampling at the wide blocks or pass anything at the narrow ones.
And a memory has to leave a wander several times what independent errors give, with a curve that climbs. Without that half the essay is consistent with the statistic being unable to detect anything at all.
Shares its objects with
Essays that name at least two of the same things, and that neither author linked.
- Matching instead of correcting — both name artefact, claim testing, ensemble, evidence, honest limits, identifiability, measurement, negative result, null model, summary statistic
- What the rule does to a drift — both name autocorrelation, ensemble, evidence, honest limits, measurement, negative result, noise, null model, summary statistic, transport
- The ratio was never about the rule — both name autocorrelation, ensemble, evidence, honest limits, identifiability, measurement, noise, null model, transport
- A periodicity is not a lattice — both name artefact, autocorrelation, ensemble, evidence, identifiability, measurement, noise, null model
- What a forgery has to know — both name autocorrelation, evidence, honest limits, identifiability, measurement, noise, null model, transport
- A period the grid invented — both name artefact, claim testing, honest limits, measurement, negative result, refusal, summary statistic
Named objects
A flat tag is an object no other essay names yet.
ArtefactAutocorrelationClaim testingEnsembleEvidenceHonest limitsIdentifiabilityMeasurementNegative resultNoiseNull modelRefusalSummary statisticTransportUntested claim