Four ways to count a neighbourhood

The rule places each organ where a sum is smallest — one term per organ already there, each falling off as a power of the distance. The exponent of that power is the only thing in the rule that says how far it looks, and it has been used as a knob: sweep it, and the neighbourhood the rule attends to changes from almost two hundred organs to almost none.

That sweep produced a refutation. A disturbance with a memory was supposed to pass through a rule whose neighbourhood was shallower than the memory and be corrected by one that was deeper, with the crossover tracking the depth. There was no crossover at any depth, and the deep rule passed more rather than less.

A negative result that leans on a measured quantity deserves a second measurement of that quantity, which is what this essay is. How deep the rule looks was reported one way; there are three others, and they do not agree — though what they agree about instead turns out to be the more useful answer.

What the profile is #

At the moment an organ is placed, the rule evaluates a sum over the organs already there. Each term is one over the distance to the power of the exponent, and the distance is measured across the surface of the stem rather than up it.

A “depth” is a summary of that sum: some statement of how many organs are doing the work. The trouble is that the sum has no edge. Every organ ever placed contributes something, so any count of organs requires a decision about how much of the total is enough — and the decision is a free parameter that nobody has ever varied.

Four summaries #

The first three fix a share and count organs. Rank the terms by size, add them up until the running total reaches the share, and report how many terms that took. The share used before was nine tenths. A half and ninety-nine hundredths are equally defensible.

The fourth uses no share at all. The weighted mean lag is the average distance back, in organs, with each organ weighted by the size of its own term. It is the centre of mass of the profile and it needs no threshold.

Why “how many organs” is a question with no answer in the rule #

The rule contains no number of organs. It contains an exponent, and a loop bound that has been shown to change nothing.

That is worth separating, because the two are constantly confused. The loop bound is how far back the implementation actually sums; it is a parameter of the program. The neighbourhood is how far back the terms matter; it is a property of the exponent. Sweeping the first from one local spacing to six moves the divergence’s wander by a factor of 1.30, against a factor of 3.28 between random seeds at one setting — which is the shape a parameter has when it is not in the answer.

So the only knob that reaches the neighbourhood is the exponent — and what a band of exponents can be made to say is its own caution — while the neighbourhood is a summary of a sum with no edge. Any number of organs attached to it is a summary somebody chose, and the four here are four such choices.

They disagree about the size by a factor of ten #

Read down the exponent-3 row: the rule’s neighbourhood is 3 organs, 30 organs, 149 organs or 21 organs, depending on which summary is asked. That is a range of fifty to one at a single setting of the rule, and every one of the four is a defensible answer to “how many organs is it looking at”.

They also disagree about how much the neighbourhood changes across the sweep, which matters more for what the depth was being used for. Across exponents 1.5 to 6, the nine-tenths count moves by a factor of 60.7; the half count by 33; the ninety-nine count by 31.6; and the weighted mean by 6.1.

So a claim of the form “the rule’s neighbourhood spans a factor of sixty across this sweep” was carrying a chosen constant that nobody had varied, and varying it moves the factor to six.

Where each definition puts its emphasis #

The four disagree in a structured way rather than randomly, and reading the structure is most of what makes the table useful.

The half count is a statement about the very top of the profile: at an exponent of 6 it is one organ, which says that a single term is more than half of everything. The ninety-nine hundredths count is a statement about the tail: at an exponent of 1.5 it is 284 organs, which is nearly the whole window the profile was computed over, and says that the tail there is heavy enough to matter out to the edge of what was measured.

The nine tenths count sits between them and inherits both sensitivities, which is why it moves furthest across the sweep: at a steep exponent it is reporting the top and at a shallow one it is reporting the tail, so it is measuring different things at the two ends.

The weighted mean is the least sensitive because it never sorts. Every organ contributes its own lag weighted by its own term, so a heavy top pulls it down and a heavy tail pulls it up, and at these profiles the two partly cancel.

And they agree about the order, exactly #

Every one of the four falls as the exponent rises. Not approximately, not with a crossing somewhere in the middle: at every adjacent pair of exponents, each of the four definitions puts the same rule deeper than the other.

That is the finding, and it is a negative one. No redefinition of the depth can change the sign of a result about the depth. The earlier sweep found the deep rule passing more drift than the shallow one and no crossover between them; if “deep” and “shallow” mean the same ordering under every available definition, then that result cannot be an artefact of how the neighbourhood was counted.

That closes off one repair and leaves the result standing. It does not explain it, and it makes the situation stranger rather than less strange: a quantity that varies by a factor of sixty by one measure and six by another is producing an outcome whose ordering does not depend on which.

The weighted mean is the one worth keeping #

Of the four, the weighted mean is the one this collection should report from here on, and there are two reasons that are not “it gives a different answer”.

The first is that it has no chosen constant. The other three all require somebody to say how much of the profile counts as the neighbourhood, and there is no principled answer — nine tenths and ninety-nine hundredths are equally defensible and give 30 organs and 149 at the same exponent. The weighted mean asks a different question, “where is the centre of the profile”, and the answer needs no threshold.

The second is that a count of organs is a count of the largest terms, wherever they are, and it therefore mixes two different facts: how heavy the top of the profile is, and how quickly the tail dies away. The weighted mean mixes them too, but it weights each organ by its own contribution rather than sorting and counting, so a handful of very large terms at short lags cannot be counted as “three organs” while a hundred small ones are ignored.

None of that changes the ordering, which is the finding. It changes what a number attached to the neighbourhood is worth, and the number the earlier work carried was the one that moves furthest.

Why a second definition was worth the trouble #

Because the alternative was to leave a number in a negative result that nobody had tested.

The nine-tenths share was chosen when the depth was first computed, and the note attached to it said the claim did not turn on it — that at eight tenths the exponents separate by thirty-eight and at ninety-five hundredths by fifty. That is a check on the robustness of the number and it is not the same as a check on the choice of statistic. Both eight tenths and ninety-five hundredths are the same summary with the dial moved. The weighted mean is a different summary, and it is the one that disagrees.

The trouble was also cheap. The profile is computed on an undisturbed stem, once per exponent, and every summary of it comes out of the same array of terms. All four definitions are four lines of arithmetic over one measurement, which is the usual state of affairs when a chosen constant has gone unexamined for a while.

And there is one more definition worth mentioning and not using: the deepest organ in the top share, rather than how many organs are in it. Those are different numbers, because the top share is assembled by size and not by lag — the thirty organs carrying nine tenths of the profile at an exponent of 3 include lags well past thirty. That distinction is recorded in the machinery and it is not a fifth definition; it is a reminder that “thirty organs” already means “thirty terms” and never meant “the last thirty”.

What the earlier sweep was entitled to say #

With four definitions in hand it is possible to be precise about which of the earlier statements survive and which were carrying the chosen constant.

“The falloff is the neighbourhood, and the loop bound is not.” Survives, and is strengthened. Every definition moves with the exponent and none moves with the loop bound.

“The neighbourhood spans a factor of sixty-one across the sweep.” Does not survive as stated. It spans sixty-one by one summary, thirty-three by another, thirty-two by a third and six by the fourth, and the figure quoted was the largest of the four.

“The deep rule passes more drift than the shallow one.” Survives, because “deep” and “shallow” are the same ordering under all four.

“There is no crossover at any depth.” Survives this essay and does not survive the next two, for a reason that has nothing to do with the definition of depth.

That is a useful sorting to have. Two of the four claims are strengthened by having a second measurement, one is qualified, and the one that eventually falls falls for an unrelated reason — which is roughly the distribution a careful re-measurement of an old quantity ought to produce.

The thing the table is quietly saying #

Look at the exponent-6 row. Half the profile is carried by one organ; nine tenths by three; ninety-nine hundredths by nine. Now look at exponent 1.5: half by thirty-three, nine tenths by a hundred and eighty-two.

The shape of the profile is changing enormously and the ordering is not. Which suggests that whatever is at the top of the profile might be the same at both — and if it is, then counting the neighbourhood in organs was never going to distinguish the two rules in the way the sweep needed.

That is checkable in one line, and the answer is the subject of the next essay.