A count weighted by direction closes a third of the gap

A drift along the spirals changes fewer counts had a clean result. Lay an error down with the head, each organ inheriting part of its two nearest earlier neighbours’ errors, and the organs move together in round patches. A count reads an organ against its neighbours, so a patch of a given size changes fewer readings than an independent displacement of the same size. And the size that mattered was the relative one: how far an organ moves against its four nearest neighbours. Read through that number, independent errors and patches fell on one curve, and a survey could reason with it.

A streak is a shear the count can see then broke it. Let each organ inherit from its nearest earlier neighbour alone and the error travels along lines through the head. By the same relative measure a streak moves an organ just as far as a patch does, and yet it changes more readings, on large heads as many as an independent displacement twice its relative size. So the relative size is not the quantity after all, or not all of it. The essay’s account was that the number averages over directions and a count doesn’t. A count reads particular directions, its two families, and a streak moves organs against them unequally.

That account leads directly to a test. Measure the movement the way the count reads it, along each of the two families separately, and take the larger: does that one number place all three errors on one curve? And since nobody can measure a head’s errors directly, is there a version of that number a survey could read from a photograph?

Three errors at once #

The three errors look quite different when each organ’s movement is drawn. Independent error is a uniform fuzz of lines in every direction. A patch shows large coherent regions, all going one way. A streak shows the same movement in narrow lines radiating through the head, with neighbours along one line going together and neighbours on the next going somewhere else. All three have the same total size per organ. The dial gives a fair idea of why a single statistic averaged over a neighbourhood could fail to separate them.

Read on three head sizes, 150, 300 and 500 organs, and two displacements, a tenth and a quarter of a spacing, and each correlated error at two strengths of inheritance, that makes thirty combinations. Each has already been counted by the essays above, so the share of readings each error changes is known. Each predictor is read on the independent curve at the same head size: convert the number to the independent error that would have the same value, and read off how many readings that independent error would change. A perfect predictor puts every point on the diagonal.

The number that failed #

The starting point is the relative size, the number that worked for patches. Independent errors lie on the line by construction. Patches lie close to it. Streaks sit consistently above, and the gap is largest for the strong inheritance of 0.9 at a quarter of a spacing: on 150 organs it predicts 7.1 per cent against a measured 8.9, on 300 it predicts 4.1 against 6.1, and on 500 it predicts 3.0 against 4.8. On the largest head that is a prediction about two thirds of the truth. Over all the correlated points the root-mean-square miss is 0.0079.

That number is small in absolute terms, and it matters more than it looks, as what a count is worth has shown about small shares before. A miss of two points on a share of six is a third of the effect, and the size of the effect is what a survey uses to decide how many specimens it can trust.

Along the families #

The candidate replaces all directions with two. On each head, the band’s own pair names its two families. For each organ in the band, take the difference between its error and the error of the organ one step along each family, separately, as root-mean-square per coordinate in spacings. On a head with no direction in its error, those are equal. The predictor is the larger.

It helps. The streaks move towards the line, the patches do not move away from it, and the root-mean-square miss falls from 0.0079 to 0.0054, about a third. On the strongest streak the predictions become 7.9, 4.6 and 3.6 against 8.9, 6.1 and 4.8. Those are real improvements. They close 44, 25 and 33 per cent of the gap on the three heads. They do not close it.

That is the essay’s first finding, and it is not the one the account predicted. Direction is part of what the average misses, and most of a strong streak’s excess is still there after direction has been taken into account. A rule that a survey can apply to all three errors now leaves about a fifth of what a strong streak does on the table.

The two families are not lopsided where it matters #

The reason direction closes so little turns up when the two numbers are read apart. On 150 organs, a strong streak moves an organ 0.160 of a spacing against its next along the smaller family, and 0.241 along the larger, a clear lopsidedness of the kind the account requires. A patch at the same settings reads 0.198 and 0.201. So far, so expected.

On 300 organs the streak reads 0.186 and 0.196. On 500 organs it reads 0.192 and 0.189, with the patch at 0.210 and 0.196. So on the larger heads, where the gap between prediction and measurement is relatively widest, a streak is hardly lopsided between the two families at all. Whatever makes a streak more damaging than a patch there, it isn’t that it moves organs further along one family of the count’s than along the other.

Part of the answer can be read from what a parent is. An organ’s nearest earlier neighbour on a head of a few hundred organs lies on one of the two families of whichever band that organ belongs in, and a head’s band spans a transition at these sizes, so parents switch from one family to another across the band. A streak’s direction is not fixed through the band, and the two families take turns being the favoured one. Averaged over the band, each gets about half. A statistic taken organ by organ through the band sees no preference, even though at any one place there is one.

That suggests where the missing part might be. A count’s reading of a band depends on the median hop of each family, and what matters to a median is not the mean size of the errors but how they are arranged over the band. If half the band’s organs are moved strongly along one family, and half along the other, each median sees a mixture that is not at all the same as a uniform noise of the same size. No root-mean-square, in any direction, will see that. The statistic that would is the local lopsidedness, sampled within windows, and it has not been tried here.

A number a survey could read #

The second question is whether any such number can be got from a specimen. Knowing how far an organ has moved requires knowing where it belonged, which no photograph supplies. But something closely related can be read. Trace a family’s spirals, and for each organ measure how far it lies from the line through its two neighbours along that spiral. On an exact head the only contribution to that is the head’s curvature, which is small and can be computed from the radius. On a disturbed head the rest is the error, in the direction across that family. It is, in effect, how straight each family’s spirals run, and it needs only a chain of organs and a ruler.

Taken as the larger of the two families, with the curvature removed in quadrature, this wobble predicts as well as the true relative movement along the families does. The root-mean-square miss is 0.0055, against 0.0054. On the strongest streak the predictions are 8.5, 4.4 and 3.5 against 8.9, 6.1 and 4.8. Across all the thirty points it does nothing worse than the number it stands in for.

So the second half of the question has the better answer. A survey could measure this. A head’s disorder, in the direction that matters to its count, can be read off the head itself by anyone able to trace spirals, without any model of how the head should have been. The price of that is having to find the chains first, on a head where they may already be hard to see, and a reader doing that by eye would have errors of their own to account for.

It also goes some way towards the correction earlier essays have asked for: the census’s specimen counts assume a null, and the null depends on how disordered a real specimen is. A measured wobble can stand in for a displacement that nobody can measure.

How much of a streak each number sees #

Averaged over the two displacements for the strong streak, the relative size accounts for 82, 75 and 70 per cent of the changed readings on 150, 300 and 500 organs. The larger family accounts for 88, 81 and 81. The wobble accounts for 94, 78 and 79. So the best anyone can do with these numbers, on the heads where it matters most, is about four fifths.

The trend with size is itself a clue. The relative size gets worse as the head grows, while the family measure holds steady, and this is what would be expected if the part of the error relevant to the count became a smaller share of the total as the bands widened and the pair changed within them. A larger head has more organs per family and a wider range of parents, so its band mixes directions more, and its count is more sensitive to the mixture.

It should be said what the remaining fifth amounts to in a census. On a head of 300 organs at a quarter of a spacing, a strong streak changes 6.1 per cent of readings and the prediction says 4.6. A census’s specimen requirement moves by little for a change of that size, as a streak’s own essay reported when it found three specimens were still enough. So nothing here threatens that. The problem is one of understanding: there is a kind of error that no currently measured number sees all of.

The weaker streak, and the patches #

The strong streak is where the gap is widest, and the other points add two things. A streak with an inheritance of 0.6 is much better behaved. At a quarter of a spacing the larger family predicts 9.4, 5.4 and 4.1 per cent on the three heads, against a measured 9.2, 6.0 and 3.9, so it overshoots on two of them and is close on all three. The weaker streak is nearly solved, and this is the case in which most of an organ’s error is its own rather than inherited.

The patches turn out to have a smaller version of the same gap, which an earlier reading of them had passed over because the relative size came so close. With an inheritance of 0.9 at a quarter of a spacing, a patch on 300 organs changes 4.9 per cent of readings and the relative size predicts 4.1; on 150 organs, 7.6 against 6.9. In other words the shortfall grows with inheritance, for both kinds of error, and a streak merely has more of it. That is a further reason to doubt that direction is the explanation. A patch has no direction, its two families read 0.201 and 0.200 on 300 organs, and it falls short anyway.

What grows with inheritance is coherence. The stronger the inheritance, the more each organ’s error resembles the errors of organs far from it, and the fewer independent errors a band’s worth of hops really contains. A bad year does not average out met the same thing in a shell’s section, in a different form, and for a similar reason: a median of hops that share their errors is a median of fewer hops than it looks.

What this says about the relative-size rule #

The rule that a count reads relative movement has been a satisfying one, and it came through a good many tests: on independent errors, on patches and on the rings of a head. It does not go away here. It is still what separates a patch from an independent error, and it still predicts most of a streak. What it is not is a complete account, and the failure here isn’t a matter of having averaged over directions, or not mainly. A count of parastichies reads a median of many hops, and two errors with the same local statistics can have very different medians if the local statistics are arranged differently across the band.

Put at its simplest: a patch moves neighbours together, and a streak does too in one direction. Both can be fully described organ by organ. The count takes a median across a whole band, so it sees whether the two families’ errors are coherent across the band, and that is a property of the whole band. This resembles what a wider window does for an independent error: there, more of the band gets the right answer because more hops are independent. A streak’s hops are not independent, so a window doesn’t average them out in the same way, and that may be the larger part of what remains.

What this does not establish #

The errors are all stationary, the same size everywhere on the head, and each has one strength of inheritance throughout. Thirty points have enough scatter that a difference of one miss for another, 0.0054 and 0.0055, is no difference at all. The wobble uses chains known from the head’s own pair, which a survey would have to find for itself; its value on a chain traced by hand has not been measured. Nothing here tests whether direction would do better on a stem than on a disc, where a band has one pair throughout and parents would not switch families. That is a possible next step.

What would overturn it #

A local statistic of the kind suggested above that closed the gap entirely would make the mixture account right and this one incomplete. A measurement in which the two families were strongly lopsided within a head’s band and that lopsidedness did not predict the excess would make it wrong. A wobble that failed on a chain traced from the moved organs alone, with no reference to the true families, would undo the claim that a survey could read it.

Still open: a streak on a stem #

On a head a band’s parents change family because the band straddles a transition, which is why direction averaged away. A stem at a fixed rise has one pair at every height, so a streak there has a fixed direction, the same family all the way up. The next measurement lays the same streak on a cylinder at a rise inside one pair, reads how many readings it changes against a patch and an independent error of the same size, and asks whether the larger family’s movement now closes the whole gap. If it does, the shortfall on a head belongs to the transition and not to the streak. If it does not, then something about a streak other than its direction is reaching the count, and a statistic that reads the arrangement of errors along the band is what has to be built next.