A wider window buys back a shaken stem

The window nobody varied moved a number that every count on the settling table had taken for granted. Counted over twenty organs or eight hundred, all three hundred and eighty-four settled runs return the same pair. A plateau the instrument should have had found two bounds outside which the window does decide things, and how many organs a pair needs turned one of them into arithmetic: a counter given too few organs cannot measure the offset it would need to find the right pair, and quietly returns the one below. The standing two hundred was eight to ten times what was needed.

Every one of those stems is exact, in the sense that nothing has jostled its organs except the rule itself. On that evidence a window is a thing to make wide enough and then forget, since above the floor it has no effect. That account never had to deal with the case a window exists for. In any sampling instrument, the usual reason to take more samples is noise, and a count taken from a photograph is full of it. The positions are traced by hand. The specimen is bent, cut or squashed. Real organs are not where a rule would put them.

So the window has a second job that has never been tested. The counter asks, for each offset, for the median length of a hop of that many organs, and takes the two shortest. A median of more hops is a steadier number. A wider window ought to read a shaken stem better than a narrow one. Whether it can, how much that costs, and whether there is any error it cannot buy back are all measurable, on a lattice whose pair is known in advance.

A stem with its organs moved #

Take an ideal cylinder lattice at a stated rise. Its pair is known exactly, from the lattice’s own two shortest vectors, and the counter on an exact lattice reads it from about twenty organs. Now move every organ by an independent gaussian in each direction, its size stated as a share of the lattice’s shortest vector. Sixty draws of those errors are read at each of fourteen windows from twenty organs to sixteen hundred, with the counter’s ceiling on offsets set by the window so that it cannot be asked for more than the band holds. Four rises give pairs from 3/5 to 13/21.

The dial makes it clear how unforgiving this is meant to be. At a fifth of a spacing the lattice is visibly disturbed but anyone could pick out its rows. At half a spacing the organs have wandered far enough that the rows are hard to see at all, and what meets the eye is a loose scatter. At a whole spacing, each organ has typically moved further than the distance to its neighbour. There is no trace of a lattice left for the eye, and a person would call it random. It is a fair question whether the word pair still means anything.

A window at work #

It does. On the 8/13 lattice with each organ moved by three tenths of a spacing, the window that the exact lattice needs is not enough, reading about four draws in five in the narrower range, and by sixty organs it holds the pair nine draws in ten. At half a spacing the same share needs two hundred organs. At eight tenths it needs six hundred. At a whole spacing, twelve hundred organs read the pair on nineteen draws in twenty.

That last result is worth a pause. A person looking at that scatter could not pick out a single row, and yet the median hop length of each offset, over twelve hundred organs, is enough to put the right two offsets first. The noise in each hop is large, and it is the same size for every offset. The right offset’s hop is still shorter on average than its rivals by a small margin, and a median of a thousand values will find a small margin. Nothing about the lattice has been recovered that was hidden. It was simply always there in the positions, at a lower signal than the eye can use.

The price of an error #

Set the same threshold on all four lattices, and the narrowest window that holds it can be read as a price. Up to about a fifth of a spacing there is no price at all: the window an exact lattice needs is still enough, twenty or thirty organs on all four. Past that it climbs. On the 5/8 lattice it is 100 organs at half a spacing, 600 at four fifths, 1,200 at a whole spacing, and 1,600 at one and a quarter. On 8/13 it is 200, 600, 1,200 and 1,600.

The climb is steep, at something close to the two-and-a-half power of the error between half a spacing and one: doubling the error multiplies the window by about six. A person planning a survey should read that as quite a restrictive curve. The window on a head is bounded, since a band of a head that keeps to one pair holds only a limited number of organs. The whole of the width to which a pair is available on a real stem is set by the rise’s own rate of fall, with a third of the step of rise between transitions as the upper edge.

Set against that, the small end is what stands out. A traced point with an error of a tenth of a spacing is a poor tracing and a large error for most of the fine readings here. A reading of the angle from a head’s largest holes has largely collapsed by then. Here the cost of that error is nothing at all. The count carries no price until about a fifth of a spacing, and it is the only reading of a head’s lattice of which that has been shown.

Four pairs and one threshold #

The point at which the price begins is close to a constant, and that is worth saying separately, because it is the number a survey would plan around. On 3/5 and 5/8 the narrowest window first moves at a fifth of a spacing, going from twenty organs to thirty. On 8/13 it moves at a fifth as well, from twenty to sixty. On 13/21, whose exact floor is already thirty organs, it stays there up to a quarter of a spacing and first moves at three tenths. The onset is very nearly independent of the pair, and of the rise, across a range of rises that differ by a factor of about seventeen.

That is what an error stated in spacings should give, if a count reads a lattice only by comparing its vectors with each other. Scale the lattice and scale the error, and nothing about the comparison changes. It also separates this from another way of treating noise, which is to give an error in absolute units: a fixed tracing error of a tenth of a millimetre is a far larger share of a spacing on a fine lattice than on a coarse one, and would start costing a survey at very different pairs on one head.

The ceiling on offsets does not account for the improvement. Above 250 organs it is held at two hundred, well past any offset that matters on these lattices, and the shares go on rising with the window from there on: on 8/13 at a whole spacing, from 73 per cent at three hundred organs to 95 per cent at twelve hundred. That rise is the window’s own effect, with nothing else moving.

Why the price should be steep #

The shape of that curve has a reason. Two offsets’ hop lengths differ by a share of a spacing fixed by the lattice, say $\delta$. A gaussian error of $\sigma$ spacings in each direction adds, in quadrature, to every hop. So once $\sigma$ is comparable with the lattice’s own spacing, each hop’s length is mostly noise, and the gap between two offsets’ true hop lengths shrinks in the measured distribution as about $\delta/\sigma$, while that distribution’s width grows as $\sigma$. A median separates them only when the standard error falls below that gap, so the number of hops needed rises like a power of $\sigma$ above two. A count of the same kind has been priced against a closing error before, and for the same reason it costs a lot per unit of error once it costs anything.

That argument gives a form and not a number, and nothing here has been fitted to it. It also suggests what limits the climb. Hop lengths from one window are not independent, since each organ takes part in many hops. A window cannot give more than about as many independent hops as it has organs, for any one offset.

Where it stops #

By one and a half spacings no window on the list holds the pair on any of the four lattices. On 8/13 the widest window reads it on about two draws in three. The curve hasn’t flattened out either, so a window of ten thousand organs might go further, and there is a purely practical reason nothing here has gone that far: no real stem holds one pair over ten thousand organs. So the edge is a matter of how far it is worth going rather than a statement that the signal has run out.

What is a clear finding is the direction it fails in. A window too narrow for an exact lattice returns the pair below, which is why how many organs a pair needs puts a floor there. A window that cannot hold a very noisy lattice errs the opposite way. The wrong answer from 8/13 at a spacing and a half is 13/21 on fifteen of those draws. From 5/8 it is 8/13, from 3/5 it is 5/8, from 13/21 it is 21/34. It is not the pair below, which has a longer shortest vector, and it is not a pair on no sequence, except in a very few draws at the largest error.

The reason follows from the same picture. A pair is the two shortest offsets. Random error adds length, adding proportionally more to a shorter hop and less to a longer one, which flattens the differences between them. An offset of 21 on a lattice reading 8/13 sits just past the third-shortest vector, so when the differences flatten it is first to overtake. So noise pushes a count one step up the sequence, in the same direction as a falling rise does, and a reader of a noisy head who has found a pair one step above what the radius predicts has one more reason to check the rise before calling it a measurement. Reading the pair from the angles found a disagreement of a similar shape from a very different cause.

A window too narrow to be misled #

One feature of the curves has to be dealt with, because it looks like a contradiction and it is a warning. On the 8/13 lattice at half a spacing, the narrowest window on the list reads the pair 93 per cent of the time, while thirty organs read it 52 per cent of the time, and it takes two hundred before the share is back above 90. On 13/21, thirty and forty organs read it on every draw, and sixty on 82 per cent. In each case a narrow window seems to beat a wider one.

What is happening is an artefact of a ceiling. The window sets how large an offset the counter can ask about, since a band of twenty organs has no hops of length twenty-one at all. On a lattice that reads 8/13, a window of twenty cannot test offset 21, the rival that noise would promote. So it finds 13 by having no way to measure what might have beaten it. That is exactly the offset ceiling that an earlier essay found silently deciding counts on exact stems, now producing a right answer for a wrong reason.

The share falls as soon as the window is wide enough to admit the rival, and only recovers when it is wide enough to measure the rival well. So the floor that means something is the one read from above: the narrowest window from which every wider window also holds the pair, which is how the figure above was built. A reader testing a count at several windows on a shaken specimen, and finding that the narrow windows agree with the expected pair, has not confirmed anything. They would need the wider windows to agree too.

What a survey should take from this #

The earlier window essays recommended a floor and declared the rest irrelevant, and that was right about their stems. The recommendation changes in one respect for anything measured off a real specimen. Width is the one setting that buys back measurement error, and at errors under a tenth of a spacing it is not needed. Beyond about a fifth of a spacing it should be as wide as the band of rise allows, and a stated band should go with any count of a head that has been traced by hand. A count at a window below the floor for its error is not merely noisy. It has a preference, and the preference is for the pair above.

There is a useful turnaround available as well. Since the floor depends strongly on the error, a person who reads a specimen at several windows can estimate how large its positional error is, by finding where the reading stabilises. That would be a single number for the specimen’s disorder, derived from its count, in units of its own spacing. This has not been tried on anything but these lattices. It is the kind of use that a measured price curve makes possible.

What this does not establish #

The errors here are independent, and the essays on drift along the spirals showed that correlated errors change fewer counts per unit of size, so these floors are presumably pessimistic for a real grown head. Nothing has been read on a stem whose rise changes, where a wide window eventually spans two pairs. So the true limit on a real stem is whichever of the two runs out first, and how they trade has not been measured. A median is one choice of statistic, and a different one might do better or worse under this error, particularly at the largest errors. Sixty draws give each share about four points of uncertainty, so the floors are good to about one step of the list, and no single pair of neighbouring values should be read closely.

How it could be wrong #

If a larger window, at a fifth of a spacing or more, gave no steadier a reading than a narrow one with the counter’s offset ceiling held fixed, the whole claim that width buys back error would go. If the wrong answer at very large error were the pair below or a pair on no sequence, rather than the next pair, the account of why it fails would. If the error at which the price begins turned out to depend strongly on the pair rather than being about a fifth of a spacing across four of them, that one result would be a property of these four lattices.

Still open: the two limits together #

A real stem gives a window two ways to fail, and this essay and the ones it follows have measured each with the other held away. The next measurement reads a stem whose rise falls at a stated rate, with its organs moved by a stated error, so that a window wide enough to cope with the noise may also be too wide for the rise. It asks for which errors and rates a window satisfying both still exists, how far along the rise a self-sized band could be stretched to pay for noise, and whether there is a measurable error, for each rate of fall, past which no band on the stem can read its pair. That edge would be the number a survey of real stems would actually need.