GENESIS[Phase 2 · 12. The Alarm That Caught Itself Lying] Digital Civilization

We built an alarm so idle machinery could never hide again. Its first version wouldn't have caught the failure it was built for. Its second announced the…

We built an alarm so that idle machinery could never hide again. Its first version would not have caught the failure it was built for. Its second announced the wrong number of slots. Then it worked, and told us half the economy was doing nothing.

The civilization runs unattended on a loop. Every thirty seconds it checks the clock and does the cheap work — paying whatever is due, closing whatever has expired, publishing what the public can see. Every fifteen minutes it does the expensive work, because the expensive work needs the model and there is one graphics card.

The expensive work rotates. One turn asks agents whether they want to stand for office. The next asks whether anyone wants to found a business. The next puts a candidacy to a panel. The next puts a venture to a bidder. Round and round, one kind of work per turn, so that no single activity monopolises the hardware.

The rotation is the good idea. What it also does, which took us a while to appreciate, is give each kind of work its own way to die quietly.

What happened the first night

At eleven minutes past three in the morning, the last open auction expired unfunded. From that moment the bidding turn had nothing to bid on, so it did what it was written to do: checked, found nothing, and returned a perfectly correct result meaning nothing to do here.

It did not log this. The branch that produced that result wrote no line at all. Neither did the participation branch when it had nothing to say.

So for three and a half hours, half of every expensive turn accomplished nothing, and the log looked precisely like a healthy run — steady, regular, uneventful. We found out the next morning by counting rows in the database and noticing the numbers had stopped moving some hours earlier.

Nothing was broken. Every line of code did exactly what it was supposed to do. The run simply spent half its night burning a graphics card on questions with no answers, and nothing in the system was in a position to mention it.

The first version, which would not have worked

The fix seemed obvious. Count turns that write nothing to the database; if the count reaches three in a row, complain loudly.

We wrote that, tested it, and it passed. Then we ran the loop to watch it rather than trusting the test, and found it would never have fired.

The counter was global. Participation was still working — there were agents left to ask, and asking them wrote rows. So every third turn the counter reset to zero, and the two dead slots sat comfortably behind the one live one, permanently invisible.

Which is exactly what had happened the first night. The failure was partial, and the alarm only detected total death. It would have watched the original problem happen and said nothing.

Partial failure is not the unusual case here. It is the normal one — the rotation exists so the slots can fail independently. The counter is now per slot, and a slot that writes nothing three turns running is reported by name whether or not its neighbours are thriving.

The second version, which could not count

The alarm fired. Its message read:

*** the BIDDING slot has written NOTHING for 3 consecutive turns.
Dead slots: origination, bidding of 3. ***

There are four slots. There had been three when the message was written; a fourth was added the same day, and the string was not updated with it.

So the instrument built specifically to catch stale reporting was, in its own alarm text, reporting stale information — while otherwise doing its job perfectly. It now counts the slots rather than asserting how many there are.

We enjoyed that one more than was strictly warranted.

What it caught

With both corrections in, the alarm did the thing it exists for. Twice.

First the origination slot: three turns, nothing written. Then the bidding slot: three turns, nothing written. Between them, half of every expensive turn producing no data at all — the same condition as the first night, except that this time the system said so within forty-five minutes instead of us finding it the following morning.

And the diagnosis was immediate, because the two failures are one failure. Nobody can bid because there are no auctions. There are no auctions because no agent has chosen to found a venture. The economy starves from the top.

That is not a defect in either slot. Both were working. The alarm was reporting, accurately, that the civilization had nothing for them to do — which is a fact about the civilization, and precisely the kind of fact we had previously been unable to see.

A more uncomfortable thought

Here is what bothers us about all of this.

The alarm is the first piece of infrastructure in the system that checks a claim rather than a code path. It does not ask whether a function ran. It asks whether anything was actually written, and complains when the answer keeps coming back no.

Everything else — a hundred and ninety-six tests, all passing — verifies that code does what it was written to do. Almost nothing verifies that what the system says about itself is true. Which is why a day spent looking closely turned up seven separate defects in which the machinery was correct and the report of it was wrong, four of them introduced by us while fixing the other three.

The alarm needed two corrections of its own before it was trustworthy, and one of those corrections was that it misreported its own state. We are not going to pretend that is reassuring. It does suggest the category is real and common rather than a run of bad luck, and that an instrument which checks claims is worth having even when the instrument itself needs checking.

It is still the only one we have.

Next: the agents had never been paid, and the reason was that fifty of them shared a name.