In November 2023, Optus’s network went down. ACMA later found that 2,145 people could not reach 000, the emergency call service.
And 369 required welfare checks on people who had tried to call? Not done.
Everybody was very sorry.
Mistakes had been made, processes were reviewed, promises for it to not happen again were made.
And it didn’t happen again. Until it did in September 2025, that is. A botched network change disrupted emergency calls for about fourteen hours across parts of Australia.
Two deaths were linked to the outage.
So as not to unfairly point fingers at just one operator here, many Australians will remember when Telstra’s mobile network fell over nationwide earlier this year. It did so because an old GPS timing equipment was reset during a routine power-supply swap and then woke up thinking it was 2006.
Its manufacturer had been warning customers to patch that exact flaw since 2020, most recently in January. Warnings were ignored. There had also already been trouble with the timing system the previous October. Staff applied a workaround, but the underlying anomaly was not escalated for further investigation.
She’ll be right. Great strategy for some things, not so great for critical infrastructure.
Anyway, I ended Part 1 with the question of “How does a technology then earn the right to disappear?”, and alluded to the answer being two words.
Those two words are:
Just culture.
Just Culture is the doctrine behind the answer to how a technology earns the right to disappear into the background of your life.
How so?
What even is Just Culture?
The simplest version: it’s a rule for what happens after something goes wrong. Instead of asking the usual questions of who did this and who do we punish, we ask what about the system let this happen.
Focus on and fix the system.
So, a blame-free idealistic kumbaya kind of a thing, then, huh?
No. It is precisely not that. The person who makes an honest mistake and reports it gets protection. In exchange, the system gets the information. In exchange for the information, the system gets redesigned, or tweaked, or otherwise improved.
Honest mistakes should be safe to report. Gross negligence and wilful misconduct can still have consequences. Even so, firing someone does not, by itself, fix the system that made their mistake possible.
Aviation gives us a particularly developed example of what this looks like in practice.
In 1974, TWA Flight 514 hit a mountain approaching Dulles. Six weeks earlier, another airline’s crew had narrowly escaped the same hazard after misunderstanding the same approach clearance. That airline shared the lesson internally, but only internally – there was no established system for sharing that warning across airlines. The disaster helped spur the creation of ASRS.
NASA’s Aviation Safety Reporting System (ASRS) turns fifty this year: a voluntary reporting system for aviation errors and hazards. NASA administers it separately from enforcement. Reports are normally de-identified and protected from use in FAA enforcement, with exceptions including accidents and criminal offences. Eligible reporters can also receive protection from penalties.
The reports (millions of them over decades) informs procedures, training, and eventually even cockpit design. For the rest of us, the CALLBACK newsletter makes for fascinating reading.
International aviation standards give the safety investigation a specific job: prevent the next accident, rather than assign blame or liability. Courts and regulators have separate jobs. The resulting reports make for fascinating reading: try the NTSB’s investigations in the US or the ATSB’s in Australia.
I have seen the same failure in technology: lessons of incidents kept inside the company, when the incidents and lessons were captured and shared at all. Often, one or both were not.
We can use another phrase to capture the essence of this approach:
Be hard on systems, soft on people.
This concept translates to a lot of domains, by the way.
There’s a very human tendency to vilify ‘others’; it’s easy to see people who hold very different political beliefs to ours as misinformed, or stupid, or evil, or any number of unproductive things. Our party often encourages us to do so.
We must not.
The same applies to people who join high-control groups or cults; it’s easy for outsiders to think they are stupid, or evil, or both.
We must not.
Calling them such gets you nowhere fast. Our beliefs are shaped by the communities around us. High-control groups can offer belonging long before revealing how much control they demand.
Calling someone stupid for being caught inside one rarely gives them a route out.
Be hard on the machinery that produced the belief and soft on the human holding it, and at least you’ve left a door open. (That does not excuse people who knowingly exploit or harm others.)
Now, back to the technology in our lives, and how its makers run the doctrine.
Backwards is how.
We’re soft on systems: we are asked to waive as much liability as is legal by license agreements, failures are explained away with what Dan Davies calls an accountability sink: a structure and an arrangement in which the people affected cannot get anyone with power to change the decision to take responsibility.
And we’re hard on people: media literacy courses as the official answer to industrial-scale synthetic content. Individual self-control as the answer to various attention-grabbing systems supported by billions of dollars. Forty opt-out privacy toggles buried under seventeen menus as the answer to surveillance. The increasingly explicit stance that whenever the product harms you, you were using it wrong.
You should doomscroll less.
You should spot the fake.
You should have read clause 43(b).
The system, meanwhile, walks away, unscathed, having extracted money out of you.
Australia has no shortage of cautionary tales. One such thing was dubbed Robodebt.
Robodebt raised unlawful debts against hundreds of thousands of people. The Royal Commission traced warnings about its legal basis back to 2014, before the scheme began. It nevertheless continued for years.
During those years, the most vulnerable among us got debt collection, garnished tax refunds, and years of fear; some paid for it with far more than money. Warnings repeatedly failed to stop the scheme.
When the Royal Commission finally arrived, it called the scheme “crude and cruel”.
Applying this principle to Robodebt should have meant the following.
Staff such as Colleen Taylor raised the alarm. Those warnings should have triggered a pause in the disputed debt recovery and an independent examination of the method.
Those who knowingly pushed an unlawful scheme ahead should still face consequences.
Just culture does not treat gross negligence or wilful misconduct as honest mistakes. Being hard on systems is never amnesty for the people who build or operate them in bad faith. And do we ever have people building systems in bad faith nowadays!
A reporting channel is only the beginning. Someone needs the authority to stop the harm, the obligation to respond, and a way for the rest of us to see whether anything changed. Otherwise we have built an extremely well-documented disaster. Just culture makes it safer to tell the truth. Regulation and liability make it harder to ignore it. We need both.
What we need
On the technology front in particular, we need at least three things urgently:
Protected reporting. We need protected reporting channels for the engineers inside these companies. X is not the right platform for this. It needs to be boring, procedural, unglamorous machinery; an ASRS of the technology sector. This is not a new ask. In 2024, the Federation of American Scientists published a proposal for an AI incident reporting system explicitly inspired by ASRS. Pieces already exist: the AI Incident Database collects incident reports, while the European Commission’s AI Act Whistleblower Tool offers a confidential reporting channel within the AI Office’s remit. We need these functions, with meaningful protections and follow-through, across the technology sector.
Independent investigation. We need an accident investigator for algorithms for when automated systems hurt people at scale. A blameless, independent, agency that publishes its findings in detail.
Regulation and liability. We need liability that lands on whoever develops or operates the system, not on its victims – whether it’s the social media companies deploying products they know are harmful, or the AI frontier labs talking about the existential risks of products they choose to develop and deploy.
This should have always been the deal. Aviation shows what parts of that deal can look like: lessons written into institutions, procedures and engineering, often after terrible losses. In return, it got to become boring, background, mostly well-functioning, trusted infrastructure.
Technology that wants to disappear into our lives must first become radically visible to the systems that hold it to account. These systems must exist and be willing and able to hold them to account.
Earn that, and you may then vanish into the walls with our blessing.
Refuse, and you stay where we can see you. And keep your hands where we can see them.
Ps. What “hard on systems” looks like inside an organization, including incident reviews that don’t eat your people, escalation contracts, AI accountability that survives contact with your general counsel? That’s what you hire me to figure out. Let’s talk.
Ps2. To the pilots reading: yes, I know just culture is under strain in your world too. It needs maintenance like everything else, and there are forces that would rather not see this ‘inefficiency’ in the system which must be resisted.