Skip to content
HN On Hacker News ↗

The Better the Autopilot, the Worse the Pilot

▲ 74 points 77 comments by julienreszka 2w ago HN discussion ↗

Pangram verdict · v3.3

We believe that this document is fully AI-generated

99 %

AI likelihood · overall

AI
0% human-written 100% AI-generated
SEGMENTS · HUMAN 0 of 1
SEGMENTS · AI 1 of 1
WORD COUNT 234
PEAK AI % 99% · §1
Analyzed
Jun 9
backend: pangram/v3.3
Segments scanned
1 windows
avg 234 words each
Distribution
0 / 100%
human / AI fraction
Verdict
AI
Pangram v3.3

Article text · 234 words · 1 segments analyzed

Human AI-generated
§1 AI · 99%

Automation doesn't make operators more careful. It makes them forget how to be. The more reliable the system, the less ready the human.The argument for automation is that it frees up cognitive bandwidth. Fewer routine decisions means more headroom to think carefully about the ones that matter. What actually happens is the opposite: when a system reliably handles a task, the human monitoring it gradually stops monitoring, because nothing ever goes wrong, and sustained attention without feedback is not something brains do voluntarily. Aviation has a name for this: automation-induced complacency, and it shows up in accident reports where pilots failed to notice system failures they would have caught immediately if they had been flying manually. The irony is that the better the automation, the worse the problem: a system that almost never fails produces operators who are almost never ready for the moment it does. The countermeasure is deliberate:

Identify the critical tasks you have handed to automation. Periodically turn them off and practice manually. Keep the interval short enough that the skill does not decay.

Not because the machine is unreliable, but because you are, and the only way to stay ready is to keep practicing the skill the machine has been covering for you. Schedule regular manual practice for any critical task you have automated. The interval should be short enough that the skill does not decay before the next failure.