There was a light that never went out

Why we picked it · the editor's summary

Working from a procedure that issued every transceiver replacement at once, a technician unplugged all fiber paths across the affected devices within 13 minutes. Google's write-up names two errors, correctly following an incorrect procedure and incorrectly following a correct one, and lists reinforced training next to more automation. Lorin Hochstein calls blame-and-train a red flag that the company does not understand how work actually gets done in its own system, and expects the new automation to bring failures of its own. If your incident reviews end in a training action item, ask how the procedure came to be that way before you sign off on the fix.