On Call Sometimes you’re the one on the phone, and other times you’re the one that issued the cry for help. Welcome to a story from the On Call archive where a Register reader turns the tables and claims the glory.
Today’s tale comes from “Rob” and is set in the 1980s, at a City insurance company. As was common in those days, the IBM System/3 midrange computer lurked within the data centre, and this particular outfit ran its underwriting software on the system.
The System/3 that Rob was responsible for luxuriated in a spacious 64KB of RAM and featured nearly 2.5MB of mass storage via an internal disk (likely the IBM 5444, which measured 14 inches in size). Backups were as important then as they are today, and Rob’s IBM was no exception. “We backed up the internal disks onto 9-track tape cartridges via an external unit,” he told us, “inside which there was a plastic rack into which a number of cartridges could be loaded.”
It was a straightforward mechanical operation. The rack moved back and forth to insert and remove cartridges from the drive. Right up until it didn’t.
Rob came in one morning to find the overnight backup had failed with a near-meaningless message. Naturally, he tried the operation again, but once more the inscrutable message popped up. “It ran for a while, started writing to tape, then broke,” he recalled.
Having likely exhausted his ability and definitely his inclination to investigate further, Rob placed a call to IBM support, who dispatched an engineer. A flurry of activity ensued, during which panels were stripped and the computer’s internals exposed to the outside world. “I left him to it,” said Rob, who wandered off. Hopefully in search of a hot beverage and morning sustenance.
Time passed, but the computer remained inactive. The engineer professed himself baffled and called for backup. Another engineer arrived, armed with more components. More panels were removed. More bits were changed. But still the problem remained: backups started, ran for a bit, and then failed.
“I left them alone,” said Rob, “partly because I knew it would have been irritating for them to have some bloke hanging around, and partly, to be frank, so that I could distance myself from their failure in case I needed to.”
A wise move, because morning was turning into afternoon. Underwriting was not happening, and the users were getting a bit shouty. Rob therefore returned to the group of engineers to get a progress update.
“They were deep in conversation,” he recalled, “huddled at one end of the computer to discuss their latest can’t-possibly-fail wheeze.”
Sensibly, he decided not to interrupt their trains of thought and so simply hung around the defunct computer until one or more of them came up for air.
Bored, he started poking at the IBM. “The door to the tape unit was open and idle curiosity made me reach inside and push the black plastic rack with my finger, as one does,” he said.
“It moved, but the movement looked odd. I poked it again, a bit harder, and realised that while one part of the rack moved, the other didn’t. It was supposed to be one unit.” How else could the tape cartridges be inserted and removed?
He looked closer and saw a jagged crack right across the bottom of the unit.
The engineers were called over to look at Rob’s discovery and all went a bit quiet. “They had inspected and replaced internal parts of the machine, but hadn’t thought to look inside the external tape unit.”
Yes, of course it all began working as soon as a new unit was delivered and installed.
And Rob? “Naturally enough, I claimed all the credit for making the problem go away.”
Which, in a very real sense, he had.
There’s a saying that goes “When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.” Although, to be frank, a cracked rack is far from improbable.
Have you ever had to call out the cavalry only to solve the problem yourself? Or been that engineer left red-faced by a user’s innocent observation? Let us know with an email to On Call. ®