Two Weeks Down for a Fifteen-Minute Fix

A real incident at a client of ours. Identifying details have been removed.

If the link between your offices died right now, who would you call? Not which company. Which person, and do they still answer?

One of our clients, before they were our client, had an answer to that question. The answer was the owner's nephew. He was away at college, out of state.

What happened

The client is a two-location company here in KC. The two offices were connected by a site-to-site VPN, the standard way to make two buildings work like one network: shared files, shared systems, one business in two places.

One day the VPN went down. The offices could no longer see each other. Most likely the internet provider had changed something on their end, which is a routine thing that quietly breaks this kind of connection. Whatever the trigger, the result was the same: the link was dead, and everything that depended on it stopped.

Their IT history up to that point was reactive break-fix, and the person who handled it was the owner's nephew, who had since left for college in Virginia. There was no provider. There was no contract. There was a relative on the other side of the country, busy with school.

So the business sat. Employees kept asking when it would be back. Nobody in either office could do anything about it, because nobody in either office knew how the connection worked or had the passwords to touch it. It stayed down for about two weeks.

The fix was never the hard part

When I got the call, the actual repair was straightforward: log into the routers at each office and reconfigure the connection with the current addresses. This is routine work. On its own it's about fifteen minutes, not a two-week project.

The hard part was getting in the door. Nobody in the company knew the passwords to the network hardware. Nobody knew the password to the server either. The only copy of those credentials was in the nephew's head, in Virginia. Before I could fix anything, I had to reach him by phone and have the passwords emailed over.

Think about that dependency for a second. The keys to the company's network lived in one person's memory, and that person didn't work there, didn't live in the state, and had no obligation to pick up the phone. He did, and that's the only reason this story is about two weeks instead of longer.

What two weeks actually cost

I can't give you a dollar figure for this one, and I won't invent one. What I can tell you is what the office looked like: two locations running blind from each other, staff asking a question nobody could answer, and an owner who had no move to make. Not because the problem was hard. Because there was no one whose job it was to fix it.

The outage wasn't a technology failure. Site-to-site VPNs break sometimes; that part is normal. The two weeks came from the gap around the technology: no one responsible, no documentation, no credentials, no plan. A company with someone already watching the connection turns this into a service call. A company without one turns it into a season.

What to check at your own business

You don't need us for this. Ask these around your office this week:

• If the connection between your locations died today, who gets the call, and is that person under any obligation to answer?
• Does anyone in the building know the passwords to your routers and your server?
• Are those credentials written down somewhere the business controls, or do they live in one person's head?
• When your last IT helper moved on, did what they knew move on with them?

If the honest answer to the second or third one is no, you're one routine ISP change away from finding out what that's like.

Who holds your passwords? Go find out today, while it's a question and not an emergency.

Tech 360 Solutions provides managed IT and cybersecurity for small businesses in the Kansas City metro. Built for HIPAA, FTC Safeguards, and other regulated environments.