The Time I Almost Got Fired

Back in 2010, I had just started at a new company as a full-time AV programmer. I already had a couple years of AMX programming under my belt, and I’d just started getting the hang of Crestron and SIMPL Windows. Most of our engineering and programming team were East Coast-based but I was the lone West Coast programmer. On top of that, I was brought in with the expectation I would work on new installs as well as handle service calls. That meant I had two people I treated as a boss, and those two people rarely coordinated on my schedule. Anyway, I’m getting off-topic. Let me tell you about the time I almost got fired.

I’d probably only been working for the company a month or two at that point. I was still fairly green with Crestron but slowly getting the hang of it, and I was in the office working on my next project when the director of engineering (my East Coast boss) called.

“I need you to head down to San Onofre and meet a tech tonight. You’re going to troubleshoot a conference room. It’s AMX. I’ll send you the program in a minute.”

A few minutes later the source code popped up in my email. I unzipped it and right away noticed the PepperDash naming convention on the program and modules. I quickly learned that our company contracted out about a quarter of the programming work, so I spent a good amount of time figuring out other people’s code. Thankfully, PepperDash code is clean and uniform. I learned how they structured their programs out of modules and could quickly tweak the UI behavior or fix bugs in device control. I looked through the project and it seemed to be a fairly straight-forward conference room with a handful of sources.

Our office was located in Camarillo, CA and driving through Los Angeles meant sitting in traffic, so I tried to wrap up what I was working on and hit the road. Camarillo was far enough north that we didn’t see much of the effects of LA, but I did have to drive to job sites on a regular basis. I didn’t appreciate getting something sprung on me last minute like this. I’d arrived at the office around 8am that morning and now it looked like I probably wouldn’t be getting home until after 10pm. I was young(er) and still new at the company, so I figured this was all part of paying my dues.

3 hours later, I’m getting close to the site and call the tech I’m supposed to meet up with. He apologizes, “I can’t make it over there, I’m stuck at this other job. Can you handle it?” I say sure, and ask for the customer’s contact info since I didn’t have it. I jot it down, call the customer, and link up with him. As we’re walking to the conference room, I explain that the other tech got hung up so it would just be me tonight.

When we get to the room, I ask about the problems they’re having with the system. I also grab the touchpanel and start pressing buttons to see if I can duplicate what he’s describes. He starts off by saying, “You’re supposed to install a Blu-ray player.”

I look up from the touchpanel. He’s glaring at me. I can see now I’m the latest person to get dragged into a situation where there’s already been a lot of back-and-forth. I check the equipment rack and sure enough there’s an empty shelf where a Blu-ray player is supposed to sit. I give the other tech a call and ask about this missing player. “I’ve got it with me, so you should probably grab another one since I won’t be there tonight.” We hang up. I apologize to the customer, but I now need to go pick up a Blu-ray player.

The closest place that sells electronics is a Wal-Mart, so I head there and find a mid-range Samsung player. I also grab a movie for testing. Round-trip there and back and I’ve wasted almost an hour. More glares upon my return.

Back in the conference room, I unbox the player, slide it onto the shelf, pull out the rack, and… we’ve got an RGBHV matrix switcher?! I call the tech again. He says, “The cable path is too far away for HDMI, so we’re planning to run component direct to the display.” I eyeball the cable pull while he’s talking to me and see we’re probably only going about 30 feet.

And this is where I hit a few snags:

• I didn’t bring cable for a cable pull
• I don’t have fish tape to get the cable up the wall
• I don’t have tools or connectors to terminate RGBHV 5-wire
• I don’t have an IR bud for transport control
• I don’t have cable ties to dress the new cables going into the rack

But what can I do in this situation. I’m the only person on-site, so I’m going to do whatever I can to make sure this visit isn’t a total loss. I temporarily hook up the Blu-ray player at the display to test my code changes. The input on the display switches correctly when that source is selected. I get the exact model of the player and find something reasonably close in the InConcert database for control. I test all the programming changes I possibly can, install the player on the shelf in the rack, plug in power, and try to make it look as neat as possible.

I’m ready to pack up at this point, but due diligence requires I ask if there were any other problems. “The touchpanel doesn’t respond to every push, sometimes it takes a few tries.” OK, I go into the panel’s settings to see which WiFi channel is in use. Luckily this panel has a built-in network scan that shows how congested each channel is. And of course, it’s setup on one of the busiest channels. I pick one that is fairly quiet and set the access point to the same. Another 30 minutes of testing and we’re satisfied that it seems to be working better.

I pack up, get out to my car, and send off an email to my boss before starting the 3 hour drive back home…

The Next Day

I get to the office and see a reply to my email. “Call me.”

I knew right away this wasn’t going to be a pleasant call. I don’t remember the exact words, but I do remember hanging up and thinking, Well that’s it, better start looking for another job. He expressed disappointment that I’d left the job unfinished, and I remember agreeing that I too was unhappy about leaving things as I did. There were promises that it wouldn’t happen again. I went to work every day for the next few weeks expecting to be let go.

But it never happened.

So was I almost fired on that phone call? I don’t know. It sure felt like I came close. Maybe having enough successes after that day saved me? Or never making the same mistake again earned me some forgiveness? My boss was a good guy, and I think he stuck up for me a lot, probably even during this screw up.

Lessons Learned

We learn when we fail, and this failure taught me some valuable lessons.

Be prepared. I went to the job site without the whole story. I had no idea there was missing equipment, so when I heard it from the customer it caught me by surprise. Take the time to ask the necessary questions, figure out exactly what a successful project means. I should have called the other tech before driving 3 hours to confirm he’d also be on-site and what he was planning to bring.

Bring tools. At the very least a screwdriver with a few different bits, cutters, strippers, and zip ties. And if someone tells you to just bring a laptop, bring the basics anyway. I don’t want to step in and do somebody else’s job, but I do want to make sure the job is completed before I leave. The more uncertain I’m feeling about who’s responsible for what, the more kit I’ll pack. Possibly need to re-terminate connections? Bring a cable tester, crimpers, and connectors… just in case.

Manage your resources. I was thrown at a problem last minute. The other tech was divided between two projects as well. This is poor resource management. I imagine there had already been a steady stream of failures on this particular project and throwing us in last minute just added one more. The result was an angry customer and less future business from them. If I had checked how I fit into the larger solution, I could have alerted the entire team prior to the email I sent to just my boss at 10pm.

Be professional. I’d like to say I wouldn’t walk blindly into this situation again, but this is AV we’re talking about. There will always be a project that falls apart because nobody gives it their full attention. In that moment when you’re sympathizing with the customer, remember you’re also the face of the company. Keep it professional, and keep things moving toward a solution. Was I setup to fail? Yes. Does the customer need to know that? No. You’re there to get the job done, so do your best to get it done.

So, that’s the one time I thought for sure I was getting fired. I’ve had plenty of screw ups since that day, but they didn’t seem to carry as much weight. Or maybe I recognized it wasn’t wholly my fault. For me, I’ve always treated these experiences the same way: How do I keep this from happening again?

Here are my methods:

Check the scope of work. You know those projects where you feel like you aren’t getting all the details, or you feel lost in what you’re trying to accomplish? It’s most likely the scope of work is vague or missing. I’m sure every small company starts out where you figure out the project on the fly, but if you want to be consistent and accurate, you need a detailed scope of work that the stakeholders have agreed upon. Otherwise, you’ll suffer death by a thousand cuts as the customer tries to hammer their idea into shape at your expense. Looking back, I would have insisted on getting a copy of the documentation for this project. At least then I’d know what was in the system before setting foot in the room.

Have a checklist. Once you’ve checked off that something is done, then it’s done, and you can stop worrying about it. Focus on the areas that are left unchecked. This will keep your systems working consistently, too. I’ve learned to push back on installers wanting to circumvent a system defect using programming. You’re just delaying another failure and adding untested complexity to the code at the last minute by going that route. Check everything off your list and if it fails, call it out, and fix it. In my case, I tested the Blu-ray player as best I could so I knew I wouldn’t need to revisit the programming later, but the installer needed to come back and finish his part of the job.

And lastly:

Accept that some things are outside of your control. I used to worry that I wouldn’t get a system working 100% before turning it over to the customer. Sometimes the project timeline is unrealistic, the design is incomplete, or the installation crew made 10 bad terminations that day. We’re just programmers. We’re the last ones to touch the system before its turned over, but we can’t own every mistake in the weeks leading up to our time on-site. It’s not asking too much of your team to make sure we’re given a fair shot at delivering a successful install. Get the Project Manager involved early when you hit snags, they’re good at clearing away obstacles.