When Worse Is Better

Let me pull back the curtain a bit on how my work week ended up. I think this is a good example of Worse Is Better that’s worth sharing.

Last Friday we had an impromptu meeting to discuss an upcoming project for a long-time customer. The scope of work seems simple enough: add a new HDMI input in their auditorium so they can share an extended desktop using a laptop dock. There are a few problems we’ve run into:

  • The staff who designed and sold this solution no longer work at our company
  • There is scant documentation about the work to be done
  • This system has a history of issues resulting in nobody wants to be the last one to touch it

So this week, I needed to track down the last revision of source code, the latest system drawings, and formulate a plan to make the necessary changes. Thankfully I got help from the right people, and I had code and drawings by the end of Tuesday.

The Past

Wednesday, I sat down with the code and a hot cup of coffee to see what state it was in. It was still a mess.

I’m not the original programmer, but I’ve worked with this code for over four years. When I was first shown the room, the customer demonstrated how simply ramping the volume on the touchpanel caused the whole system to bog down. I couldn’t believe what I was seeing. Opening up the program, I noticed that serial communication from the DSP was passed between four separate programs, each one duplicating the processing logic. So, a response to a volume change in one program was causing all four to process the incoming data coming back from the DSP. Visually, the problem looked like this:

The outbound signal flow isn’t too bad, but the incoming responses are processed multiple times and this was visibly slowing things down for the end user. For a single core processor, the duplication of work came at a cost. I saw noticeably better response times after changing it to this:

That was enough to make the customer happy. I had a short window of time to work in (“Hey, since you’re here, can you look at something real quick…”) so I couldn’t sit down and really understand everything that was going on in this program. And since the logic was spread across all four, making changes meant having to visit every program. And SIMPL Windows compiles slowly.

This situation has continued for the past few years: I’ve never been given ample opportunity to rewrite this code. Every time it’s been touched, it’s an ad hoc support issue, so there’s not enough time to bill against it. The weight of all the technical debt crushes my spirit any time I’m told there’s another issue to resolve.

The Present

On Tuesday, I found out that we would be installing/commissioning the new addition to this room the following week. I had almost a full week to rewrite this mess and I was going to take a chance!

I’m fully aware of the Second System Syndrome, so I told myself if I wasn’t showing a fairly completed system by end of Thursday, I would use Friday to hack in the new feature to the existing pile of code. Giving myself two days to completely rewrite this code was a stretch, but I had some advantages:

  • There’s existing, working code I can reference for how the devices are controlled
  • The UI is already done, I just need to create the back-end logic
  • I’ve built a collection of standard modules I can use to speed through my rewrite

First, I sketched out how the program architecture would look. I planned to keep the same overall structure, but move more processing into the main program. The other three programs would only handle UI elements. I started work from an empty program and could feel the weight of the old cruft melting away.

As Thursday drew to a close, I felt the panic of the impending deadline. I had finished all the programming for the main module, so the system was functional. But I was still working on the in-room UI handled by the other three programs. Finishing one program would mean that I’d be nearly finished with all of them (after a quick copy and paste), but I started noticing each room behaved slightly different depending on the room combinations (did I mention this room was divisible?). So I had to face reality: there just wasn’t enough time to pour over each room and figure out the little bits that made it unique.

I think this is an important moment to highlight. I’ve written code in the past where halfway through the project I wish I could throw it out and start over, but you need to resist that urge when there’s a limited amount of time to deliver the product. I set myself a clear guideline: if I didn’t have 90% of the system working, I’d have to abandon the rewrite and go back to the original (messy) version.

On Friday, I stashed my changes away and pulled up the original program. Three hours later, I had all the changes made, tested, and packaged for next week. The time I spent working on my rewrite meant I knew the UI inside and out and had looked deeper at the logic going on behind the scenes, so making changes was painless this time.

The Future

I am hopeful the day will come when we do a serious renovation to this room and I’m afforded the chance to finish my rewrite. I’ll keep the code around and poke at it when I find time or get an idea, but at least next time I won’t be starting from scratch.

I think this story emphasizes it’s important to not blind yourself to doing The Right Thing if it keeps you from delivering anything at all. We can satisfy this project’s scope today, and I can incrementally improve the code until the next time this system comes up.

And after 10 years of Crestron programming, I still feel like I haven’t found the best way to program a divisible space.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s