Technical Litigation

Disclaimer: I do not take credit for this term, I first had it explained to me by our vice president of operations at my current employer.
Everyone thinks they are special. You can have an application doing the same thing for hundreds of customers and each one thinks that the unique problems they are experiencing must be your fault. Quite simply, technical litigation is the process taken to prove that [technical situation X] does not lie within your jurisdiction of control.

Recently, I wrote two programs for the single purpose of technical litigation. I spent approximately 3 hours of my 12 hour workday writing code to prove a point. The first program simulated the digital signature component of our testing environment. In reality, after I pulled the correct data the coding only took about 30 minuets. Still it was time I could have spent doing other tasks that would have a more long-term impact. The second program was designed to prove that the way we base64 encode some binary data was correct. This second program compared byte lengths and hashes of the decoded strings from four different base64 decoder implementations.

To be fair, this is an atypical example but I would say that roughly 10% of my time as a developer is spent dealing with issues of technical litigation. I am admittedly curious how much other organizations deal this issue - particularly among developers. During the course of technical litigation I have learned a few things from my experience:

#1 I tend to pursue problems differently when I hold the premise that I'm not the one at fault.

I can't really say why, but I have a tendency to be more methodical with problems that I don't think I caused. It is almost as if I have a clearer mind when I am looking at someone else's mess. Maybe it is the the fact that personal emotion is taken out of the situation. Maybe I am not distracted by the possibility of learning a hard lesson at the end of the day. Maybe it is simply my professional inexperience. I cannot say definitively, but the fact remains that I approach problems of technical litigation very differently.

#2 My job is to attack the problem, not the customer.

This is not something that I have a major issues with, but I can see that there is a natural competitive tendency to beat the other side when trying to prove a point. When the other side helps pay your salary then it it important to look at them as a partner who ultimately wants the same thing you do: for the problem to go away.


#3 Pointing fingers is never more important than moving on.

I learned this by observing breakdowns in #2. Whether you are a developer, a customer, or a manager, accepting responsibility is often a difficult emotional process. The end goal of technical litigation should always be a problem resolution. Technical litigation should be though of as simply a middle process to determine who needs to do the solving. Therefore, once the responsible party has been identified, there is no need to dwell in the technical litigation phase. Pointing fingers and arguing will delay the problem solving process and will not benefit either side.


#4 Admit when you are wrong.

This closely ties into point #3 but is in my opinion the most difficult aspect of technical litigation. The software industry attracts many individuals who have a dominant "C" personality (refer to the DiSC model). The high C personality values accuracy and detail which gives them an advantage when working with complex technical concepts. Given that accuracy is important to the C, it is often much more difficult to accept being wrong.

List of my Favorite Podcasts

1. Manager Tools
2. Java Posse
3. IT Conversations
4. FLOSS Weekly
5. Assault Rifle Basics Video Podcast
6. Ask A Ninja
7. Despair, Inc. Video Podcast
8. UPDATE: Gun Talk
   

Application Visibility

So it's your first week at the new job and you are charged with building a new product for one of the largest customers in your organization's history. After a short period of flattery, the gavel that is reality smacks down hard to let you know that your ass is on the line. You have the much needed guidance from your team and extremely competent management that understands software - but ultimately success or failure is at your doorstep.

So you are in pretty deep - what's the plan?

1. Carefully test your code
   
2. Perform code reviews
3. Put your pride away and ask for help - even if you think you have the right answer you may not!
   
4. Be honest

Fast forward to a few days after you have successfully released your application into production. You stuck to the plan: you tested very carefully, you had your peers scrutinize your work, you asked for help, and you reported the state of the project honestly and consistently. The customer was surprised that things went so well. Congratulations, you just did the impossible: exceeding expectations in a situation where things could have easily gone horribly wrong.

After a week in production some issues are discovered. It's your application so you need to fix it. Easy enough right? Just plow through thousands of log entries and piece together what went wrong. Not exactly because you completely neglected to think about what types of problems might occur in production, therefore you are not logging the information needed to investigate the customer's complaint. You have no visibility into your application. The only course of action to take at this point is to retroactively update the code in production to give you the visibility needed to solve the problem.

The mistake was costly in terms of personal pride and hardship, but the lesson is valuable. You learn that there is no way of telling what an application is going to do in a high volume production environment where real data is being passed in. Thus, it is absolutely essential that you build in enough visibility to resolve issues that appear in production. You also learn that nothing of value comes easy.

Beware Of The Technology Advocate

Windows or Linux? RedHat or Ubuntu? GPL or BSD license? vi or emacs? JSP or ASP? I can't tell you how many times I have had a discussion with a friend or co-worker that starts like this:

"You use technology x?! ... Did you know about technology y, its way better..."

or the ever popular negative approach:

"System x sucks, system y can do..."

I hate such discussions, mainly because (generally) both sides have a valid point. Technology does something to us when we learn how to use it to solve our problems. Suddenly the hot new whatever becomes useful, and despite its quirks makes us feel so good that we have to advocate its usefulness to the rest of the world. This would be fine if there were no technological overlap. Fortunately for all of us technologies do overlap.

Enter conflict. It is inevitable whenever two technology advocates get together who have similar strategic objectives but different technological toolboxes. The conversation will usually start out like a game of chess but typically ends in a pissing contest. Nobody benefits from this and everyone leaves irritated. So if you like your Windows Vista work environment and that obnoxious Linux guru comes by your cube to harass you, try to keep a few things in mind:

1. Technology shouldn't be a religion, so don't treat it as such. Unlike with your religion, technologies have gray areas that overlap. Don't look at competing technologies as exclusively good or exclusively bad.
2. Java is paying for my home. There are a lot of people who swear Java sucks (just ask Google). But I write Java every day, I get paid good money, and I like what I do. My customers are happy and my company is making money. Despite the laundry list of reasons why Java sucks from very competent people, it is still self-evident that Java doesn't suck.
   
3. Is a drill better than a chainsaw? A prudent person could not answer this question without knowing specific situational information. However, the technology advocate will swear that he or she has used a chainsaw before and it worked every single time: faster and with better results than your drill. Don't let the advocate talk you in to installing a wall anchor with a chainsaw.