The Best Type of Customer is The One That Makes You Better

I have had to work with cryptography and digital signatures exhaustively on an application level. One of the hardest things I had to do was create and verify digital signatures based on data being passed around via HTTP. I quickly learned in practice what I already knew in theory: characters are not bytes!

Case in point. One of our largest customers came over to our platform from a competitor that was getting out of the payment business. This customer was very concerned about digital signature verification, as this had been a long-standing problem with our competitor. I was tasked with building an interface into our existing application for this customer so they would not need to make any code changes. Testing went well and we hit our go live date.

It quickly became apparent in production that too many international transactions were failing the digital signature check - the same type of problem this customer had been experiencing all along. Our cryptography expert suggested that I explore character encoding as a source of the errors. After some testing, it amazed me how easy it was to inadvertently change data while passing it from country to country, application to application, and programming language to programming language. Character encoding seemed very likely to be the avenue to the solution. Unicode, ASCII, UTF-8, big and little endian - there were a menagerie of topics related to character encoding I needed to consider. After spending several days trying to understand the complexities involved with mapping characters to numbers, I still felt that I had only an elementary understanding of character encoding.

After frantic research and troubleshooting, the problem was caused by the translation occurring between our web and application tiers. The customer was encoding the data in UTF-8, signing it, and passing it along to us for verification. We were taking that data in UTF-8 on the web tier, but in our application it was decoding as US-ASCII. The final fix consisted of (1) altering a few characters on one line of code and (2) me becoming a slightly better developer.

Lesson learned: From a developer's perspective, the best type of customer is the one that makes you better.