Sunday, September 30, 2007

How I Spent My Time at Work the Past 7 Weeks

Over the past 7 weeks between 8/6/2007 and the end 9/21/2007 I conducted an experiment in which I tracked time spent on different tasks at work. Weeks were considered between 7:30 AM and 6:00 PM Monday through Friday. Overtime was not considered. The summaries are given below as a percentage of total time tracked for that particular week. For example, if I only tracked 30 hours worth of tasks during a week, then time spent per task was calculated as a percentage of 30 hours.

My goal is to gain a deeper understanding of the software development process at my organization, to quantify assertions I hold regarding my activities, and to empirically measure changes by way of historical context.

Below are the summarized categories over the 7 weeks with a short description:


Development: 48%

Development is the task of software construction. I count development time as reading, writing, or debugging code.

Customer Support: 12%
Customer support related tasks are counted as the time I spend on requests initiated by our customers. This generally involves communication and log mining activities.

Time in the ROC: 08%
Due to the secure nature in which we must conduct business, all access to our production machines must be done in a secure location known as the remote operations center or ROC.

Meetings: 07%
No explanation needed.

Report generation: 05%
Creating both internal and customer reports.

Team Help: 05%
Receiving or giving help on specific technical issues. Team meetings were not categorized here.

Design: 04%
Software design activities.

Conference Calls: 03%
No explanation needed.

Testing: 02%
Software testing activities.

Integration: 02%
Development time spent integrating our software product into a customer's system.

Requirements: 01%
Requirements gathering activities.

Email: 01%
Responding to email messages.

Documentation: 01%
Writing technical documentation.

Releasing code: <1%
Releasing new code into our production environment.

Help desk: <1%
Internal help desk requests.

* * *

Analysis

Development Time
I was surprised to see how much time I actually spent time writing code. Although I am a software developer, my perception was way off from what the actual data shows. I think that any developer spending almost half of his or her time coding is pretty good. One factor that may be altering my development time perception is the number of interruptions incurred when coding, I may track this in the future.

Customer Support
This was fairly low but if I was a manager, I would keep a close eye on how much customer support my developers are providing. If it gets up around 15% to 20% it may be time to hire or train level 1 support personnel. Customer support specialists will always be less expensive than software developers.

Meetings & Telephone Calls
As a general rule I dislike meetings and conference calls. My company has a weekly staff meeting every Monday for 30 minuets to get everyone on the same page. I feel that very little if any of my time is wasted in meetings. When we do meet, an agenda is followed and everyone stays on topic. While we are not quite the model of a Manager Tools meeting, I think that we still score an A- when it comes to meetings. Kudos to excellent management.

Telephone calls are a different story. While 3% may seem like a small number, nothing is more effective at getting me off track than a telephone call. A conference call with a customer is rarely a discussion on the thing I just happen to be working on and usually a call requires me to spend a short amount of time preparing. While the call may only be 20 minuets, there may be action items and/or new priorities that cause further disturbance in my schedule.

Requirements and Design
After some reflection on the proportionally small amount of time I spent on requirements, I conclude that most of the requirements have been either already well defined or self-evident (bug fixes). While my role is not that of a systems analyst, I was a bit surprised at how little time I spent thinking about requirements.

Design is another matter. For better or worse most of my design happens as I am coding. There have been a few exceptions to this, however the design while you code methodology is certainly the norm. Coming from a software engineering background this was a red flag and is a topic I will be meditating on in the future.



Conclusions

This was an extremely interesting endeavor for me. I am looking forward to the feedback I receive when I give this data to management. Going forward, I plan to continue tracking my times and refining my methods (mostly self-discipline).

Thursday, September 27, 2007

The Best Type of Customer is The One That Makes You Better

I have had to work with cryptography and digital signatures exhaustively on an application level. One of the hardest things I had to do was create and verify digital signatures based on data being passed around via HTTP. I quickly learned in practice what I already knew in theory: characters are not bytes!

Case in point. One of our largest customers came over to our platform from a competitor that was getting out of the payment business. This customer was very concerned about digital signature verification, as this had been a long-standing problem with our competitor. I was tasked with building an interface into our existing application for this customer so they would not need to make any code changes. Testing went well and we hit our go live date.

It quickly became apparent in production that too many international transactions were failing the digital signature check - the same type of problem this customer had been experiencing all along. Our cryptography expert suggested that I explore character encoding as a source of the errors. After some testing, it amazed me how easy it was to inadvertently change data while passing it from country to country, application to application, and programming language to programming language. Character encoding seemed very likely to be the avenue to the solution. Unicode, ASCII, UTF-8, big and little endian - there were a menagerie of topics related to character encoding I needed to consider. After spending several days trying to understand the complexities involved with mapping characters to numbers, I still felt that I had only an elementary understanding of character encoding.

After frantic research and troubleshooting, the problem was caused by the translation occurring between our web and application tiers. The customer was encoding the data in UTF-8, signing it, and passing it along to us for verification. We were taking that data in UTF-8 on the web tier, but in our application it was decoding as US-ASCII. The final fix consisted of (1) altering a few characters on one line of code and (2) me becoming a slightly better developer.

Lesson learned: From a developer's perspective, the best type of customer is the one that makes you better.

Saturday, September 22, 2007

The Mac as a Software Development Platform

Many of my co-workers (and people in general for that matter) find it strange that I use a MacBook Pro as my primary development platform. The two main camps of developers I have encountered either lean towards Microsoft Windows or some flavor of UNIX. Most of them have never even considered using a Mac for software development. While I acknowledge that some tasks are platform dependent (ASP.NET for example), many tasks allow developers the freedom to choose their own environment.

The rest of this article will elaborate on features the Mac has that makes it a good development environment. In the conclusion I will speculate on reasons why I think the Mac has not become popular among the development community.

Where am I Coming From?

Most of my development tasks involve writing Java code and solving technical problems. I work in the payment industry and our customers use just about every platform under the sun. My company produces software written in Java, ASP, ASP.NET, Perl, PHP, and ColdFusion.

Security

By default my MacBook Pro was fairly open. No password was required at login and account names were displayed on startup. It was easy enough to reconfigure and go back to a more traditional UNIX login situation that prompted for a user name and password. In addition, I was able to easily and transparently encrypt my home directory with file FileVault. Finally, the MacBook uses ipfw, the FreeBSD firewall. Ipfw can be configured through a tool in System Preferences and optionally its full power can be exploited through the UNIX terminal (more on this later).

Text Processing

A good portion of my time during the week is spent on text processing tasks. Usually I am creating a report, mining log files, or changing character encodings. Darwin conveniently provides almost every tool I need such as sed, awk, grep, Perl, cut, and iconv. A great way to impress someone who brushes Macs off as a computing platform for hippies is to open up a terminal and burn through an ugly log file with a only few commands.

Darwin Ports

I have always liked the OpenBSD ports system. While not exactly the same, Darwin Ports makes it very easy to install open source software on the Mac. I have used Darwin Ports in the past to quickly create a UNIX environment I am accustom to without having to worry about dependencies and compatibility.

Java

Mac OS 10.4 comes with JDK 1.3 through JDK 1.5. Swing components are beautiful, all the standard tools such as Ant and Javadoc are bundled, and documentation packs are free to download on Apple's developer site.

X11

One of the things that surprised me the most when I purchased a Mac was how painless it was to get X11 set up. Although I was slightly disappointed that it was not pre-installed, Apple did provide the X11 package on the companion CD. After a few clicks, X11 was automatically configured and running properly. X11 is another essential programs that I have grown accustom to. By installing X11, an entirely new category of applications can run on a Mac.

VPN

I have never met a VPN I couldn't talk to. I am able to connect to my corporate VPN via PPTP on the Mac at least as easily as on a Windows PC. My previous employer used SSH tunneling as a point-to-point VPN. No problem on a Mac, OpenSSH is installed by default.

Windows Remote Desktop

Microsoft provides a free version of remote desktop for the Mac so it is easy to work from home or access a Windows PC. One of the nice things this allows me to do at work is to hook up a second monitor and connect to my Windows PC via Remote Desktop. By doing this, I have a duel head Mac with a connection to a Windows box that can be controlled by a single keyboard and mouse.

IPFW

I mention this specifically because it helped solve a particularly difficult problem. In a nutshell, we were attempting to improve the performance of our primary product. Our thought was that network latency was causing connection delays when the application's threads were entering a synchronized block. We ended up configuring ipfw on my Mac to introduce controlled network latency that made it possible to test and eventually correct the synchronization problem.


* * *

In conclusion, I assert that there are three factors that work against the Mac as a developer platform:

  1. Until OS X, the Mac was limited when it came to development. For me, limited software availability made Macs an ugly alternative to Windows or UNIX. I believe that Macs are still haunted by the stigma they carry from the pre-OS X days.
  2. When Apple switched to Intel processors it opened the possibility of running Windows in a non-emulated environment. Windows runs well on Intel Macs, Apple even provides Windows drivers for all their hardware. However before this recent move to Intel, Macs used PowerPC processors which would likely have caused unfavorable compatibility issues in the eyes of software developers.
  3. Price. Macs are about as expensive as high-end PCs. This is their biggest negative in my opinion. Since OS X is UNIX-based, a Mac platform is more attractive to open source developers. However, many open source developers may not want to pay for such an expensive machine.

Saturday, September 8, 2007

Higher Education In The Software Industry

After finishing graduate school I felt quite eager and prepared to enter the work force. For the most part I was very far beyond many of my classmates in many academic disciplines except when it came to real world experience. At the time, I never gave much thought as to where a graduate-level degree would take me. Having been out of school for some time, I believe the purpose of a college level education is a combination of items in the list below:

To provide a way for individuals at a novice skill level to break into the software industry.

Software is a relatively new discipline that under most circumstances does not require any special certification to practice. Unlike in civil engineering where by law formal certification must be obtained before signing off on the design of a new structure.

A self-motivated person may educate themselves at a local bookstore and has the potential to be just as successful as the college graduate. However, for a true software novice who may not know where to begin or what to focus on, time may be better spent in a program designed to start at the beginning.

Reference: check out SWEBOK for a collection of disciplines a software engineer can be proficient in.


To make yourself a more attractive hire by reducing the risk a future employer takes when it is not ideal for them to pay more for an experienced candidate.

My former boss and chair of the computer science department once said to me that getting a Ph.D. is a lot like graduate school, except it shows that you can survive. Although many graduates leave school with little work experience, by earning a degree you have proof that there exists various qualities within you to:

  1. Personal responsibility
  2. The will to finish things
  3. An interest in a given discipline
  4. Forward thinking
This gives an employer a bargain opportunity when hiring. While a person with five years professional experience would almost certainly be more proficient than a recent graduate, the employer would also need to pay for that proficiency with a higher salary. By hiring a recent graduate the employer can offer a lower salary at the cost of taking on some risk. This is why a high GPA is crucial, it is an indicator that you represent a reduced risk and are therefore a more attractive hire.

To produce better research.

This is especially true at the graduate level. I was not training to develop great software systems, I was training to produce research on topics related to software. I concede that not everyone studying computer science wants to be a software developer. There are those who want to be on the cutting edge of some of esoteric technologies. Higher education trains you to do produce research.

To pursue a professional career in academia.

The next time you visit a college make note of faculty who have earned their Ph.D. If they look like they are in their late twenties to early thirties, chances are their primary chosen professions is a career academic. Obviously higher education is a requirement for a career academic.

I was close to choosing a career in academia. Almost immediately after graduation I was teaching many of the courses that I had to take only a few short years ago. For financial reasons I decided not to pursue the academic path at this time in my life. For better or worse there is a gap between academia and the business world. I was mostly ignorant to this fact and wore a wet blanket for a few months after graduation. Unpleasant as it was at the time, I now appreciate the fact that I have insights into both worlds.

To participate in education for its own sake.

I never quite understood this one but nevertheless I saw individuals accumulating degrees without any apparent reason.

To have a social experience.

I believe this is a symptom of either immaturity or short-sightedness. Regardless, there are those that decide to participate in higher education because of the social opportunities associated.

Tuesday, September 4, 2007

Goals for Today

  1. Appreciate God
  2. Commit to learning something of value
  3. Be honest
  4. Handle a situation better than I would have yesterday
  5. Be transparent
  6. Experience a paradigm shift
  7. Don't break anything
  8. Think about new goals for tomorrow
Goals for tomorrow:
  1. Appreciate God
  2. Commit to learning something of value
  3. Be honest
  4. Handle a situation better than I would have yesterday
  5. Be transparent
  6. Experience a paradigm shift
  7. Don't break anything
  8. Ask someone to explain something I don't understand
  9. Think about new goals for tomorrow