A Bad Call

In an earlier post, I referred to the thankfully few instances of poor estimation by a Quoin project team. Since examples of what not to do can be as important as how to correctly estimate development work, I wanted to share some thoughts on a particular project, with the names changed to protect the guilty. 

The project started with a call from the corporate counsel, which perhaps was a sign that any project might face some challenges. He explained that his company, which we will call ‘AP,’ was a well-established consulting and training firm that wanted to create a new line of business, and had engaged a firm to design and build its first software product. This firm had worked with them in the past, but AP had grown increasingly concerned that this vendor was out of its technical depth. Could Quoin conduct a review of the application under development, help get the project back on track, or even take over development, if necessary?

Photo by Steve Johnson on Unsplash

We spent several weeks reviewing the in-progress code and meeting with the current development team, and quickly understood that the client should be concerned about the project. The code, what little there was, was incomplete and amateurish. We even had to go through a scheduled beta test with the vendor in which the application simply did not function. We later found that many of the scripted beta tests were hard-coded. Soon after this debacle, Quoin agreed to attempt a rescue of the project. 

In our work, clients always want to understand schedule and cost, but I shared this background to emphasize the necessity for AP. The failed implementation had already taken a year, and the company needed to deliver this promised application to its clients as soon as possible. We worked out an initial schedule of 6 months, but recast this with the client after the first few weeks of re-engineering to 10 months. What followed was the type of project all software engineers seem to experience at least once in their careers -- a 14-month grind, 6+ days a week for long periods of time, and an increasingly frustrated client. Since I made the decision to take on the project and helped create the 6-month and 10-month schedules, I was often there with the team, home at 10 pm to a justifiably irritated spouse and a pre-school daughter long since asleep. So, how did this happen?

  1. Failure to deliver incrementally – The fundamental issue is that we accepted a project where the client’s only measure of success was a complete product delivered in a single release. We were experienced in the agile concepts of iterative development and incremental delivery, and structured our work to follow this approach. Despite making progress throughout development, the goal as far as the client was concerned remained the full application – the overall schedule with every promised feature - and this inevitably led to problems. 
  2. Failure to adapt – A key aspect of agile is to respond as a team understands more about the domain and its implementation. We had numerous opportunities to simplify the application throughout development, but remained focused on the initial feature set. One example was building both a desktop and an enterprise version of the application in parallel despite strong indications from clients that the enterprise application was not the right target. The team also spent an extraordinary amount of time and effort on a search feature that was beyond the needs of the vast majority of end users. What we built was a masterpiece of search logic, supporting full text, keywords, synonyms, operators including esoteric ‘Nor’ and ‘Nand’, and other capabilities. Our stakeholders really liked this capability, but their users just wanted a Google-style text search.
  3. Failure to start new – We considered whether to jettison the half-baked application from the first attempt at development, but ultimately tried to refactor and reengineer this code. Although sometimes keeping the existing code base is entirely the right decision for a project, in this case it turned out to be a serious mistake. We should have abandoned the poorly written code and perhaps even changed technology stacks. Our team spent too much time and effort working with an unstable application, and we ended up re-writing it anyway.
  4. Failure as a team – Finally, not to evade any well-earned blame, I’ll just say that managing people is never easy. We promoted the wrong developer to technical lead, failed to spot the warning signs, suffered staff attrition as two engineers showed the good sense to leap from the train, no matter the landing. I look back on this project now, several years later, and see it as both extraordinarily hard at a personal level, but also a real period of maturation for Quoin. I believe in lessons learned, and this project gave us a few very sharp lessons.

These failures really relate to the overall project and estimation, which is fundamentally difficult. One of the basic tenets of agile is to work in a series of sprints, and similar to any project team our sprint planning and execution improved after the first month or so of work. We had good velocity and few broken sprints. In some ways, this example shows how important sprints are for iterative development and incremental delivery. Yet, we just had no reliable way to know how many sprints we would need, and due to poor decision making (note the irony given the client), we had limited flexibility to reduce scope or simplify the application. Eventually, these changes were forced on us and the client, but that is no way to run an effective development process. Perhaps, I should not have taken the phone call.