Avoiding Pain in Software Estimation: Part 2 – The Devil’s in the Details

Some well intentioned project planners, in an attempt to provide the most accurate estimate possible, will begin by breaking down the project into small tasks no larger than about 3 days, dump all these tasks into a large project plan, assign resources, identify dependencies, resolve conflicts, and boil the entire project down to a specific completion date which is, let’s say, six months from now. This, in actuality, is a very risky approach to software estimation because it completely ignores the concept of visibility.  A project planner really has very little visibility into what the project is going to look like, the issues that are going to be faced, the roadblocks that will be encountered, the compromises that will be made two, four or six months out. The effort that goes into creating a project plan to this level of detail is, just frankly, wasted effort. Some would argue that putting so much detail into planning early on in the project is just a bureaucratic exercise and is worse than not planning at all.

Worse yet, when so much effort is put into a plan prematurely, there is an increased likelihood the plan is never revisited or revised. This is because, as reality sets in, project planners have to decide if they need to force-fit the project’s reality into the plan, or revise the plan to reflect reality. The latter is usually the most time consuming option for the planner and is therefore discarded. And it’s typically difficult (or impossible?) to force-fit anything into such a rigidly structured and detailed plan. So that option is also discarded leaving the plan to collect dust on a shelf somewhere.

So, the answer is to treat the plan at a very high, macro-level early on in the project. Micro-level planning should only be done for a few weeks at a time. Using Agile/Scrum terminology, each Sprint has planning time where tasks (or Stories) are selected for inclusion in the upcoming Sprint. Resources are identified and development activities begin. Now the planners (or Scrum Masters) only have to manage the details (removing roadblocks, reallocation of resources, etc) for the activities surrounding a single Sprint.