There's been a few posts recently regarding estimation on agile projects, all expressing issue with the way it is most commonly approached. A common theme to these posts seems to be a slight under-appreciation for the multiple purposes of story estimation.
I consider there to be 3 roles for estimation, which can be categorized with 3 Ps as follows:
Prediction (or Planning)
The most commonly observed purpose of estimation is for prediction - trying to determine at what time a certain amount of work will be completed, or what amount of work will be completed at a given time. With an estimate on each story in the backlog and an understanding of the rate of completion of those stories (velocity), either of these can be predicted with an accuracy equal to the accuracy of the estimates. Note that the accuracy tends to rapidly decrease as the amount of work or the timeframe gets larger.
Prioritisation
Prioritisation can be considered as the task of scheduling work by decreasing value. So when prioritising stories, the one with the highest value is scheduled first, followed by the next, and so on. This of course begs the question of how we determine the value.
I've blogged this topic previously, but I consider that value is essentially determined by a comparing benefit and cost. The story describes the benefit and an estimate on it can be used as a measure of the 'cost'. Thus estimation is essential for prioritisation.
Note that for prioritisation it doesn't matter what sort of estimation metric is used. It will still be treated as relative, as the prioritisation is about the relative value of each story.
Performance
Estimation is also required to measure changes in the performance of a team. The throughput (velocity) of a team is determined by the amount of 'work' being done over a given time-period and is measured in terms of the estimate. When changes are made to the team or the environment, the previous and new throughput can be compared to see if an increase (or decrease) in performance has been observed.
Wednesday, July 9, 2008
Subscribe to:
Post Comments (Atom)
6 comments:
Since I wrote two of those posts y ou mentioned, I'd like to clarify my view.
First of all, I love story oriented project management versus the traditional task oriented. Both have their merits. It is top down and very easy to comprehend and fun to move story cards around for the faster and more visual feedback of accomplishment, which is an important aspect of human factor in development. Of course, the story board also serves as a great communication tool. Task oriented approach isn't totally useless. And story orient approach does not need to be mutually exclusive to the task oriented approach.
Secondly, what units to use does matter when it comes to prioritization. And while preserving the relativity, we can still make the benefit and cost measures more concrete. For instance, cost in a software project is usually man hours. As explained in my posts (agilesanity.blogspot.com), it is beneficial to use man hours for effort management if done properly. As far as value goes, I have also explained a few ways to assign dollar values to the outcome. Granted that this is only ball park, but it will make it easier to evalue priority because cost( man hours ) can also be measured in dollars.
Lastly, the emphasis on performance is not just to take note on variations. I'm sure you were aware of it, maybe just not expressing it fully. The real gain is process fine tuning to improve performance.
Thanks for the reply!
I still stand by the assertion that the unit of estimation is irrelevant for prioritisation, only the relativity. If you have two stories that provide equivalent benefit, but one costs 2 man days and the other 1 man day, then the second is more valuable. Not that this still holds if I say 2 story points vs 1 story point. The actual measure is largely irrelevant.
This holds true even for stories of differing benefit. To demonstrate:
An important point is that development time (man hours, etc) are NOT fully variable costs on a software project. They are an operational expense that is incurred to have a development capacity with a certain level of throughput. Given that the development capacity is typically the constraint in the system, it is important to maximise the benefit obtained for a given constraint time. So the priority should be the story with the highest benefit per constraint time, compared to other stories. Assuming an estimate in a fixed measure, like man hours, this is roughly:
benefit / estimate
Comparison is straight forward:
benefit1/estimate1 <=> benefit2/estimate2
Now if we assume the estimate is actually relative, like story points, then the actual constraint time can be obtained by dividing by the velocity:
benefit1/(estimate1/velocity) <=> benefit2/(estimate2/velocity)
Reordered, this looks like:
velocity * benefit1/estimate1 <=> velocity * benefit2/estimate2
Cancelling both sides, we end up back at
benefit1/estimate1 <=> benefit2/estimate2
QED.
I do agree that it matters when you are attempting to calculate ROI. But that is a subtly different concept from value, which I intend to blog about in the near future. It is also not required to know ROI in order to prioritise, as I have shown.
I also agree that the process fine tuning is an important goal in any system. But to enable this requires a way to measure performance variation when changes are made, and this is what I was alluding to.
There seems to be 2 missing links.
(1) For example, two features A and B, with the following (benefit,cost) points (4,2) and (8,4) are equivalent in your formulas. However, for scheduling and collaboration purposes, actual man hours matters. And for budgetting and cashflow purposes, actual cost in dollars matters. And they affect planning decisions.
(2) Using concrete units also assist clearer communication, and cross projects/initiatives planning.
1) "for budgetting and cashflow purposes, actual cost in dollars matters"
Absolutely. This is why you establish a performance metric for the team based on the rate of consumption of the abstract relative estimates. Otherwise known as velocity. Using this, you can attempt to Predict when work will be delivered by the team and then budget for that. Over time, you will get a better idea of velocity and you can also narrow the accuracy of the relative estimate, and thus you can get a better idea of actual costs. Attempting to Predict actual costs with any precision at the time of initial estimation is foolhardy, considering the amount of uncertainty in any software delivery project.
Have you actually ever done what you suggest with success?
As for 2) I think we'll have to agree to disagree. In my experience, the clarity real numbers gives is deceiving, as it asserts a feeling of certainty where there is none. Software development is not a precisely predictable activity. We have to deal with uncertainty explicitly, and using relative metrics and then tracking actual progress helps to do this.
Let's say the deadline is 5 days from now. And how should we slot certain story if we just use the ratios you suggested?
Let's say the company has a cashflow of 50k that month. If we just use ratio, can we take on a project which require 5 additional developers to help out in order to complete the task? The ratio is insufficient.
Also, I don't believe adding indirection or abstraction can solve the problem. And the real point is that we don't need the abstraction. We only need stakeholders to know what the number means. I am sure that if something got repeated 100 times, it will register. If not, then it is a different matter. After all, stakeholders can still question you regarding cost, benefit, prioritisation, budgeting, and timeline. And I am not the only one thinking along this line. Take Mike Cohn, and Kent Beck for example. They have reverted to the use of ideal developer days for good reasons.
I have done project management for about 5 years, and there were 2 projects i did something like that. One of the project was going so smoothly that such measures no longer needed because we were delivering ahead of schedule under budget. The other project need such analysis to sell to investor.
Sorry Charles - your comment makes very little sense (to me).
I'm not sure what you mean by 'slot' a story in a 5 day deadline, nor the 50k company example. If you have 5 days until you have to stop work then you can get your team to work on whatever is your highest priority until the 5 days is up. If you want to try and work out if a story will fit into that short timeframe just ask a developer if they think they can do it. Maybe this is what you mean? I think, however, this only applies if the developer is immediately going to start work on the story.
I'd also suggest that if repeating something 100 times is never a good approach to making someone understand. I find they shut off after about 3. If you think it takes 100, they you need to reconsider if it's really the best approach to explaining something. Personally, I find ideal days are very confusing for long term planning, as they have nothing to do with 'days' and are just an irrelevant distraction.
As for Mike Cohn "reverting" to story points - you might want to check this. As far as I know, he has only suggested that he would use ideal days instead for his 'sprint backlog', ie. the stories in the immediate view of the team, less than 1 iterations worth. Much like my point above. And certainly not for planning beyond that. This is a very different thing from what we are discussing. I don't disagree with him at all - if I'm trying to determine if I can finish a particular piece of work in the next few days, then a time based metric makes perfect sense. Of course, this is only a good thing to do when your trying to plan how much to put into your immediate 'sprint/iteration backlog', and since I'm in favour of removing these entirely and favouring a more lean 'pull' approach, it's kind of a moot point to me.
Post a Comment