What is the biggest concern of studio heads and producers? Labor pool and finding good talent. But after that, usually schedule is the most important. This is strongly related to budget and engineering uncertainty. Basically, having decent estimates, even in an agile schedule, is essential.
But squeezing good estimates out of programmers is difficult. The Dunning-Kruger effect almost guarantees that programmers will overestimate their ability to give good time estimates. Those who are good will give +50% estimates (150% of a good guess) and those who are terrible will always underestimate the tasks in order to "get the job". What should a producer or scrum master do?
Well, if you have any historical knowledge of the programmers involved, then you can use this as a tool for guiding future estimates by programmers with less-than-perfect scheduling history. Also, a word or two before every sprint planning session ( or work estimate bidding) can work in your favor to clear the air and remove any pressure to "get the job" by underestimating. Lastly, breaking tasks into simpler tasks that are achievable and demonstrable is key. I will cover all of these in some detail below.
Firstly, estimation is a difficult thing and engineers have historically had dubious results. There is an entire wiki page dedicated to this topic. http://en.wikipedia.org/wiki/Software_development_effort_estimation
Quoting from that page, this captures the essence of the problem.
- It's easy to estimate what you know.
- It's hard to estimate what you know you don't know.
- It's very hard to estimate things that you don't know you don't know.
Secondly, estimation can be overdone spending entirely too long mitigating possible risk and not enough time working. For software and engineers who have a good track record of estimation, you can simplify your effort. Take that programmer's best estimate and multiply by 150%... the additional 50% will cover nearly all overages. You will be wrong on occasion, but you will usually be under schedule and finish early. Then both you and the programmer will look good historically. Another way of putting this is that people will trust you and your estimates. For all estimates, always assume a 6 hour work day.
But for those who have a track record of badly estimating, make sure to involve other engineers, ask hard questions (covered later), and then double that estimate. Those who are bad at estimating are also often bad with distractions and time, so they may take longer. If they demonstrate that they are better than the double-estimate over time, then you can reduce that number to 150% and maybe remove the need for the opinion of other experienced programmers.
Thirdly, estimating big tasks without decomposing them is impossible. Without decomposing the problem, saying that it will take 18 months to build a car from scratch is at best a wild guess (and wrong) and at worst, if you've made promises to deliver, likely to cause you a lot of pressure and overtime. Do yourself a favor, and make sure that all time estimates are under 5 days, and ideally, 3 days and under.
Asking the hard questions:
People tend to shy away from the hard questions and they stick to "how long will it take?" This question, by itself, is not quite useless... but it's not very helpful. For any given task, you should be able to ask at least the following questions:
Is this your expertise?
Do you need to look anything up or can you get to work immediately?
Do you have things that will prevent you from working like subsystems that are undone, existing bugs, etc?
Is there any integration with other systems? Do you know which systems those are?
Who else needs to help you?
Can you work in an isolated sandbox? If not, how likely are you to break the build?
Of any of the other people that you need to contact, will those people be available? Vacations? Sick time?
When you commit your code, will it be done or will there be multiple checkins?
Assuming that you are distracted by other things, how much time could this take?
When you have an engineer who historically is bad with estimation, or maybe you are new to a project and have no history on the people involved, you should use this handy guide for guessing the amount of work. Notable is that all of these estimates are high. We are not interested in exact estimates and estimates that are too low do us all a disservice. We shoot a little high until we have a better sense of the estimation ability of the engineers involved.
Given that producers generally have little insight into software development tasks, use this as an estimation guide:
- Small task: 1 day
- "one hour task": 3 hours
- Easy task: 2 days
- Medium task: 3-5 days, when in doubt, use 5
- Large task: don't bother to estimate. Break it up into smaller deliverables that can be estimated at less than 5 days.
Now add the following for additional unknowns:
- Working with an external library + 5 days
- Working with a previously unused external library + 10 days
- Integration into a memory system + 5 days
- Integration into an existing graphics system + 5 days
- Integration into a 3D graphics system + 10-15 days
- New tool chain + 5 days
- New complex tool chain + 15 days
- Conversion of code to another language + 3 weeks minimum
So, those rough guidelines should give you an idea on the real world numbers to use for estimating tasks. Of course, given expertise in a particular area and an expert engineer, these numbers can be reduced... by about 50%
Dealing with uncertainty
Programmers are usually sure of themselves. Even the very experienced, good ones, are not good at telling you when they are not sure of something. This leaves product owners and scrum masters in a bit of a dilemma. The bigger issue is that designing a reasonable schedule and providing good estimates becomes a crap shoot when everyone offers wildly optimistic or pessimistic viewpoints on the effort required.
This is where the effort bidding comes in. During sprint planning, developers are supposed to look at feature and estimate effort. This should include programmers, QA, and possibly producers. Unfortunately, this is the part that programmers hate the most, and is the most useful tool in decomposition and effort assessment for scrum masters (producers). I believe that the reasons that programmers hate it so much is: it's time consuming, it feels like random guessing averaged, programmers are forced to justify their positions, and it's something that is not like programming.
Whatever the reasons, tearing apart a feature into tasks and then into subtasks, assigning those, adding QA tasks, identifying dependencies, and building a schedule is the most sane way to fix the schedule and increase software development processes and reduce effort and rework due to unforeseen dependencies.
What to do when you get it wrong
You will get it wrong. Occasionally, a task will be misestimated and you will overestimate the effort. It is rare that you overestimate, but make sure that if the teams does, that you call them out, try to understand what happened, and acknowledge the great work of the engineer; this is usually appreciated.
But managing perceptions is what you do before you get it wrong... talked about below. Once you've missed, the following steps should be taken.
- Make the hard choice if the work should be continued, reassigned, or dropped altogether. This is usually based on the importance of the feature.
- Estimate the work remaining, if possible.
- Discover a mitigation strategy. e.g. Bringing in a domain expert for a consult.
- Put that task into the backlog.
- Wait until the next sprint planning session to try again.
Your last thing that you should do is inform the major stakeholders as soon as you can. Sending an email is helpful, or even a quick meeting to verify that your mitigation strategy is correct, and that the feature is still important.
Managing perceptions
This may be the trickiest part. Overestimating the length of feature development may mean that the feature is cut because it appears too costly. Underestimating it may also mean the same thing since it is likely that the feature will not work correctly and much later put the project at risk.
So, describing the risk of a feature is tough. This becomes much easier if the task is broken down into much more manageable pieces given that only a few of the pieces are likely to be over and with enough slack in the schedule, may not affect the overall delivery schedule at all. Still, overages do occur and these need to be managed in order to reduce pressure on engineers (leading to mistakes) and to help mgmt understand what a great job everyone is doing which leads to perceived success and company happiness. In addition, a well-managed schedule has enough slack that people have enough free-thinking time to properly evaluate engineering needs, pursue alternate solutions, and debug difficult problems.
Also, with enough smaller tasks used to describe a large feature, an overage is much easier to explain and to accept from your boss when a 3-day task surprisingly takes 5 days instead.
Even with all of these pieces, it is a good idea to identify and call out features that are large and inherently riskier. If you have a feature that breaks down into 20 tasks, then it is likely that some tasks were missed, some tasks will run over, and the feature will be delayed. Other things like engineer fatigue, sick days, and surprise tasks will cause changes to the schedule and further delay the delivery of larger features. Calling these riskier features out to mgmt during alignment meetings, casual talks over the coffee machine, and hallway conversations keep other managers aware. Also, telling the engineers involved during sprint planning meetings shows a level of understanding, maturity, and mgmt expertise creating a more collaborative atmosphere when attempting to work with major unknowns.
Another strategy is colouring tasks based on engineer assessment. A simple question like "what is the chance that this might be over you 3-day estimate that you gave on a scale of 1 to 3... 1 being none and 3 being likely." This is a simple scale, it doesn't take much thought, and allows you to color code tasks in your backlog. You can also adjust your turndown, or allow more time for riskier sprints. This kind of communication takes a little getting used to, but allows everyone to know that somethings are harder to know up front and may be delayed.
Possible conversation:
Bill: "So how long will it take to integrate that 3D library?"
Steve: "about 2 weeks."
Bill: "Alright, let's break this up. We want a lot of smaller task to help us deliver stuff so that mgmt will be happy with our progress and to communicate better with the team. What are the first steps?"
Steve: "Well, I have to look over the docs. That will take a few hours."
Bill: "That usually takes a long time... let me give you a day. You don't need to use the whole day to do that, bu you can use a few hours here and there as you need. Consider it an open tab at a bar. Then what's next?"
Steve: "Then I need to find the files and put them into our project and start integrating."
Bill: "Whoa... too many 'ands'. Let's break that up. How long to find the files?"
Steve: "an hour or two... I need to find the download site and make sure that I have the right version, pull it down, etc."
Bill: "Actually, that sounds pretty easy... is 3 hours about right?"
Steve: "That's way too much.. I'll only only need about 1 1/2 hours. Tops."
Bill: "Nice, but what if there are Linux and Windows versions of the libs? What if some compile only in 64 bit? What if the library requires an older version of Visual Studio? Let's leave it at 3 hours, and if you don't use it all, then we have given you a little slack. On a big task like this, you will probably need extra wiggle room, not less."
Bill: "... so then you said that putting the files into the project was next... that means adding those to Perforce too, right?"
Steve: "Yeah... but I should be able to dump the files in and just compile. That should only take a few minutes."
Bill: "Doesn't this need to compile in Linux too?"
Steve: "Yeah... so I'll need to update the make file on the Linux side and the project in Visual Studio... still that's pretty easy."
Bill: "Aren't there usually compile issues when you first integrate a new library? Then include paths because those libs may not know where boost is and other kinds of issues? And you still need to get that into Perforce.... let's give you 3 hours for that. We're up to two days before the integration truly starts."
Steve: "That seems too high."
Bill: "Well, if you finish the full integration early, then you can work on other stuff that you think that we need to do, but you never have time for. Weren't you going to cleanup that Lua library? If you finish early, you can do that."
...
No comments:
Post a Comment