"How to estimate software development time” is a perennial question that faces us all at some point in our career. How to manage software projects is a complex problem, there is a lot to understand, and there is a lot of misunderstanding about what is sensible and possible in software estimation. So in this episode, Dave Farley explores the ideas of estimating software, or not: the #NoEstimates movement say don’t estimate at all, and many successful teams work that way.
Software project management would naturally like to know how long something will take, and it is a reasonable question, but one that is very difficult to answer with any degree of accuracy. So whether you are a developer being asked to estimate next week's work, a team leader looking to figure out when the next release will be, or someone trying to sell a project to someone else, what do we know about what works in estimation, and what doesn’t?
-------------------------------------------------------------------------------------
🎓 CD TRAINING COURSES 🎓
If you want to learn Continuous Delivery and DevOps skills, check out Dave Farley's courses ➡️ bit.ly/DFTraining
📚 BOOKS:
📖 Dave’s NEW BOOK "Modern Software Engineering" is now available on
amazon ➡️ amzn.to/3DwdwT3
In this book, Dave brings together his ideas and proven techniques to describe a durable, coherent and foundational approach to effective software development, for programmers, managers and technical leads, at all levels of experience.
📖 "Continuous Delivery Pipelines" by Dave Farley
paperback ➡️ amzn.to/3gIULlA
ebook version ➡️ leanpub.com/cd-pipelines
📖 The original "Continuous Delivery" book by Dave Farley and Jez Humble
➡️ amzn.to/2WxRYmx
📖 "Rapid Development: Taming Wild Software Schedules", by Steve McConnell
➡️ amzn.to/38OKgtP
📧 JOIN CD MAIL LIST 📧
Keep up to date with the latest discussions, free "How To..." guides, events and online courses.
➡️ bit.ly/MailListCD
-------------------------------------------------------------------------------------
CHANNEL SPONSORS:
Equal Experts is a product software development consultancy with a network of over 1,000 experienced technology consultants globally. They increase the pace of innovation by using modern software engineering practices that embrace Continuous Delivery, Security, and Operability from the outset ➡️ www.equalexperts.com/
Harness helps engineers and developers simplify and scale CI/CD, Feature Flags and Cloud Cost Management with an AI-powered platform for software delivery. ➡️ bit.ly/3Cfx3qI
My Tips on How to Start a New Project in a way that Improves Your Chances of Success, in this FREE Guide ➡ www.subscribepage.com/new-project-guide
There was a "Blondie" cartoon I saw in the newspaper years ago and didn't save. Dagwood: How long will it take you to paint these rooms? Painter: Two weeks. Dagwood: I could do it myself in one week! Painter: Are you going to do it? Dagwood: No. Painter: Well if I weren't going to do it, I could do it in three days!
I can do it in two! (as long as I don't have to)🤣
3 days times 4 is about 2 weeks, that seems like an accurate estimation!
@@cavorkehl6777 My current proj took 5 months, and possibly need another 2 to finish. Initially I thought it wouldda took 1-2 lol. This is surprisingly accurate!
@@twinkytwinklier4047 Currently on week 4 of a “2-week sprint”. One of our (genuinely) genius mentors that I went to for help completely overhauled the approach, then had to bring me up to speed which put a bomb under everything time-wise. The point is that something like this, something completely unpredictable, seems to happen almost every time!
Awesome Quote!!!
I once had a manager who insisted on coming to my desk with an idea for a feature or something for which a client had asked. From the very first time they did that I told them I couldn't even give a meaningful guess (and by meaningful I mean something that is not just obviously ludicrous) without more information about the idea or feature. That didn't stop them pushing me then and there for an estimate (with the false promise that they wouldn't hold me to whatever I said). Nor did it stop them coming to me and doing it again. What eventually worked was that I started responding loudly with, "One million years!". It still took three or four times, causing everyone in the office to look up and wonder what was going on each time I belted it out. I understand that managers want to know so that they can budget. And project managers want to know so that they can allocate resources. What they fail to understand is that needing to know something doesn't make it possible for the knowledge to be made available. And then, after agreeing that the estimates we give are just guesses and will always have significant margins of error, they get upset if software isn't delivered at the time given in the estimate excluding any margin of error. The only time I worked anywhere that had anything approaching accuracy in estimating was one where the task of estimating was so involved that what we did was go through the code, figure out exactly what needed to be done, down to which lines of code needed to be changed or added, what tests would need to be written and anything else that might be needed. We'd sometimes spend days coming up with estimates, coming fairly close to doing the actual work to come up with an estimate for the effort required to do the work. That company continues to be very successful in their part of the industry, but I think that it's more because the charge a lot for their software (which is very good, to be fair) than because their approach to software is good.
Can i say i have a word to word situation as well. This is not what Agile rulebook says about what to do in this scenario. Because it is agile, it is expected that we can just do it on the go.
"We should be, by definition, always be working on new problems." - I'm always amazed at how this confuses managers and developers. They think their job is to crank out the same thing over and over again. To which I reply, "We have these devices that are very, very good at repetitive tasks. I think they're called Computers? If a task is repetitive, a computer should be doing it."
Right. If they want to have someone do the same coding done in many different applications, let the developer work on making proper and complete solutions and libraries and hire some kids right out of highschool to work under them to do the repetitive grunt work.
@@MrNathanstenzel If you’re doing it right, grunt work doesn’t exist. Or at the very least, it dries up very fast.
@@thewiirocks regardless, having new workers do the grunt work and then get assigned more difficult tasks will let the main programmers focus. They can recieve training or self train if there is not much for them to do.
@@MrNathanstenzel It is really, really, REALLY hard to assign out grunt work that doesn't exist. If you're doing it right - GRUNT WORK DOES NOT EXIST. It gets automated out of existence very rapidly. Self training is nice. Directed training is better. Paired programming in a highly effective team is unstoppable.
@@thewiirocks you must have worked for a place that allowed proper programming instead of essentially forcing you to do the same thing over and over in a copy and paste fashion. Doing things right takes more time than doing it stupid and quick.
Rule of thumb I leaned once: Take your best guess. Double it. Go to next higher unit of time. Seems to work out right for my home improvement projects.
1. Take your best guess. 2. Double it. 3. Go to the next highest unit of time. 4. Lose the job to some other company who will just do step 1 and then figure out the best ways to tell the client they're sorry.
You forgot to add a zero at the end and then square the result
Then some new guy shows up, writes it in an afternoon, and makes everyone look bad.
"It is difficult to make predictions, especially about the future." (Various)
I ran a dev-ops shop at ATT for 4 years with 150+ devs in four cities. Dave, I wish I could have played this video to the stakeholders every Monday morning. Damn sure would have made them sign an awk that they "viewed and understood" the content of this video.
1) Software engineers tend to give estimates based off of an optimistic idea on how long it will take to implement something. They don't tend to factor in unanticipated problems and what it'll cost to overcome them. It's these unforeseen problems which throw estimates off. 2) Not all software engineers are equal with levels of proficiency and talent. If you take your standard developer and assign them a productivity value of 1.0 as the standard baseline, then the range of developers will be anywhere from -10.0x to 10.0x. You might ask, "How is it possible to be negative in productivity?". Some "developers" are going to cost other developers time as they need to have someone hold their hand through development, or they introduce so much sloppy work that it costs another developer significant time to refactor it. Always factor in the talents and abilities of the people working on the project. 3) I tend to look at features and try to assess if its going to take days, weeks, months, or years to implement. These are the orders of magnitude, and if you look at the problem as variations in orders of magnitude and each order of magnitude is treated as a standard deviation, then the true time cost is going to be within one order of magnitude of the estimate (with a bias towards a pessimistic estimate). If you think its going to take 1 week, it'll really take anywhere from a few days to a month to implement. 4) The other factor that's often overlooked is the organization / company which is implementing the software development. I worked at a big tech company recently and I was finding that 75% of my day was spent attending meetings. The other 25% was for actual development. I was amazed that anyone could ever get anything done. The larger your team is, and the more surface area you get with feature exposure, the amount of time spent on new development seems to decrease exponentially to an asymptotic zero. So, estimates can't be done in a vacuum because organizational dynamics will affect productivity.
Well said and quite true.
Well said sir. The first point is the most crucial. Even if it’s a single line of code change, you never know what kind of errors are you going to cause because of it. Estimating anything is pure delusion.
Do you charge for estimates ?
I have found that in case 4) most of the work gets done by those people on not paid overtime. Run.
I was often told that, as I gain experience, my estimates will get more and more accurate. Well, it's been decades now. I guess they are a little more accurate, mostly because the design I think up initially is closer to what gets implemented in the end, but I still consider myself lucky if I'm within a factor of two. Getting within plus or minus 10 percent, even in an "ivory tower" situation, is still way beyond me. I can't possibly speak for everyone but I suspect what managers are actually seeing when they see estimates become more accurate is that the people doing the reporting are learning to lie more effectively. They think project A will take 10 hours and project B will take 20, so they tell you A will take 35 hours and B will take 50. Project A actually takes only 15 hours but they still tell you it took 35 because B ended up taking 100. The other 15 hours came from working over the weekend without telling you.
That is the truth and the most hated aspect of it.
The truth shall set you free. But first it’s gonna piss you off.
...because the software you're developing has grown in complexity over the same period of time. You're constantly estimating what you've not done before. Your experience does help but I believe most have the same problem.
Nailed it
Yep, ask extra time on simple tasks to get time on complex things
Manager:”how long?”. Me:”I don’t work with deadlines”. Manager:”then maybe you should not be working here”. Me:”way ahead of you”.
I laughed way to hard about that xD
🤣
"When its ready"
some serious Dilbert vibe here
Nice.
The "throwing estimates" routine is actually an informal form of the Delphi method. This is one of the best ways to get a forecast from a group of experts.
I recentlu found this channel and just spent this weekend to watch as many videis as I can! Being in the field for more than 20+ years already thought me about 90% of the things, though now is the time to say a HUGE thanks for validating most of the ideas I came up with and dismissing others. Every video I watched was to the point, presented with great examples and explained in a "as simple as possible" way. Thanks again here, for being a great example on how this is done! I strongly believe that everyone in our industry should benefit from the knowledge base here. Further more every C-level person(or maybe everyone in the industry) should sit back, take their time and listen closely. I do believe the content here could be a game changer and a reason for success. Thanks! (again and again)
Thank you for the very kind feedback 😁😎
One of the most valueable video i have watched in youtube so far. Love this guy.
Wow I have spent around 3 hours watching this video, reading some references in the video, by making my ultimate notes. I think it's worth it because, from now onwards I'm sure I am not going to feel extremely overwhelmed after watching my JIRA dashboard every week.
Great video, I tend to work more on the infrastructure side of a software company so it is always important at the very first conversation to get as much info as possible so I can hold them to it without "scope creep" becoming an issue. It becomes a fine art after a while. I also have that t-shirt!
I've given up estimating as the specifications changes as we progress - so it doesn't really make much sense to spend time up front. Trust is key - just work and get things done.
I always think the question is what is the value in trying to be accurate up front by spending time figuring it out vs actually doing it
Exactly. But business-wants-estimates-to-calculate-budget. I guess it's a major reason why we still doing this crap.
@@MikeStock88 Sometimes making the estimate would take longer than doing the work. I think they should alot an hour of automatic billable time for every estimate request. That could perhaps actually get clients to choose not to ask for an estimate and pay less.
@@MrNathanstenzel Exactly! If you want accuracy it costs time and sometimes that time would be better spent actually doing it :)
Excellent talk. Thanks again. I'd like to suggest a few related matters that you might want to look into as well: The relationship between software cohesion and coupling with the ability to estimate large systems. The effect of integration (people and components) on estimations of large systems. The identification of critical paths of dependencies of both tasks and the people executing them (TOC throwback).
This was a very interesting listen. In my previous job, at a very small company, the owner was very insistent on getting estimates for entire projects at a time and was very upset if they turned out to be inaccurate but was also very upset if they were too long. Now I'm consulting for IBM, and it is as though they'd heard this exact talk some years ago and have built these ideas into their processes. You can easily guess which job is less stressful and more productive.
I've used function point analysis a few times and it worked for me. Overtime you can fine tune the function point weights to get even better estimates for your specific environment.
I also use the "small-medium-large" process, usually with a multiplier upon submission :) People expect software estimates to be nothing more than a combination of known tasks, like your example; making coffee, or changing a tire, grilling a burger, etc.. - except it isn't like that. The REAL DIFFICULTY is providing estimates around a large project for an external customer, it decides whether or not you get the contract. No two companies will estimate the same, nor make the same assumptions. A naïve customer may go with the lowest bid and experience endless project change requests demanding more money, all because stated assumptions were broken. Having more experience only helps but doesn't guarantee a thing, you just have more projects to compare to. That said, if what the customer wanted already existed they wouldn't be coming to you to create something new. We've all seen and heard about very large software projects going way over initial estimates. This video is helpful to anyone, not just software developers.
This is the best channel on KZhead for software developers
I just "discovered" you on KZhead Absolutely EXCELLENT (maybe it reflects what I *have* been finding on KZhead). (I, too, have read "Rapid Development" back in the day)
Great video! Your experience is appreciated, it's great to be able to hear these broad concepts expressed by someone like you.
"How long will it take" was always a big challenging question for me. Damned time quotes. I do not know much about a time estimate until I start. This video got a "Subscribe" from me.
I will watch this video after first mentioning my approach. I want, from a developer trying to write a deliverable product, two numbers: an interval of time, and a probability of delivery on the end of that interval . By setting the probability low, the developer gives himself/herself increased allowance for having missed it. My assumption (good or bad) is that if the product is not deliverable at that end of the interval, the same interval and probability reapply. I do this for estimating my own time in producing something for myself. I estimate (say) that I have a 90% chance of completing this in an hour, with the thought that if I fail to do so I will have a 90% chance of completing it in the next hour. This lets me, as a planner, create a probability function over time for product delivery, and reduces pressure on my programmer to have to live up to one single number. By allowing them to choose duration and probability, they have more flexibility in expressing a commitment.
the trick here is to manage expectations. so also to tell the client/manager what they get with the budget and timeline they offer. so tell them the truth. or else you will have to meet there expections which will be very hard to meet.
This is sage advice. Listen to this man, he knows what he's talking about.
I love all your videos. Best source to practice interview questions
This channel is absolute gold.
I really enjoy this channel, the topics are spot on. Thanks!
I've made a mistake when I've once gave optimistic and pessimistic estimates - 'cause business took optimistic estimate and ignored another which led to bad circumstances and I became scapegoat. And I warned many-many times that they shouldn't rely on optimistic one. But they didn't listen So now... I don't give 2 estimates anymore - only 1 - the pessimistic one
So good, so concise, so on point!
90% of the work takes 90% of the development time, the remaining 10% of the work takes the remaining 90% of development time 🤣
Another twist on the same topic is that 90% of software projects are 90% complete 90% of the time...
A reliable estimate will take 110% of the delivery time.
This is painfully true
Oh my god. This channel is so informative and helpful. Thank you so much!!!
This is valuable. And the format is unique. Keep it up!
This video is so spot-on, I'll need to share this with my whole team.
"The hardware is mostly in place...." Classic red-flag.....
...depends on industry. In mass production automotive, silicon (HW) contracts are done before the SW is barely started. I'm not saying it's good or bad, just what it is. In general though I'd agree...you know what HW you'll need once the SW solution is well understood.
Great video. Surprisingly interesting! Love the Solar system shirt too. 🌞
Keep up the good work, I love your videos. By far the most accurate representation of real world development.
Thank you 😎
Best value for time, this video! Thanks 👌
watched again - still gold
Aways brilliant! Thank you mr. Farley. I've been learning so much with you.
Thanks 🙂
Thanks for an excellent video and a lucid explanation.
15:50 "..is an exercise in learning and one of the things we learn as we proceed is what the target actually is" Now that is PROFOUNDLY accurate
Wow your advice is gold. Thank you for being real x
Thanks.
This was a very good video on estimation with some good points a s/w developer can covey to stakeholders on why estimates are just guesses. Thanks for the video.
Totally agree. Broken estimates is the one of major reasons of burnout
Best summary on "how to estimate" I've come accross so far. Thanks for sharing !
Glad it was helpful!
I think planning is about carefully choosing what to discontinue under some sh*t storm that will sure come. Great channel.
You are so good. I'm a complete beginner as a developer but everything you say strikes me as incredibly invaluable based on my experience in other fields. Thank you very much for your sharing your thoughts.
You're very welcome!
You understand how it works and you say it. What you are saying is the obviousness I have felt in my bones for 25 years of work, but so much misunderstood by business. I wish my bosses would pass exams on what you say. I want to work for you, be my boss :)
😁 😎
I find that the best way to estimate is to forget all specific details of the current project/task and to think about similarly complex and small/large tasks/projects in the past and how long those took. Then just take the maximum of those previous ones and that will be a fairly good estimate. If you are doing something genuinely unlike everything you've done so far then ask people who have such experience. The beauty of this method is that it does not rely on any specific project details. Of course the aim here is a ballpark estimate - 1 man-day vs 10 man-days vs 100/1000 man-days...not an exact number of days. So in this way it is like the T-shirt size method.
As a developer I hate giving estimates, especially when I'm working on large old projects. Sometimes I have a rough idea, e.g. a week but I will use the "scotty principle" and multiple it by 4. Then when I do finish it in a week I'm a genius, if issues happen and it takes 4 weeks, they (I told you so)... Scotty Principle is from Star Trek - Scotty always multiplied by 4 when Cpt Kirk asked him how long repairs would take... Great video though. Thanks!
Great video Dave! Cheers!🎉
I got super super excited when I heard the name Steve McConnel, the author of my all-time-favorite programming book: Code Complete. I'll start reading Rapid Development ASAP.
Yes, both are excellent books
It sucks I didn't have you as one of my CS teachers.
😉
When I was a CS undergrad, we had a professor who wanted each function/procedure so well documented it became a disincentive to making small functions. You were better off with a huge function because of the time savings in documentation.
@@IntraFinesse Yes, it is not uncommon outside accademia to say such things, but I believe that comments are often used as a crutch to support poor code, but to write good, readable, code. I think focusing on minimising the need for comments helps focus our minds to write better code!
@@ContinuousDelivery I agree. I have found that using long and descriptive function names, and modularizing code, its much easier for others to follow. "go read the massive and outdated documentation" doesn't help much.
Unless you have many chances to practice software engineering in school, or you probably won't understand what the teacher is talking about. More likely one recieve these ideas too early would become too defensive in work place, I think.
I told a group in my company that I will write an ETL tool for them and give me a few weeks. So far Im into month 3. This ETL thing is complex to write but its designed to be easy for the user. Making anything that's easy for the user, is hard for the developer.
Bro, just point them on an AWS glue/SQL integrations services/informatica power center course and be done with it.
@@JorgeEscobarMX don't know much of these things, will look into it.
use kafka - unsolicited advice :)
This is another great video, from a great channel :)
Great video. Thank you so much, Sir!
What a gem! Thank you very much, Dave 🙏
Glad you liked it!
Fascinating. Thank you
This is something I have been looking for a long time! Thanks for sharing your expertise
Glad it was helpful!
The lengths that we must go to for project funding and the amount of diplomacy to hide the fact that Estimating = Guessing. This is especially true when this type of work has never been done before with a team that has only just met.
I fully agree with the stated impossibly of estimation. Even SP is of limited value. S/M/L is about the possible precision. Unfortunately this leads people to insist that you can’t plan what you want to make in any detail
This is a simplistic view. It only works for agile inhouse development where there is a full trust relationship between the product owner and the developers. Many times public organisations have to define the scope to some level of detail and accept a development offer from the lowest bidder. In many cases organisations have far more pending projects than budget or time allows. Then the project with the best return on investment has to be chosen. If no attempt is made to estimate it is impossible to prioritize sensibly. Function points are widely used for estimates and for some governments a requirement for project estimation. An ISO standard exists for function point estimates. Organisations keeping track of functional size, development work and quality can estimate with a useful level of accuracy. Nobody is asking for 100% accuracy.
How many big projects are late or fail to deliver at all? The data suggests that it is about the same number (about half) as if you used random numbers as estimates. The evidence that I have seen (in the Rapid Development Book that I mention in the video, which is from a long time ago now) says that even if you take a rigourous function-point based approach (or any other) there is no improved correlation with accuracy. Of course organisations buy software services on the basis of these estimates, but that doesn't mean that they are accurate. I think that assuming that such accuracy is possible is the mistake. We don't fund start-up compaines based on such estimates, we take a more iterative approach to figuring out how they are progressing (I mean the venture capital process). Software projects, for anything not completely trivial, are exercises in similar levels of complexity and uncertainty.
@@ContinuousDelivery Thanks for replying. Function points only provide an indicator of the size of the software. Metrics on development effort, productivity or quality are meaningless unless qualified with the software size. Steve McConnell in his book Rapid Development in 1996 did include a 40 page chapter on estimation. We can only assume that he did think attempts for estimation are not a pointless exercise. Browsing through it now, it seem surprisingly relevant today. Then McConnell did better, he wrote an entire book on estimation, "Software Estimation" in 2006. A very good book in my opinion. He wrote it with an open review process. I did send in comments on chapters on size estimates. I tried but failed to convince him to base size estimates on function points rather than lines of code. He did however acknowledge my contribution :) For agile projects exploratory in nature developed in a full trust relationship between the product owner and developers, estimation should not be a high priority. However I believe there are many situations where software metrics and estimates can be helpful. People controlling budgets also tend to like estimates, even if they are not accurate. For anyone interested in software metrics and estimation I recommend reading some of the books from Capers Jones. He has spent an entire career on the subject and has written more than 15 books about it.
trust me bro! los of people are asking for accuracy - even making sue we don't overrun a sprint is almost impossible. never mind a project
I agree on everything you said 100% and it's very inspiring that people start talking about the burden of "estimates" - guesses as you said - to the industry. But I think it's also important to name as well the elephant in the room, the magic word that is to blamed for: Budget. This all happens because of "budget restrictions". The fact that someone signed at some point in time a paper that locks down the scope to a fixed date preagreeing everything without consulting you first, most times without even consulting other members, experience from other projects etc. And now you have to pay the price about it and deal with it however you can. Budgeting seems to me a very dated idea that cannot stand the challenges of software development. Yet it somehow persists throughout the industry as some (managers mostly) see it as the only possible tool to do business with, however catastrophic its results prove to be.
Yes, really common for orgs to be a bit agile in developent and totally waterfall in terms of governance, budgetting and org-structure - doesn't work very well. There are other models for governance, some described in the "Accelerate" book, but all of the ones that I have seen have been iterative and more like "internal venture capital" than "planned budgets".
Management probably just wants to be able to do price comparisons. Feature A is going to cost $50,000, feature B will cost $80,000, and so on. What happened where I worked was that estimates were "fairnessed" away. You clearly can't be developing while you're helping with some other emergency, so the estimate only covers hours spent on the feature set, not calendar days. Also, if the requirements are off base, obviously you can't be expected to estimate the time spent going from what they asked for to what they really ended up wanting. Oh, and who's going to do the coding is a big deal, of course - junior developers can't possibly work as fast as senior developers. By this point, though, the estimate really isn't useful for doing the planning that management wants to do. I don't know why those went over well while "it's really not possible to say" didn't, but there it is. Don't be too harsh on the management side, though. The other side of that price comparison is figuring out how badly the business needs the feature, which is at least as nebulous as our estimates. They are essentially being plopped down between two clouds then asked to figure out which one is heavier by sticking one hand in each. Still, what happens when programmers are forced into these estimates tends to be really ugly, both for the programmer and the company as a whole.
Dear sir, thank you very much for the great content. Just wish that you could bring the echo down, using a better recording/editing.
Yes! I am an electronic hardware engineer and have been designing high technology products with HW, FW and SW for a long time. People try to do what I call "Estimation by Synthesis" where they break stuff ddown and say "That bit will take 2 months", etc. This gives hopelessly optimistic estimates. I do "Estimation by Analysis" where I try to find a recent project that is somewhat comparable and use that as a gauge - multiplying and divivding to allow for differences. This usually gets within a factor of 2x up or down, compared with >5x for the synthetic estimate! Syntehtic estimates usually need to be multiplied by Pi to improve them... :) Estimating a whole SW project by creating stories and scoring them is totally fatuous, IMHO. It is completely against the principles and ends up with a hopeless estimate.
And ex-manager once asked me: "What's the part that you struggle the most with?" I responded "time estimation, do you have any literature or advice on that?". Then, he went "It's indeed the most complicated part of our profession. I sometimes use a simple trick: your realistic estimation times 2 (at least), or even 3 if the system is very complex or you don't have enough information". My face was like "oukey :v", but it kinda works xD. If you want to do it properly, I'd suggest tracking your time taking into consideration all possible variables. For instance, type of task, technology, complexity, unit tests, integration tests, lack of information, level of dependency from other colleagues and teams, etc.. Also, the number of interruptions (meetings, etc.) or level of insulation from distractions (colleagues interactions, noise, etc.), and even your physical and mental condition. There're already apps and tools that could be integrated in order to apply time estimation formulas.
I was brought into new to a new team and was being "forced" to estimate a large project once. The team had no history, and no one on it knew much about what all needed to be done. I analyzed the work and produced an estimate that I was very unhappy with. I told the manager "here you go but it's b*******". She was pretty unhappy with that response, and I felt bad about it. I found McConnell's 2006 Bill "Software Estimation" and it really helped frame the problem. We re-estimated and reframed the estimate with a lot of his material including the Cone of Uncertainty. It went down much easier this time. There are reasons people want estimates, and they are not terrible because they want them. And estimates do provide an opportunity for the team to understand the work. It takes time to do that though and McConnell does stress that and has a take on diminishing returns of re-estimation. Overall, for most software projects it's art not science. McConnell states this as well and does a great job at looking at the science of estimation and how it only applies to a narrow set of cases.
Great video! Recently had a conversation with a stakeholder. Was forced to give an estimate on a feature and guessed 1-2 weeks until it was in production. After 1 week, I get an email about me being late on a deadline I apparently committed to… 🙄
We all learn this the hard way My CS professor told me to estimate as large as I could possibly imagine And when the customer balks, talk about reducing the feature set or defining the problem more and then pair it down by 10% each time
"How long is a piece of string?" I'll know when I've unravelled it
Thanks for share your thoughts. Great content.
Thanks for watching!
If the new task is very similar to what you did last week, expect to be faster this time. Practice. And familiarity with the concepts. This doesn't apply much if last weeks project was also the same as the week before.
What is typical or good value for a team's velocity volatility that you have seen? To make sure our definition of volatility is aligned: volatility = stdDev(of: velocity, over: 4+ previous iterations) / expectedTeamVelocity. Example: my team has an expected velocity of 7pts, their observed velocity has a standard deviation of 2pts when looking at the last 4 iteration's velocities. So their volatility would be: 2pts / 7pts = ~28%. In my limited experience of teams that actually bother to measure and quantify such things, this is the best value I have ever seen. Most teams are in the 40-60% range. I hope I'm not so late that you won't see this question, I would love to get some thoughts about where you think 28% sits even if that comes with all kinds of assumptions. Love your content! Without fail you are able to clearly articulate something about software engineering craftsmanship that I completely agree with and couldn't say better myself! My co-workers and I are a small fan club of yours.
In my experience - 40 years of Software design and development - I can tell you that almost all projects go as follows: 1) The customer decides what it will cost. 2) The customer decides when it must be finished. 3) The users tell you what they think they need. 4) You make an estimate based on 1) 5) You start designing the software 6) The users change their mind every week. 7) The customer's project leader - who doesn't know anything about software development - asks you if it cannot be done faster and for less money. 8) Go to any of the previous steps. By the way, I am talking about global players. Thank God I am almost 68...
Thank you for these sir, much respect. :D
What a great channel. Thank you
Great explanation.
I had a professor that said when estimating times, take into consideration everything you know and then double it and then double it again.
I'm french speaking and I just realized we're using the same word - in french - for both accuracy and precision.
A Hungarian here: we too. However, sometimes we use words coming from other languages for these.
When I free lance I tend to estimate and educate. I have a previous background in global business supply chains and was at a director position before I stepped down to fully dive into app/api engineering. If the potential client is essentially a start-up, I tend to speak in terms of sunk cost, opportunity cost, MVP, and risk/reward potentials: since that's the language they understand. They don't understand dependency inversion and creating proxies and have no intention of learning the concepts. I will happily give them the reality of how long a project could take based on known knowns, known unknowns, and unknown unknowns in terms of cost and talk about budget. If I can see myself EASILY (keyword) hitting that budget with dev time, I'll let them know that upfront and help them avoid a business-killing decision where the risk far exceeds the reward. It takes good communications skills to establish trust to get as close to the real numbers as possible, because the first budget they tell you is a lie when their defenses are up. I'm sincere in wanting to protect their business and not unfairly enrich myself at their expense. I can guarantee this is much harder to do with larger projects as a member of a team without access to financial data. Besides, there's too many hands in the cookie jar to figure this kind of thing out with any kind of real accuracy. I'm curious how different companies manage their projects' timeline goals and budgets for core and desired features. My guess is, from my experience, they don't and essentially have no idea what's going on. That goes into another topic of the systemic failure of properly knowledgeable managers and visionary leaders. It's rare. You care more about them than they do you and their own business, and they exploit it.
Yes, I have seen a lot of similar things. Where there is sufficient trust established, my preference is for "venture-captial" style, incremental funding, "We think we we see an opportunity here", "here is enough cash to explore, come back when you know more" and so on. I also agree with you that most orgs don't even track this stuff and make decisions based on hunch and "we have always done it like that" rather than data. I also agree that there is often a dearth of leadership, with many people in big orgs focusing on personal, tactical advancement, rather than the real outcomes of the org. Sigh!
As à Product manager I often get to see both sides. Contract worth big money (and commission), verses fluffy specs and a “ideological” approach to estimate are Satans love child. What tended to work well was to make t shirt estimates and then ask customer to pick key ones. We would commit to those by date X. Other t shirts delivered and paid for in a soon as ready. Company got a commitment to revenue, customer had a promise of essential delivered on a certain date. Did we always get it right….. no. Specs change, surprises happen, but there was less panic and heat. It brought sales and development working together rather than beating each other up. Best day was when head of sales turned down a contract based on an estimate. Trust was key. The big fish contract we were working on came home on time.
100% plus or minus 100% spot on. New subscriber here. And my new guru.
Thanks and welcome
Estimate = guessing. Planning, researching and breaking down the features into tasks (WBS) would help with this. Ideally, the manager should sit with the developer to map out all the tasks and the scope of the project before you can assign time to each task. This should help with the estimation. The problem is that managers are lazy and think that their job is to forward emails to developers.
BS
@@rishatsharafiev great response 👍
@@reagang8038 there is no point to estimate, you will waste each other’s time, when actual problems of customers will be frozen. I tried all this things, it’s basically became group masturbation session.
I have not found that adding detail makes the estimate more accurate and it is a waste of time. Just look at smaller chunks and get started. Or compare with a comparable project in the past and guess based on that.
"The infinite fractal coastline of estimation", I love that term.
Good content! And I like your explanation of precision and accuracy. Personally, my estimate multiplier is Pi. It's worked out well so far.
Not e?? :)
Another brilliant video!
Thanks 😎
In answer to this question from a renowned slippery business analyst in my then company who always blamed the programmer. I said "well if you can tell me how many words will be in your specification" His sharp answer was "I can't tell you that I haven't written it yet.." This was many years ago before 'Drops Mic' was a thing
as always, good explanation. whould do please give us some examples of features that take one to two weeks to be completed?
Your videos are always very interesting because you are transfering your experience from industry and are not just academic knowledge. Because in most of your videos you mention the need of continuous learning i would like to ask you to make a video about effective learning strategies and tactics for software developers-engineers.
Thanks, I will think about that.
@@ContinuousDelivery That'll be very helpful, please.
Thank you for this summary. There are many good ideas but what was left out are the real world problems. You said you estimate only, if you are forced. But being forced to estimate complete projects is common because managers learned it that way and customers require it. The question is how to resolve this natural conflict. You cant say "don't estimate".
I try to have a rational conversation of avoiding fixing time & scope, and ask which is the priority. That allows you to start with a rational plan instead of a fairy tale. If you can't make any progress with that, then cross your fingers and give them you guess, but ideally with some error-bounds, a best and worst-case guess.
@@ContinuousDelivery Thank you for your answer. There are projects where this strategy works. But those projects are probably no fix price projects. It is just the nature of fixed prices that you want to shift the risk to someone else. But you right giving this a try is always a good idea because a working result is in every case better than being feature complete.
Estimation, like design, is as much art as it is science -- it involves being able to account for both known and UNKNOWN variables... The larger the number of unknown quantities and the greater their impact on the critical path of development, the wider the range of probability will be. The key to successful estimation is to separate the known from the unknown quantities, and estimate each separately. Due to the high level of uncertainty in some projects, we should engineer our process to absorb failure. The beautiful thing about software development is that it is not an all-or-nothing proposition. Code that is developed for one project can easily be repurposed for another, if it is documented and well-organized. Every project should be seen as an opportunity to develop a knowledge-base that can be leveraged for future development, consulting, or as a product in itself. Known Quantities should be kept to a tight budget and schedule to maximize time and resources available for unknown quantities. We should also assume that any unknown quantity will require time for additional research. Projects with large number of unknown quantities should be treated as exploratory expeditions --- you don't know what you are going to find. You may find that your original goal is infeasible. You may find an alternative goal that is even more profitable. Or, you may find that you have hit a dead-end. Again, unknown quantities mean that you DO NOT KNOW how much it will cost or how long it will take -- You are going to discover these values as you progress.. The key to a successful PROJECT in this situation is NOT an accurate estimation, but to look for additional value beside your central goal. Do not pass up platinum or silver in your quest for gold. Every challenge is an opportunity, every set-back a new path forward.
Never give a casual estimate to answer the question "When will it be done?" Any answer you give, no matter how conditional and off-the-cuff will be taken as a hard deadline by people who see your task as yet another empty checkbox in a long list of checkboxes. If someone is asking you to estimate something complex and expecting an on-the-spot answer, you're already in a precarious position. It means you haven't educated them on the nature of your work. It's OK to refuse to answer. As much as it pisses them off, it will piss them off FAR LESS than giving an under-estimate and causing project slippage, regardless of how hard you work. I had to learn that the hard way earlier in my career. No one cares about heroic efforts which miss an unreasonably low project deadline estimate (especially if you quoted it yourself). It's always seen as a failure. Much better to give phased target dates for pieces of the deliverable and adjust subsequent targets if any previous targets slip.
I quite working with deadlines long ago, we get much better results that way
My response frequently resembles something like: "It takes as long as it takes." This position makes me very unpopular.
Well from a good product standpoint thats viable. But from a buisness side certainty about some uncertainty makes it easier to get money in the right places.
Only one problem ... We are forced to make the estimate anyway. Sales wants to sell projects and customers want a budget proposition. No two ways about it really. On top of that, you're never just the only one bidding for the project. So you simply will not sell 'we shall see when we get there' when the competition is like 'gimme 50 days'. Even on internal projects management always requires an estimate. Yes ... They really are the bane of any software development team.
I was just thinking of *Bargue Drawing*. You make rough straight lines first and make them shorter and more detailed by the time... 🤔 Of course software is sometimes about implementation details that are hard to nail down. But having a broader picture asap is always good. Also for ETAs ;]
I was ripped apart by our chief engineer in front of the whole team when I was unable to provide an estimate to design and implement an entire application for a client. Two weeks later he apologised , but I had already lost a great deal of respect for him after his little outburst.
I would have immediately started looking for another job and not working too hard on that project at that point Unprofessional attitudes like that deserve passive aggressive punishment
If you are in a position, then you should try to move to a different project or change jobs
Where have you been? Subscribed.