The Estimation Paradox: Why Developers Aren't Bad at Estimating, They Just Need Better Conditions

The Great Estimation Dilemma

“How long will this feature take?” - probably the most dreaded question in software development. We’ve all been there: sitting in a meeting room, staring at a user story, and feeling that familiar knot in our stomach as we try to conjure up a number that somehow represents the complexity of turning an idea into working code.

The conventional wisdom suggests that developers are simply bad at estimating. They’re too optimistic, they don’t account for edge cases, they forget about testing, and they underestimate complexity. But what if I told you that developers aren’t inherently bad at estimation? What if the problem lies not with the people, but with the process itself?

The Core Problem: Developers aren’t bad at estimating - they’re just not given the right conditions to estimate well.

The Classical Enterprise Estimation Theater

Let’s paint a familiar picture. You’re in your typical “enterprise” Scrum setup:

The Estimation Ritual

1. The Setup: Stories are presented, often with minimal context
2. The Rush: Quick discussions because “we have many stories to estimate”
3. The Poker: Planning poker cards fly, influenced heavily by the loudest voice
4. The Magic: Excel spreadsheets transform story points into “real” estimates
5. The Promise: Timelines are created and commitments are made

This process feels structured and professional. It has ceremonies, it has numbers, it has spreadsheets!

But does it actually work?

The Velocity Myth

For smaller, well-understood tasks (typically 1-3 story points), this system often works reasonably well. Teams develop a rhythm, velocity stabilizes, and predictions become somewhat reliable. The magical Excel sheet learns to account for the team’s patterns.

But then we hit the wall: larger stories (5+ story points).

The solution? “Let’s split these 8+ story point epics!” - often done arbitrarily, without deep understanding of the technical complexity or dependencies involved.

Why Traditional Estimation Fails for Complex Work

The Unknown Unknowns Problem

The fundamental issue isn’t that developers can’t estimate - it’s that they’re being asked to estimate work they don’t fully understand yet. For complex features, the real challenge lies in:

• Technical architecture decisions that haven’t been explored
• Integration complexities that become apparent only during implementation
• Performance considerations that require experimentation
• Third-party API limitations that surface during development
• Database schema changes that ripple through the system
• Internal team dependencies where other teams don’t have capacity right now

The Confidence Gap

When developers work on familiar territory - fixing similar bugs, adding comparable features, or using well-known patterns - their estimates become surprisingly accurate. The problem arises when they venture into uncharted territory without adequate exploration time.

The Discovery-Driven Alternative

What if instead of forcing developers to guess, we gave them what they actually need: time to discover?

The Discovery Phase

Before asking for estimates on complex work, teams should be empowered to conduct discovery:

The Restart Philosophy

Here’s the counterintuitive part: after discovery, throw it away and start fresh.

The discovery phase isn’t about building the final solution - it’s about building knowledge and confidence. The messy proof-of-concept code, the experimental approaches, the quick-and-dirty integrations - all of that gets discarded. It also means that during this phase, the focus is on learning, not delivering. No TDD, no production-quality code, no sonar - just exploration.

What remains is:

• ✅ Clear understanding of the problem space
• ✅ Validated technical approach
• ✅ Realistic estimates based on actual exploration
• ✅ Risk mitigation strategies
• ✅ Team confidence in the proposed solution

Building a Discovery-Driven Culture

Empowering Teams

The shift to discovery-driven estimation requires fundamental changes in how we approach software development:

Management Perspective:

• Accept that complex work requires exploration time
• Budget for discovery phases in project planning
• Measure success by delivery quality, not adherence to initial guesses

Team Perspective:

• Request discovery time for unfamiliar or complex work
• Document findings and share knowledge across the team
• Provide estimates with confidence levels and risk assessments

The Economics of Better Estimation

While discovery phases require upfront investment, they typically result in:

Traditional Approach	Discovery-Driven Approach
Frequent scope creep	Controlled scope changes
Late-stage architectural changes	Early architectural decisions
Team stress and burnout	Team confidence and satisfaction
Unpredictable delivery	Reliable delivery timelines

Introducing: Discovery-Driven Estimation Framework

I’m currently developing a comprehensive framework for implementing discovery-driven estimation in development teams. I will be sharing the details in upcoming blog posts.

Stay Tuned

If you’re interested in being part of the pilot program feel free to reach out!

Conclusion: From Guessing to Knowing

The problem with software estimation isn’t that developers are bad at it - it’s that we’ve created systems that force them to guess instead of know. By shifting from immediate estimation to discovery-driven exploration, we can:

• Increase estimation accuracy
• Reduce project stress through better understanding
• Improve code quality through thoughtful architectural decisions
• Enhance team confidence through knowledge-based planning
• Deliver more predictably through realistic timelines

The next time someone asks “How long will this take?”, maybe the right answer isn’t a number - it’s “Let me explore that and get back to you with a confident estimate.”

After all, the best estimates come from knowledge, not guesswork.

---

What’s your experience with estimation challenges? What solutions have you found for this problem, or don’t you experience it in your team? I’d love to hear your thoughts and experiences.

Further Reading

• The Mythical Man-Month - Frederick Brooks Jr.