Speed – being first to market, rapid innovation and conducting fast cheap experiments – is critically important to startups and many high tech firms. This is where Lean Startup ideas and Continuous Deployment come in. And this is why many companies are following Agile development, to design and deliver software quickly and flexibly, incorporating feedback and responding to change.
But what happens when software – and the companies that build software – grow(s) up? What matters at the enterprise level? Speed isn’t enough for the enterprise. You have to balance speed and cost and quality. And stability and reliability. And security. And maintainability and sustainability over the long term. And you have to integrate with all of the other systems and programs in your company and those of your customers and partners.
Last week, a group of smart people who manage software development at scale got together to look at all of these problems, at Construx Software’s Executive Summit in Seattle. What became clear is that for most companies, the most important factor isn’t speed, or productivity or efficiency – although everyone is of course doing everything they can to cut costs and eliminate waste. And it isn’t flexibility, trying to keep up with too much change. What people are focused on most, what their customers and sponsors are demanding, is predictability – delivering working software when the business needs it, being a reliable and trusted supplier to the rest of the business, to customers and partners.
Enterprise Agile Development and Predictability
Steve McConnell’s keynote on estimation in Agile projects kicked this off. A lot of large companies are adopting Agile methods because they’ve heard and seen that these approaches work. But they’re not using Agile out of the box because they’re not using it for the same reasons as smaller companies.
Large companies are adapting Agile and hybrid plan-based/Waterfall approaches combining upfront scope definition, estimating and planning, with delivering the project incrementally in Agile time boxes. This is not about discovery and flexibility, defining and designing something as you go along – the problems are too big, they involve too many people, too many parts of the business, there are too many dependencies and constraints that need to be satisfied. Emergent design and iterative product definition don’t apply here.
Enterprise-level Agile development isn’t about speed either, or “early delivery of value”. It’s about reducing risk and uncertainty. Increasing control and visibility. Using story points and velocity and release Burn Up eporting and evidence of working software to get early confidence about progress on the project and when the work will be done.
The key is to do enough work upfront so that you can make long-term commitments to the business – to understand what the business needs, at least at a high level, and estimate and plan this out first. Then you can follow Agile approaches to deliver working software in small steps, and to deal with changes as they come in. As McConnell says, it’s not about “responding to change over following a plan”. It’s having a plan that includes the ability to respond to change.
By continuously delivering small pieces of working software, and calibrating their estimates with real project data using velocity, a team working this way will be able to narrow the “cone of uncertainty” much faster – they’ll learn quickly about their ability to deliver and about the quality of their estimates, as much as 2x faster than teams following a fully sequential Waterfall approach.
There are still opportunities to respond to change and reprioritize. But this is more about working incrementally than iteratively.
Kanban and Predictability
Enterprise development managers are also looking at Kanban and Lean Development to manage waste and to maintain focus. But here too the value is in improving predictability, to smooth work out and reduce variability by finding and eliminating delays and bottlenecks. It’s not about optimization and Just-in-Time planning.
As David Anderson explained in his keynote on Delivering Better Predictability, Business Agility and Good Governance with Kanban, senior executives care about productivity, cost and quality – but what they care about most is predictability. The goal of a good software development manager is to be able to make a customer promise that their organization can actually meet.
You do this by keeping the team focused on the business of making software, and trying to drive out everything else: eliminating delays and idle time, cutting back administrative overhead, not doing work that will end up getting thrown away, minimizing time wasted in multi-tasking and context-switching, and not starting work before you’ve finished the work that you’ve already committed to. Anderson says that managers like to start on new work as it comes in because “starting gives your customer a warm comfortable feeling” – until they find out you’ve lied to them, because “we’re working on it” doesn’t mean that the work is actually getting done, or will ever get done. This includes fixing bugs – you don’t just fix bugs right away because you should, you fix bugs as they’re found because the work involved is smaller and more predictable than trying to come back and fixing them later.
Teams can use Kanban to dynamically prioritize and control the work in front of them, to balance support and maintenance requirements against development work and fixed date commitments with different classes of service, and limit Work in Progress (WIP) to shorten lead times and improve throughput, following the Theory of Constraints. This lets you control variability and improve predictability at a micro-level. But you can also use actual throughput data and Cumulative Flow reporting to project forward on a project level and understand how much work the team can do and when they will be done.
What’s interesting to me is seeing how the same ideas and methods are being used in very different ways by very different organizations – big and small – to achieve success, whether they are focused on fast iterations and responding to rapid change, or managing large programs towards a predictable outcome.
Reference: Predictability – Making Promises you can Keep from our JCG partner Jim Bird at the Building Real Software blog.
