Once you have a program (a collection of interrelated projects focused on one business goal) and you have technical debt, you have a much bigger problem. Not just because the technical debt is likely bigger. Not just because you have more people. But because you also geographically distributed teams, and those teams are almost always separated by function and time zone.
So, my nice example of a collocated team in Thoughts on Infrastructure, Technical Debt, and Automated Test Framework, rarely occurs in a program, unless you have cross-functional teams collocated in a program. If they do, great. You know what to do.
But let’s assume you don’t have them. Let’s assume you have what I see in my consulting practice: an architecture group in one location, or an architect in one location and architects around the world; developers and “their” testers in multiple time zones; product owners separated from their developers and testers. The program is called agile because the program is working in iterations. And, because it’s a program, the software pre-existed the existence of the agile transition in the organization, so you have legacy technical debt up the wazoo (the technical term). What do you do?
Let’s walk through an example, and see how it might work. Here’s a story which is a composite from several clients; no clients were harmed in the telling of this story.
Let’s also assume you are working on release 5.0 of a custom email client. Release 4 was the previous release. Release 4 had trouble. It was late by 6 months and quite buggy. Someone sold agile as the way to make software bug-free and on-time.
You do not have automated tests for much of the code, unit tests or system tests. You have a list of defects that make Jack the Ripper’s list of killings look like child’s play. But agile is your silver bullet.
The program manager is based in London. The testers for the entire program are in Bangalore because management had previously fired all the testers and outsourced the testers. That was back in release 2. They have since hired all the Bangalore testers as employees of the Bangalore subsidiary. The program architect is based in San Francisco, and there is an architect team that is dispersed into 4 other teams: Denver, LA, Munich, and Paris. The developers are clustered in “Development Centers of Excellence:” Denver, LA, Cambridge, Paris, London, Munich, and Milan. That’s 8 development teams.
Oh, and if you think I’m kidding with this scenario, I’m not. This is what most of my clients with geographically distributed teams and programs face on a daily basis. They deserve your sympathy and empathy. Do not tell them, “Don’t go agile.” That’s nuts. They have a right to go agile. You can tell them, “Don’t go Scrum.” That’s reasonable. Scrum is for a cross-functional co-located team. Agile is for everyone. Scrum is for a specific subset.
What do you do?
- Assign specific testers to specific development teams. No calling people resources; that allows managers to treat people like resources and plug-and-play them. You need to get rock-solid teams together. Once you have teams together, you can name them.
- Name teams so the teams reflect the feature groups they work on. What does an email product do? It gets email, it sorts email, it deletes, it forwards, it creates new mailboxes, and so on. The eight feature teams had to be named for the feature areas: Platform for the general features, Sort, Delete, Forward. There were two teams who worked on Platform. They were called Platform 1 and Platform 2. At one point, someone suggested they call themselves Thing1 and Thing2 from the Dr. Seuss book.
- Make sure you have enough product owners so they can develop roadmaps for each feature area. With a roadmap, the teams know where they are going. Even more importantly, the architects know where the program is going.
- Architects think and provide just enough guidance ahead. In a small project, the architecture can probably evolve with the project. In a larger program, that risk is too large. You have too many people developing in parallel for the architecture to evolve on its own with no guidance. But I do not mean there should a Master Architect Who Knows All Handing Down the Architecture From On High. NO NO NO.I want the architect who is a working member of the development team, who also is part of an architecture community of practice team, who curates the architecture, who guides the business value of the architecture. I do not want Big Architecture Up Front. But Thinking Up Front? Sure, that’s a great idea. Stuck on only one idea? Bad. Willing to spike an idea? Great. Willing to play in a sandbox and debate several ideas? Great. I wrote about this before, in How Agile Architects Lead.
- Decide what done means for every feature. You must have acceptance criteria for each feature. What does that mean? You need a product owner present for each team. You still need the conversations with each team to discuss what done means. Especially with a geographically distributed team, you need the conversation when you create the backlog at the beginning of the iteration.
- The US development teams had trouble planning their iterations with their testers, because of the time zone differences with the testers. So, they asked their product owners if the product owner would write more than just a few phrases on the cards, because that would help them get through the iteration planning meeting faster. Someone was going to get up early or stay up late, and either way, someone was going to suffer. It made more sense to have a little bit more preparation than less sleep.
- Decide to do continuous integration and stick with it. Especially if you know you have technical debt and you don’t want to create more, you have to do continuous integration now. That prevents more technical debt.
- I have recommended to some teams that they have one-week iterations so that they stop the estimation nonsense and make their stories small. The point of estimation is so that you have an idea of what you can do as a team and not commit to more than that. The idea is that if you know what it takes to make your stories small, you will.Instead, we have all these crazy rituals around estimation and management tracking velocity of all things. (Yes, I’ve been drafting this post for a long time, and I wrote Why Does Management Care About Velocity? last week.) You know, velocity is a little like weight. Only you and your doctor need to know your weight. If you are healthy, you are fine. If you are not, you need to change something.If your team velocity is not healthy, you, as a team, need to change it. But, your management has no business butting its head in. Only you can change it.
- When you limit the iteration length, you tend to have the team swarm around a story. This is a tendency, not a given. If I really was the Empress of the Universe, I would decree this, but I’m not, so I won’t. If you want to decrease technical debt, or even eliminate it on your program, explain that your team will only work on one story at a time until that story is done. That story will be polished and gleaming. Fast. You will not have to worry about what kinds of testing will be done. All if it will be done.
- Explicitly discuss what you will automate for testing and when. In a program, I assume we will have automated system tests first. I assume we will do exploratory tests later. That’s because if you don’t start building something for test automation when you start the program and refactor as you proceed, you can never catch up. I assume every time we fix a defect, we will have an automated test for it. I also assume we build these assumptions into how we develop :-)
So far, this is all about preventing more technical debt, not what happens when you trip over technical debt as you enter code or tests you never looked at before.
If you expected to walk into a closet, take out a shirt, and close the closet door, that’s one thing. But now, you stepped into something out of one of those death-by-hoarding shows on TV, you have an obligation to do something. You can document the problem as you encounter it; you can let the product owner know; file a defect report; write a test so you can contain the debt; and maybe you have more options. Whatever you do, make sure you have done something. Do not open the door, see the mess inside and close the door on the mess. It’s tempting. Oh my, it is tempting.
See, on programs because of the size, everything is magnified. With more people and more teams, everything is harder. Things happen faster. If you have co-located cross-functional teams, no problem. But if you don’t have co-located cross-functional teams, you have to work with what you have. And, if you already have a big legacy product, you want to address technical debt in small chunks, refactoring in small bits, integrating as you proceed.
My philosophy is this: the bigger the program, the more you need to become accustomed to working in small chunks, integrating as you go. Fully implement a small story, integrate it on the mainline. Everyone on the program does that. If you need help from an integration team, so be it.
But, if everyone only implements small stories, and everyone takes care of their own technical debt as they discover it, you don’t need an army of integration people. You only need an army of integration people when you have technical debt around integration and release. Fix that, and everyone can become responsible for their own integration.
And, if you can’t release, that’s where the architects should start. If you can’t do continuous integration, that’s where the architects should start. Because that’s what’s preventing you from making progress on the product. Work backwards from release, and then the architects can work on the rest of the product. Until you can release and build reliably, the rest of the product doesn’t matter.
Reference: Programs and Technical Debt from our JCG partner Johanna Rothman at the Managing Product Development blog.
Everything seems to be fine, but: 1) There are huge number of assumptions on things being done fine, for example the stories being written correctly, the people across geographies are highly cooperative. 2) The technical dept is shared across product team members equally, where as the same is divided to members depending on their experience and level. 3) Large technical debt going on from years, and suddenly someone hired is expected to change it overnight. 4) Forget the incompetent management who let the technical dept grew to such an extent. 5) Teams driven by product management and marketing rather than… Read more »
Hi Rahul, Yes, I am assuming these things. Assumption #1, #2, and #4 are about transitioning to agile. Assumption #3 is about paying down technical debt as it affects you on the story, as you encounter it, or as it affects you. If you have a program, that increases the magnification of the technical debt. You have to pay it off, or you can’t release. I just wrote something else on my main blog you might like: http://www.jrothman.com/blog/mpd/2014/02/cost-of-delay-due-to-technical-debt-part-4.html Assumption #5 is about transitioning to agile. Agile is a set of cultural practices, not just engineering practices. If your organization can’t… Read more »
Thanks Johanna for the quick reply. 1) We have followed Agile for quite a long time in our org. but what was observed is it truly needs a disciplined team. Unfortunately unit tests are still considered a zero return effort when it comes to meeting deadlines and churning out features quickly. 2) In my org. engineering is primarily driven by Marketing and Sales, hence it is not up to the engineering to follow a tight principle in making deliveries and decide time of completion. The pressure to GoToMarket overpowers anything else. 3) How does agile factor the different levels of… Read more »
Rahul, yes, agile requires discipline from the team. It requires the most technical professionalism from the team. If you only use the project management practices from say, Scrum, and do *not* marry the Engineering practices from say, XP, you will fail. However, the Engineering team gets to say when a feature is done. No one can tell you how long a feature will take. You should read my post http://www.jrothman.com/blog/mpd/2011/06/do-you-have-feature-itis.html To answer your question #3: Stop working alone. Start swarming together. It won’t matter how much experience anyone has. And make your stories smaller. I am a big fan of… Read more »