This post helps you use your product demos as an effective agile market research tool: to collect relevant feedback in order to validate your ideas and improve your product. If you employ your demos to sign off user stories then this article will show you how to get much more out of them.
In an agile approache like Scrum, the latest product increment is demoed to the stakeholders in the sprint review meeting, as the picture below illustrates. (Please see my post “The Scrum Cycle” for a detailed explanation of the picture.)
While the product demo allows understanding which stories have been completed, I find that using it as qualitative market research technique unleashes its real potential.
Its primary goal is then to collect feedback from users and other stakeholders in order to validate ideas and improve the product. The demo is best done in person with everyone being present in the same room, but you can also conduct it as a videoconference. The following tips should help you leverage your product demo as an effective market research method.
Be Clear on your Research Goal
Understand what questions you would like to get answers to, and what ideas you would like to validate before conducting the demo. Your sprint goal should help you with this: If, for instance, your sprint goal is to test your user interface design ideas, then you should plan the demo accordingly: You may want to present different versions as mock-ups to the users to understand which one they prefer and why that’s the case. Having one sprint and research goal helps you focus the presentation. It increases the likelihood to collect relevant feedback, and it makes it easier to analyse the feedback.
Invite the Right People
Use your sprint goal to decide who can help you validate your ideas and improve the product and who should therefore attend the demo. If the goal of the sprint is to establish the right software architecture decisions, then end users are probably not the right attendees. In the worst case, the demo could be a frustrating experience and prevent them from attending another review meeting. But if the goal is to better understand how users are likely to interact with the product, then end users should be present. Otherwise, you are in danger of collecting lots of interesting but irrelevant or misleading data.
Explain what the Product does for the User
Avoid listing features and functionality in your demo, and describe what the product does for the user in order to receive meaningful feedback. A great way to do this is to use a scenario. If you develop a mobile banking application, for instance, you may want to say: “Imagine you are on the train on your way to work, and you remember you still need to pay your water bill. You open the banking app, log on, and then you would see the screen I am showing you now.” If you employ my Product Canvas, then you should be able to use its scenarios and storyboards for your demos.
Engage in a Dialogue
An agile product demo should not be a one-way communication or a sales event. Instead, its objective is to generate valuable feedback that allows you to gain new insights. Unfortunately, users and other stakeholders don’t always provide helpful feedback straight away. You sometimes have to ask the right questions and create a dialogue. For instance, if the feedback you receive is “Great demo, I really like the product”, then that’s nice. But what does it actually mean? How does it help you, and what can you learn from it? Dig deeper, ask why the individual likes the product, which aspects are particularly valuable, and which could be improved.
Take Notes
To be able to analyse the feedback afterwards, I recommend you record who provides the feedback and what you hear and see. Ask the team members to take notes too. This reduces the risk of overlooking feedback. I also suggest you record relevant background information about the attendees including demographics and job role. The information will be handy when you analyse the feedback.
Separate Research from Analysis
I prefer to separate collecting the feedback from analysing it. This allows me to listen to the users, and to decide afterwards what I can learn from the information gathered by carefully considering if the feedback is relevant and how it is best acted upon. It also makes it possible to compare notes with the team members thereby leveraging the collective wisdom of the team and mitigating cognitive biases. But I do suggest that you reject an idea or request immediately if you know that it does not make sense or that it is impossible to take it on.
Understand the Limits of the Product Demo
A product demo is a great tool for getting feedback particularly in the early sprints when the product has not enough functionality to be exposed in other ways to the users. But it does have a drawback: Users provide feedback based on what they see and hear. The demo does not validate how people actually use the product. I hence recommend you employ user tests and software releases once your product has progressed further. This allows you to understand better how the users interact with your product, and how well your product meets their needs.
Summary
A product demo is a great tool for collecting feedback particularly in the early sprints. To fully leverage your demos, make sure that you understand your research goal, invite the right people, explain what the product does for the user, create a dialogue, record the feedback, do the analysis afterwards, and consider employing user tests and releases as soon as your product as progressed further.
Reference: The Product Demo as an Agile Market Research Method from our JCG partner Roman Pichler at the Pichler’s blog blog.
