Using open and closed questions

Closed questions are those which can be answered by a simple ‘yes’ or ‘no,’ and maybe a ‘maybe’! Open-ended questions are those which require more than a simple one-word answer. Both have their place.

Closed questions can be used to establish facts. They can help you to decide whether there are further questions you want to ask, so a closed question can be followed by an open question as shown here:

Did you enjoy reading this book? (closed question: yes, no, possibly a middle ground ‘I quite enjoyed it’ answer)

What was it about it that you liked / did not like? (open question)

Open questions will get you more information as they prompt fuller answers. So, when you are trying to get information from an interviewee, you should design more open questions like these:

Tell me about other books you have read that you enjoyed…

What puts you off reading a book?

When do you tend to read?

Avoid leading questions

What do you like about this book? This is a leading question that assumes that they like something about the book. You may be subconsciously or perhaps overtly trying to get them to answer a question in a particular way. Either way, it leads the interviewee. Instead, you can ask something like Is there anything you like about this book? It is particularly important to avoid leading questions when you are trying to establish that the problem exists. Can you get them to tell you that it is a problem rather than you suggesting to them that it’s a problem? So, for example, you are interviewing a parent and you want them to tell you that they cannot get on with their chores because little Billy is demanding attention. Your line of questioning will ask them to talk to you about the last time they were doing a chore while in charge of little Billy. If you do not detect any frustration, you ask them about the time before that and then the time before that. If there is any pain there, it will come out.

Avoid hypotheticals

If we put these features into this app would you download it? What people say and what they do are totally different. Instead, you will dig for real evidence of them having done something similar, to draw a parallel. When did you last do that? is a good question to start the interviewee talking about a real example. Ground your conversations in real examples.

Simple questions generate clearer answers

If you ask complicated questions, they will be difficult to answer as the interviewee will have to work out what you are trying to ask before they can answer it! This will generate complicated answers that you will then need to clarify and break down further, wasting time and energy. For example, avoid asking multiple questions as one question, for example How did you feel and what did you do? It is easier to answer if you split them into two questions. Use plain language, so rather than ask What is your strategy for dealing with this eventuality? you can just ask What did you do when that happened?

Write your big questions first

For the purposes of writing the research questions and completing this step, it is best to focus on the big questions first. Think of these questions as a funnel, where you start more broadly and then get more specific based on the answers you get. You may see that some big questions in my examples are closed, soliciting just a yes or no answer. When you actually carry out the interview, you will add in follow-up questions to dig out all the information you need based on the answers you get. At this stage, I am not going to write down all the follow-up question possibilities as I want to keep it simple and make sure that the most important areas are covered. You will not be able to predict these follow-up questions as they will be a spur-of-the-moment response to what your interviewees say. We will discuss how to follow up on those questions and dig more in Step 6 (Conduct interviews) when we actually carry out the research.

Ten is a good number

Try to keep within a maximum of ten big questions overall. This does not include all the follow-up questions you will add in during the conversation. If you find that you have more than ten, then either:

• Your value proposition and hypothesis are too complicated and will not all fit into one research interview or
• Your questions go too far beyond just the value proposition and hypothesis or
• You have gone beyond the main ten and are counting all the follow-up questions too or
• You have too many questions for one research session, so prioritise your top questions and then if need be, hold another round of research. You will probably find that your second round will be far more focused and may even have taken you off in a different direction as you will have learned a lot from your first.

Section A question tips

Does the problem, need or desire exist? As you will see in the above example, the first set of questions are around whether the problem, need or desire exists as you envisage it. The potential test trader interviewee is being quizzed on where they have sold their product/service to date and whether they have ever sold through or considered a retail opportunity. They are also being asked what appeals about each particular opportunity. All these questions will unlock information around the first part of the value proposition statement which is about who the proposition is for and what needs/desires they have.

They also help you to understand how big the problem, need and desire is, which is really important to establish. You need to know whether it is big enough for them to do anything about.

To avoid getting into hypotheticals and to make this whole conversation as real as you can, use these first questions to get the interviewee to start giving you real examples and stories about their recent experiences. You can continue to refer to them in the rest of the interview.

Try to write questions that get the interviewee to tell you the story of what happened, rather than just a straightforward account. If you can encourage them into storyteller mode with the questions that you ask, then they will become more animated in their responses and you will get greater insight of their feelings.

Here are some other examples that may help you to craft your own questions: In the taxi app example, you might ask:

Have you ever had a poor experience from a taxi company?

When did that last happen?

Can you tell me about what happened?

Do you have any other examples you can tell me about?

Again, using these types of example-based questions, you are getting them to start to tell you the story and you will be able to assess whether it is a problem or need from the way they talk about it, and you will get a good sense of how great that problem or need is.

Another good tip is to try and get them to tell you what the problem, need or desire is rather than you telling them. The issue with you telling them, is that you may be leading them to an answer that you want to hear. It might be that the problem you have identified is not as big as other problems they have. If you ask a suitably open question, like this one about the poor experiences they have had in general then you will hear whether your problem features and if so, whether it is one of the most important ones or not. Of course, if they reel off lots of problems, you can ask them to prioritise the problems that they have listed.

It is a bit of an art to get them to tell you what the problem is and you will get better at this over time.

Later on, as you get ready to carry out the research in Step 6 (Conduct interviews), you will need to work out how long to spend on each section. This is a classic place where interviewers over run as they probe to get the interviewee to tell them what the problem is. They find the interview time has gone and they are still in section A!

Section B question tips

Are the existing solutions good enough to solve their problem/meet their needs or desires? Now you’ve explored the existence and had some indication of the level of the problem, need or desire, you can move into what they did to solve it and how well that solution served them. Now it may be that they used a pre-made solution, or maybe they cobbled together their own solution. It is also possible that they did nothing at all and just suffered. Whatever the answer is, you need to write questions that delve into how well that worked for them. By doing this you will get more information about whether there is room for another solution. If they are not totally satisfied with today’s solutions, then there is an opportunity for you to give them something better. If they had no solution and just suffered, it is not a given that there is an opportunity for you. The opportunity only exists if the pain, or desire, associated with having no solution is big enough for them to do something about.

In the above indoor market, potential test traders example, you will see that the questions are going to get them to talk about other sales experiences. If they have no previous experiences and have never looked for premises before then you will not be able to pursue this line of questioning, so instead you could try to uncover any previous thoughts and feelings that they have by asking something like Why have you not looked for premises to date? That will help you to see their desire and give you a gauge of how strong their desire is. You will also hear all about their concerns which is part of your assessment about current solutions.

By talking through what they did, or thought about doing, you will also get information about your competitors. Remember here that ‘competitors’ is used in a broad sense as discussed in Step 1 (Write hypothesis), in the ‘unlike’ section on value proposition statements. Whatever they did to address the problem is your competitor – even if they did nothing!

Here are some generic questions you can use here in section B. They have just told you about a problem, need or desire, so these questions will help you find out whether the current solutions on offer are up to scratch:

Through these answers, you should also get some help on what your category of product is. Listen carefully to how the interviewee describes their solution as they may tell you what category their solution was in. You should hear about the job that it is doing for them, which may change the category of product that you wrote in your value proposition statement. For example, thinking about a gaming app for children, this could be in the category of ‘things to occupy your child while you need to do something else’ rather than just the category of being an app.

If you are concerned about the interviewee’s ability to recall what they did to the level of detail that you want, or about whether you will be able to imagine what they describe, then you can ask them to do some preparation before the research interview. If it is appropriate, you can ask them to bring information about the existing solutions they use; for example, if you are looking into online learning products, you can ask an interviewee to bring a list of current websites and apps that they use; or you can ask potential market traders to bring any photographs of stalls or shops where they have sold their products before and so on. Maybe it is relevant for them to keep a diary over the past week that they bring with them and talk through at this point.

Section C question tips

Are they prepared to take the necessary actions to get another solution? This section is going to help you with your risky assumptions around the action that you need the interviewee to take so that you can reach your goal. You need to ask the interviewee to give you examples of when they have taken the sorts of actions you are looking for them to take. This will give you evidence that they would be prepared to take the actions you need them to. In some cases that is easy, for example with the indoor market project, you can ask the potential test traders whether they have spent any money on a sales opportunity before, or taken a gamble and tried out a sales opportunity in an unknown environment. If the actions you need them to take feel very new, then you can try to find a parallel situation. For example, if you are looking to be the first-to-market with a taxi app, you will not be able to get evidence that they have paid for taxis using their mobile before. However, you can ask them about other things that they have paid for using their mobile. Dig into those examples and try to find real incidents where it was efficient for them to make payments from their mobile. And there is your evidence! Of course, there are all sorts of other actions, such as whether they would be prepared to download something or plug something into their existing gadget, or consider an alternative way to do something, or change suppliers and so on.

If they did adopt a solution from a third party, get them to tell you the story of how they came to adopt it as this will generate insight around how you can attract your future customer and/or users. How did they know that the solution existed? Can they talk to you through the signup process? These sorts of questions will give you more understanding about your target customer and what job they are trying to get done.

Here are some example questions for this section:

Tell me how the final decision was made about the option you chose?

How much did it cost?

Have you ever spent any money on anything else like this before?

By asking these questions, you should be able to discover whether there were any influencers (think back to the ‘people map’). You may also find that someone else paid, which means that the person you think is your customer may actually be your user.

By answering this last question, (whether they have spent money on anything like this before), you will discover which category they put your idea into. Think back to the value proposition statement where you reference which ‘category’ you are in. Note that your customer and/or user may have just told you which category they see you in and potentially which job you can help them to get done. They are telling you the benefit you can bring them, so listen carefully!

More example questions for this section are:

If you have had information from previous sections, you can frame the question this way:

There is also an opportunity to add The Trilogy questions to find out what they liked about the process, did not like and would improve.

Section D question tips

How do they feel about your idea? Do you have an early design, a prototype, a beta, a half or fully complete solution? Well here is your opportunity to add in a section D. I know that some of you will be trying to find a business model for something you have already designed, built or thought through. Well if you have, and if there is time, you have a chance to open your kimono and give your interviewees a sneak preview!

You can do this only if you have exhausted questions around the value proposition and hypothesis and there is genuinely time left in the interview. If you move onto this section too quickly then you may not have fully understood what the customer and/or user needs and we know where that ends up – a solution that is not wanted. We need to find out everything about what they are doing and how they are feeling first before we share any of our ideas. Talking about solutions any earlier can cloud and bias their answers. I know how enticing it is to share a solution idea and I know that innovators cannot get there fast enough, but please be disciplined. I add this fourth section in as I understand the passion, but it must not be abused! So please, no rushing through the first three sections to get to this fourth.

If you do not have any ideas about your solution then please do stay that way. Don’t force it. But if you have some early ideas, you certainly can bring them out of the cupboard right now!

If you have an interviewee who has not answered positively to previous questions, you could still show them your potential solution as they may have interesting insight to give you.

Here are some example questions that work well in section D:

Have you seen anything else like this? This will flush out competitors that you may not have thought of and other offerings that you would not have thought were anything to do with your offering!

How would you describe this to someone else? This is a brilliant question to get them to articulate the benefits in their own words. Then you can add that straight back into the next iteration of your value proposition

Does this feel like something you would like? This question goes straight for the jugular. Observe body language carefully on this one as you may see a squirm which somewhat negates a positive answer.

I have been very surprised at the answers to this. I would have sworn that an interviewee did not like a solution but overall, they definitely thought it was for them and the opposite has also happened. So I would definitely ask this question if you are showing something.

Avoid overloading the interviewee

I recommend that you ‘drip feed’ the description of the solution so you don’t overload the interviewee. You do not want them to feel bombarded. Remember that this will be the first time they see an idea of your solution and it may take a while for them to take it all in. Whatever material you use, including just describing it there and then, it is best to try to keep your descriptions to less than three minutes. Videos are best at less than three or four minutes. I have used videos for up to seven minutes and that felt too long. If you find your materials are much longer, then you can take a break halfway through and ask relevant questions. The Trilogy format works well here too.

Competitors can provide great materials

It is a good idea to use material to bring a concept to life even if it is not created by you. If there is already a product, service or offering on the market that has some or all of the elements that you are considering, then you can use that as material in your research too. This will save you loads of time! You can also think about adding annotations to paper versions of other products so you can better explain what you are thinking.

Pre-recording to keep consistency

I am a big fan of using pre-recorded descriptions so that what you show and the way you show it is consistent throughout all the interviews. So if you have something to show, for example a mock up on your computer, then you can make a video of yourself talking over it. There are plenty of free video tools you can use as well as taking video from your mobile.

Sharing the value proposition statement

I have shared printed out value proposition statements in research sessions and asked for interviewees to critique what is written there. This works really well as a structure. The interviewee will tell you whether they agree with pain points or desires you have identified, whether you have understood the situation in which they experience the pain or desire, whether you have understood the benefits properly and whether you have identified the competition as they see it. You can see all the areas of conversation this can spark, and it also works as a nice checkpoint for the first few interview sections. There is no harm asking a question a number of different ways to see if you can get at more information each time and as a way to check that you have heard the answer correctly.

In the story below I explain how I shared value proposition statements in interviews and had the interviewees help me rewrite them so that they were truly reflective of how they felt as potential customers. The project was to establish the branding and positioning for a new business consultancy.

Avoid getting into conversations about branding or features

If you are showing some material, it is best to leave off product names and/or branding. If it is there, then it will be a distraction and now is not the time to get into that discussion! Try to focus on the benefits that you can deliver them first rather than get into feedback around the product name or visual brand. If your imagined idea does not resonate that it doesn’t matter what you call it, what whizz and bang features you are going to add or how you are going to brand it.

Avoid conducting a usability test!

A usability test is where you have an interface and you want to find out whether it is easy to use, whether it is intuitive enough for the user to get from A to B. But it is way too early for that in the idea test! So, if you show them a working prototype, remember that just as you are not there to test branding or the detail of features, you are also not there to test whether it works as they would expect it would.

Even if you have a prototype you may decide not to show it so that you avoid falling into this trap. I have been in the situation a number of times and recommended that we keep the prototype away from the interviewee at this stage to avoid getting into the usability conversation as we really needed to focus on their needs and desires first. You can always just show a description of your intended solution with an explanation that shows how you are going to deliver the benefits that respond to their problems, needs and desires that you have identified.

Using materials in a video call

When I use materials in a physical interview, I tend to use printed out material and give the interviewee a pen to scribble their notes straight on to the material. If I have something to show them on the screen, such as a video or a mocked up website page, then I just turn around my laptop so they can see it.

If I am on a video call, I share a document online with them, send the link in the video call chat box and give them edit rights. So those who want to can make notes directly on to the document. Where relevant, I also show material by just sharing my screen, making sure that I can still see their face to gauge their response.

What do I write in this box?

Add the relevant questions against the relevant risks that you completed in Step 2 (Identify risks). Here is example of questions for the potential test traders in our example from the indoor market project. I have left in the Step 2 Risks so you can see how the questions correspond to them.