Identify risks
KEY POINTS
Identify where you have a lack of evidence on high-impacting elements of your hypothesis
Spend time carrying out some desk research to see if there is any other evidence that already exists
Categorise your riskiest assumptions against the four main sections of risk
Many innovators fail to recognise that their early-stage assumptions may contain different levels of risk and likelihood. In some cases, they think these cannot be addressed until the product or service is launched. Well they can! In this step we will look at identifying risky assumptions at an early stage.
A risky assumption exists where there is something of high impact to the success of the idea and where evidence is low, so they are likely to be the elements that you will feel the most uncomfortable about. The riskiest assumptions will be those things that keep you awake at night as they could cause your whole business model to fail. They are the things that can trip you up later on, once you have spent time and money developing your idea and there is no need for this to be the case.
In this step, we make sure that the riskiest assumptions are uncovered, rather than burying our heads in the sand. Once we find them, we write questions in Step 3 (Create questions,) for carefully profiled group(s) of customers and/or users who we find in Step 4 (Find interviewees), which will get evidence for or against each of those assumptions.
Four main categories of risky assumptions
In nearly all the early idea tests that I run, the riskiest assumptions map into four sections (labelled A, B, C and D). You can use these sections as a shortcut and a checklist for identifying the risks in your idea. You will see that the sections are a natural follow-on to the work that you have already done in your Step 1 (Write hypothesis) work. These are the four sections of risk:
Section A: Does the problem, need or desire exist? Is there a group of people who have the problem/need/desire that you think they do? Are you right about the benefits that they are looking to get from a solution?
Section B: Are the existing solutions good enough to solve their problem/meet their needs or desires? Is the problem/need/desire big enough for them to be looking for a different solution? Are your thoughts about the competition and the (value proposition ‘unlike’) landscape correct? Is there an opportunity for another player in this market?
Section C: Are they prepared to take the necessary actions to get another solution? Will they put their time, effort, money or other action towards adopting a different solution?
Section D: (Optional) How do they feel about your idea? Is your current thinking about a solution on track? (if you have got that far)
Section D is optional for those people who have progressed their thinking already. It is fine if you have not started thinking through your solutions yet but for those who have, you will be in front of customers and/or users so it would be daft not to get some feedback if you have time after you have gone through A to C.
If you look at sections A to C in particular, you will see that they are all high impact. That is to say that if your assumptions were not right, then the whole idea could fail.
Prioritising and finding existing evidence
Prioritise the assumptions that have the highest impact
Ask the question: ‘If our assumption is wrong, what would the impact be on the success of our idea?’ You can choose from three answers:
- HIGH (H) This idea is not going to fly
- MEDIUM (M) Could be problematic but this is not yet an idea killer
- LOW (L) Does not feel too dramatic at this stage
If you are answering ‘low’, then you do not need to address the assumptions at this stage. You may be in the weeds and getting bogged down in unnecessary detail. If you are answering ‘medium’ then you could include them in here if you do feel particularly uncomfortable, but they could be left until a bit later on. If you are answering ‘high’ then yes, you must include them now!
No medium or high risks? Think again
If you honestly feel that you have no medium or high risks then you should already be confidently building and delivering your idea … but you are not, you are reading this book! It is very common for innovators to take their new ideas to market with risky assumptions that they could have reduced before getting that far. So perhaps it is worth considering whether you are being too optimistic on your scoring. Put your low-risk items into the four categories and go through the test. Every time I carry out research, I learn something that surprises me.
Do not focus on solutions (D) before the underserved problems, needs and desires (A to C)
Please avoid putting too high a risk rating on assumptions that are related to how to deliver solutions at this stage. Right now, it is far more important to make sure that there is a target user and/or customer with an unmet or underserved problem, need or desire before getting into the detail of the solution. A customer and/or user may love your solution, but they may not have the need or desire for it so you could end up with a great product or service with no market.
You can see that in section D, you get an opportunity to test out your early ideas, but not before you cover sections A to C.
Of course, some ideas have a certain technology or solution-based element at its heart so there will be some technology-related assumptions that need working on. This will be the case where you are working with a particular technology and trying to find the best user cases for it. Take the taxi app example, the idea is all about using an app and the functionality that it can offer to a customer, so the fact that the customer is already an app user and prepared to use the relevant functionality is most important.
It is too early to identify timeframes and cost to build as risks
I have seen innovators get very hung up around whether they can deliver their idea in a certain timeframe, or to a particular budget. This is a major distraction at this early stage. It is likely that you will not have evidence of the customer need and desire yet, so how can you know whether it is feasible to deliver the solution within twelve months?! Allowing yourself to go down this path will also tempt you into doing a business case, which again, is too early to do as you won’t have your solution designed so you won’t know how much it will cost to deliver.
Spend time trying to find existing evidence
I often find innovators planning to carry out research for evidence that already exists. What a waste of time! So it is worth spending some time digging around first to see what evidence already exists to answer the high- and medium-impact questions you have raised. Perhaps you have some useful information from previous research that has already been carried out, if not by you, by someone else? It is time to do some good desk research to find any evidence that already exists.
Search like mad! Use all search mechanisms available to you on the internet. Is there any published research where you could find some answers? This could be data from professional research and insights companies or publications from industry specialist groups, published demographic data, press articles and so on. Are there other products, services or solutions that are in your space? What can you find people saying about them online? Google Scholar is great for academic papers that are full of references you can dig in to. Following references from reports in mainstream media is very useful as they often summarise and cherry pick information.
Example: Online research for evidence on the indoor market project
An internet search on ‘empty shops report’ revealed a wealth of data in numerous reports that are based on data from very reputable well-known research and insights analysts. A search on ‘empty shops projects’ reported the growing trend for artists using empty premises for exhibition and performance. A search on the area’s demographics and local economy led to reports commissioned by the council sharing all sorts of useful information. One could find the amount of new shops opening and the number of shops closing that helped to find out more about local traders and so on. You can see how all this data was valuable evidence for this project.
Learn from what has gone before. Have any other companies or competitors shared insights about their experiences, successes or failures? Check blogs, articles, business books and so on. For example, if you are looking at opening a restaurant, there are many examples that have been written about. In particular, you will find a lot from high-profile entrepreneurs who love to share their stories. There are plenty of people who write about their and other people’s successes and failures. Learn from them. Talk to people who’ve tried it before. (I worked with someone once who said that it did not matter what you did, but what you wrote about what you did.) Maybe this will help you also work out whether the environment has changed and you can do better now. Filter out what you think applies and what doesn’t.
Despite all this good learning, you need to be careful about applying it directly from one customer segment to another, as you cannot assume that all people will behave in the same way. Here is an example of how it could be dangerous to assume UK users may be the same as US users.
Example: Education company wants to launch successful US product in the UK
An education provider already had a successful online education offering in the US. A lot of the assumptions were absolutely going to be applicable to the UK; for example, around how best to keep students engaged. However, there were some differences that also posed many questions about how different the reception by a UK audience would be. To start with, the education system is quite different in the UK compared with the US. Then there are geographic differences that affect how children are used to studying. The US is around forty times the size of the UK and the UK is roughly eight times more densely populated than the US which will affect all sorts of physical versus online behaviour.
If you do find that your idea already exists, don’t give up! It is still worth taking it through the test. In fact, it is quite comforting to see that another innovator also thinks that there is an opportunity! Maybe there is room for another player? Remember that those who are first to market in the short term are often not market leaders in the mid to long term.
Ask people around you if they have any evidence. Put the call out! I am not suggesting that you share your whole hypothesis or post your entire value proposition out to the big wide world; instead I am suggesting that you can ask some choice people from your professional and social circles whether they have any insight that they can share to support a particular element. Particularly in larger companies, there is often a wealth of evidence if you just ask the right people. There is so much insight held in pockets all over larger organisations and I see so much repetition where the research is duplicated to get evidence that already exists.
Here is a great story about digging for and discovering evidence internally which totally changed our direction of travel. It helps to show that data may be more available than we think.
Story: New internal evidence totally changed marketing campaigns to drive mobile phone picture messaging usage
It was a year after mobile picture messaging launched in the UK. The usage figures were way below what was projected. The second wave of camera-phones had arrived, so the camera quality was not bad and market penetration of those phones was at a level where our targets should have been possible. Around the company there were lots of theories about why people were not sending messages, but I needed to get some evidence.
My team embarked on some research. More casually, as we were out and about, we all observed strangers and our own social groups. We found that people were taking photos and showing them to each other but they were not sending them.
We carried out some street interviews with a short questionnaire. We were near a university so were close to a traditionally early adopting group of people. We found that they were not sending photos as it cost to send them and they were watching their pennies.
But here was the big discovery: I discussed this quandary with the business analysts who mined data from the customer call records. We thought maybe there were some answers there around usage patterns and so on. We discovered something that blew our minds: More picture messages contained GIFs (graphic interchange formats) than JPEGs (which are a format for compressing image files). Photos taken on the phone were all presented as JPEGs but the majority were GIFs which were animated or static images that had been created elsewhere. This was totally missed in the observation and interviews as we were so busy looking at photo-taking and sending.
Further research told us that usage was around expressing emotions (note the current popularity of emojis) and key events were a time when people sent these GIFs to each other. So we sourced a whole range of content from birthday cards to seasonal greetings and ‘I’m sorry’ to ‘give us a kiss’. Within these categories, we also found out that ‘edgy’ content was popular, so I famously had the management board sign off to ‘Bondage Santa’ whose cartoon walked onto the screen and opened his cloak to reveal…
We then flooded the camera-phone customer base with free animated content encouraging them to send it on, with a promotion that they could send their first five for free.
The result of what we did in response to this evidence got us to the highest number of messages being sent of all networks in the UK, despite having third place in overall customer numbers.
If we had not found all of this out, we would have spent time and money on campaigns to get people sending photos, when the real key to driving the revenue line was pre-created content. The risky assumption was that the customer desire was around sharing photos they had taken when actually the bigger desire was to express emotions and entertain each other, particularly given the quality of the cameras at the time.
While we did manage to boost usage, picture messaging has never really taken off in the way we expected. It was the quality of the photos, the difficulty receiving messages (as there were some user experience issues) and the price per message that were the main restricting factors for people sending photos. In time, one could send photos through instant messaging and upload them on social media sites. These were charged as data, as part of a customer’s data allowance, not as a cost per message. They also were sent and received using a different technology and by that time, the cameras had also significantly improved on the phones. Social media has tapped into the whole broadcast behaviour and people now share photos publicly in a way that we had not seen before. And now, we only wish they would just stop!
What constitutes good enough evidence?
I often get asked how to work out if you have sufficient evidence or not. This is, of course, a judgement call. There is no scientific formula I can give you. Perhaps imagine yourself in front of the toughest CEO or investor. Do you feel comfortable enough with the evidence you have to forge ahead? Does it keep you awake at night? Numbers are of course good, but not always possible. I often work with innovators who can’t give me the exact numbers but, for example, they hear a problem regularly enough to be sure that it is a large enough problem to address. They will tell me that they hear something consistently from calls to customer services; it is consistently in feedback from sales people and it always comes up in satisfaction surveys. Well that is good enough for me – that is probably a low-risk assumption. Ask them whether they could sleep at night knowing that time and money is going to be spent developing a solution based on the evidence they have provided, and their response will help you work out whether the evidence is good enough!
Even if you have some evidence for the highest riskiest assumptions, you may still feel that you should gather some more. If they are risky enough, then they will be worth double checking. In this table you will see how impact and evidence work together. You can plot your assumptions in this table to check the level of risk that you assign.
| RISK ASSESSMENT | Sufficient evidence | Insufficient evidence |
| High impact | Medium-risk assumption | High-risk assumption |
| Medium impact | Low-risk assumption | Medium-risk assumption |
| Low impact | Low-risk assumption | Low-risk assumption |
Using this process to manage other people’s opinions
Before we move on, I want to explain that this step provides a great opportunity to include hypotheses from other people if they have given you any opinions to date, or if you want to continue to get their buy-in to The Really Good Idea Test’s seven-step approach. I often hear from innovators (and have of course experienced first-hand) about the many views and opinions of others they work with. Surely not – a senior management team with a strong opinion?! In a venture of more than one person, this is kind of inevitable. There are always lots of opinions about what this idea should do, what it could be, who it is for and so on. Right now, you have a great opportunity to speak with your colleagues, investors and so on, and get their input to this process of identifying risky assumptions.
Here is an example of how that conversation can play out. The context for this idea is that you are looking at developing a way for school-based students to study a subject online. The theme here is pretty common: You get a point of view which is based on a sample of one customer and/or user who is close to the opinion-holder!
How to handle a typical conversation (using the online study idea at the educational company)
You: You have said before that you do not think many fourteen- to sixteen-year-olds would want to study a subject in an online environment. Why do you think that?
Your colleague: Because my kids told me they wouldn’t like to miss out on the social bits of learning a subject in the classroom.
You: Ah ok, so do you know any other kids of that age who also feel the same way? Perhaps who are from a different segment to your kids?
Your colleague: Yes, I asked a few of their friends that were over at the weekend.
You: I think that is an important hypothesis to test as we don’t want to get too far down the line and find out that our target users are not going to want to study online. I am going through a process right now of capturing all these important hypotheses from around the business and working out whether we have enough evidence to validate or invalidate them. Where we do not have enough evidence, I am going to go and get that evidence. Do you think that you have a broad enough bank of evidence for this hypothesis or do you think we should get some more?
The chances are that they will suggest you get some more evidence. You have seeded in their mind that even if they have heard it strongly from their own kids and their friends, while they could represent a segment, there are surely not representative enough of all fourteen to sixteen year olds?
It may be that you feel their hypothesis or fear is not quite as high a priority as something else, but if the challenge is causing you any discomfort (for example, this could be from your line manager or an important internal stakeholder), it may be that you give it a higher priority and tackle it right now.
Examples of riskiest assumptions
Here is an example of a set of risky assumptions using the indoor market project, focusing on the potential test traders. In this example, they are all high impact and there is very little evidence in existence, so they are the riskiest.
They are mapped to the four main categories of risk which you will see described in sections A, B, C and D. These four categories form a structure that you continue to work with through the rest of the test.
Here is another example using the taxi app idea.
| Four main categories of risk | Risky assumptions |
| Section A: Does the problem, need or desire exist? | That they have a problem with taxis being unreliable |
| Section B: Are the existing solutions good enough to solve their problem / meet their needs or desires? | That the pain is big enough for them to try something new |
| Section C: Are they prepared to take the necessary actions to get another solution? | That they are prepared to use a different taxi company |
| Section D: (Optional) How do they feel about your idea? | That they are positive about ideas on how it would work and what they would be charged |
Completing The Really Good Idea Test templates
Are you working on your own idea as you go through this book?
If so, then you should still be using the CORE template. Remember that you can download these templates from productdoctor.co.uk.
At this point, you should have already completed Step 1 (Write hypothesis), which you can see is shaded. Steps 3 to 5 are also shaded as you are not there yet.
You are ready to complete Step 2 (Risky assumptions) which is highlighted.
What do I write in this box?
Just write in your risks under each section. If you have more than one main risk in a section, it is a good idea to number them within the section as you want to address them individually as you move forward. Here you can see how to complete the Step 2 box with the indoor market project example.
Try it for yourself!
Remember you can download the template from productdoctor.co.uk
TOP TIPS
Take this opportunity to tackle those elements of your hypothesis that make you most nervous!
Use the four main sections of risk as shortcut and checklist for your riskiest assumptions
Enter your risky assumptions into the CORE template so you can easily move onto create corresponding questions in the next step
COMMON PITFALLS
Prioritising risky assumptions related to solutions rather than first focusing on customer and user problems, needs and desires
Carrying out unnecessary research where evidence already exists
Obsessing about delivery related risks when you don’t yet know what it is you need to build!