The Framework: How to Spot a Disruption Opportunity in the Wild

Those requirements all sound great, but how would you map them to real life? Even though we mentioned enabling technologies, it bears repeating that disruption is rarely about the technology itself. It’s more a story of a novel application of that technology, using an existing (perhaps well-worn) technology in new ways.

In other words, spotting a disruption opportunity requires that you keep your eyes peeled and be able to match patterns between spaces. We call this ability to spot such opportunities Disruption Goggles because it’s like seeing the world in a new visual spectrum: disorienting at first, but with time, you come to appreciate and even make use of this new perspective. Although Disruption Goggles don’t exist as real hardware that you strap to your face, you can develop your own, in your mind. Doing so takes practice.

To develop pattern-matching skills, get into the habit of breaking down a business or situation to get to its core. You begin to see that “situation X is, deep down, an example of situation Y.” If one field has solved a problem, and you can pattern-match that old problem to your new problem, you can borrow the old solution. Why not? Someone’s already done the work for you, and because they’ve tested it, they’ve also found most of the edge cases. Keeping your eyes peeled is more involved. To begin, inspect your supply chain and workflow:

Play the “what-if” game with various components of your business

Really question all of your assumptions. The big question is: what if this piece were to suddenly cost nothing? This is another way of asking: what if we no longer had to think of this, but we could just take it for granted? Costs for bandwidth and digital storage have dropped to the point of being effectively free (see Chris Anderson’s Free for a deeper discussion of this concept), and in turn, the internet is full of podcasts, camera-phone videos, and cat pictures.

Find friction

Some aspects of your business are like driving a car while the parking brake is still engaged: you’re moving, but you’re hurting yourself in the process. Find those places and ask how much business would improve if that friction were to not just decrease, but completely vanish. The friction of the IT purchase-and-install cycle, which can drag on for weeks and months, drove a lot of cloud adoption.

Keep an eye out for sources of waste

As a twist on the previous point, consider any excesses you currently generate. Certainly, follow your reflex to eliminate waste and consider the cost savings thereof. Don’t forget to also ask yourself what would happen if that waste suddenly had no dollar cost nor other ramifications: what if you could generate tons of waste product, but never have to clean it up? Better yet, what if that “waste” were suddenly available for sale as its own revenue-generating product? What if you could make money handling others’ waste? Commercial document-destruction companies are an example: people pay them to shred sensitive documents, and those services turn around sell the paper shreds as filler (instead of paying for disposal).

Note underserved markets, and consider the full definition of “underserved”

The term “underserved” is usually a business-speak euphemism for smaller market spaces and budgets that the incumbents won’t touch due to perceived issues of scale, capacity, or need. Don’t forget the literal meaning of “underserved,” which is: “people who are unhappy with the status quo or whose needs are ignored by incumbents.” There are entire markets, right in front of you, that don’t fall into the neat age/gender/location buckets typically used to identify an audience. For example, AirBnb serves the (clearly sizable) “want to travel but don’t like big chain hotels” market.

In his book, Christensen explores cases in which the newer players grew to completely swallow the incumbents (and, as a more recent example, people look at how Netflix crushed brick-and-mortar video rental services) but note that you don’t need to dominate or eliminate an existing market to be successful. Sometimes, you create an entirely new market for yourself and you stay there.²

Constantly ask what you can outsource

Sure, you could run a completely self-sufficient, vertical business by building every little piece yourself from scratch, but why? There exist specialist firms that have already mastered some aspect of what you need to achieve, and there are only so many things that one company can do exceptionally well. Explore engaging third-party services for anything that’s not your core competency. Stack them like Lego bricks to build your foundation and, over time, you focus your energy on your secret sauce.

Fill gaps, literally and figuratively

In a twist on finding an underserved market, you can create right-sized offerings based on the gaps left by larger competitors. While vetting services for an international move, a friend noticed that a large commercial shipping company offered small gaps of space on its international flights. These gaps were spaces that are too small to fit any additional large commercial orders, but they were more than large enough for consumer shipments.

Spot artificial (or “crumbling”) barriers to entry

Note your barriers to entry to a given market, and then ask what it would take to jump or circumvent them. When people see that phrase “barrier to entry,” they most often think of dollar cost. It also includes gatekeepers (people who charge tolls along traditional routes, such as record labels) and credential requirements (for example, a computer science degree to write software for a living).

Play matchmaker

Some people are natural connectors: they meet lots of people, get to know their needs, and introduce them to others when the circumstances fit. If you know (or can find) people of two groups, and these groups have trouble finding each other, you can build a business. These plays are sometimes called multi-sided markets and we’ll explore them in detail in a few pages. In construction, general contractors pair people with specialized skills (plumbing, painting, laying tile) to people remodeling their home. Staffing agencies pair people who want jobs to companies with open positions. What are your opportunities to help people connect?

Once you’ve tried these on your own operations, extend your reach to hypothesize about other businesses you encounter. Think of your competitors, yes, but don’t forget the everyday: if you’re waiting in line at the grocery store, or at the airport, or anywhere else, put on your Disruption Goggles and see what opportunities you can find. Try to fold the preceding exercises into your daily routine. It will feel counterintuitive at first, but you’ll soon be well-tuned to spotting opportunities for disruption, and the disruptive forces that will drive them.

As you look through the list of exercises, you’ll see three themes:

• Many of these pointers involve a forced efficiency of small scale. As a small (or otherwise disadvantaged) player, you need to make the most of limited resources. You’re on a constant hunt to get the most bang for your buck.

• Some disruptors are not yet within reach. Sometimes you’ll spot an existing opportunity, but in many cases, you’ll see one on the road ahead. You’ll want to position yourself to profit once it is available.

• Some of these disruptors overlap. They needn’t exist in isolation; you can mix and match.

Let’s take a look at two disruptors from the 1990s:

The internet and, specifically, the web

The web is both a technology and, one can argue, a business model. It demolished the barrier to entry for a lot of businesses and business models by creating a common infrastructure, protocols, data formats, and tools for consumption (web browsers). People can shop online and need only a browser and an internet connection. Merchants can sell goods online and need only to design a website. Neither party needs tools (such as an application on the consumer’s computer) that were specific to that particular merchant/consumer relationship.

Open source software

The open source movement of the 1990s started as people giving away tools they had written (especially lower-level development tools such as compilers and editors), and it grew into a tidal wave that changed how software was written, who would do it, and what was written. The ability to create software was suddenly in the hands of anyone who was interested, and they could use the web to distribute that software to anyone who wanted to use it or help them improve it. That, in turn, drove adoption of newer, friendlier programming languages—notably, Ruby and Python—which further lowered the barrier to entry for software developers. Having a formal computer science degree and professional experience as a developer were no longer prerequisites to writing software, which in turn meant that having that computer science degree was no longer a barrier to landing a job as a professional developer. Because people could write in their spare time, they could create whatever they wanted to meet their needs, without having to run it past company leadership for approval and market fit.

Both of these examples have technology at their core, yes; but to focus on the technology would be to miss the point. This is an idea that you’ll see again in this paper: the technology takes a back seat to what it made possible, which is precisely why the web and open source software are disruptive forces.

You might also argue that neither example feels disruptive. It’s easy to forget how much change they’ve brought about because we’ve come to take them for granted. The web and open source software are omnipresent and therefore invisible. If you look at the world through a 1990s lens, though, you’ll see how these two forces—especially when combined—redefined business and consumer experiences.

Disruptor: The Cloud

It should come as no surprise that the “cloud” makes this list: it provides access to on-demand, metered storage and compute power, with the added bonus of automating all of this through software.

It’s tough to understate the power that comes with this flexibility. Before cloud services, building new computing infrastructure followed this pattern:

1. Figure out what you need in terms of processors, memory, and storage. Tack on some extra for fault-tolerance. This effectively requires you to predict the future of your business so that you get the right amount of hardware.

2. Shop across multiple vendors. If you’re small enough, struggle to find someone who will actually take your call, and who won’t bump you down the list when a larger customer comes along.

3. Get internal approvals, which probably require some discussion and justification.

4. Place your order and wait for delivery.

5. Deploy the hardware to your datacenter and install the operating system.

6. Repeat the process a couple years later, as improved hardware hits the market in time to match your increased technology demands. Try to repurpose or sell off the old hardware as you do so.

This process could easily take weeks in smaller firms and several months in larger organizations. Heaven forbid you get it wrong. At best, you have overbought and now have hardware gathering dust. At worst, you have underbought and limp along with less compute power than you needed, potentially crippling your business or slowing time to market. In both cases, you punctuate that disappointment by having uncomfortable conversations with company leadership on how things went awry.

In contrast, using a cloud service is far more streamlined:

1. Sign up for a cloud hosting service and provide a credit card number

2. Click through menus to choose your operating system, processor, storage, and memory.

3. Click “Launch.” Enjoy your new hardware within minutes.

Because the (virtual) hardware arrives so quickly, you can scale up as you need. With a little elbow grease and your preferred scripting language, you can write tools to bring up a fleet of servers by entering a command. You can also turn off individual services at will, and those machines just disappear. You stop paying for them the moment they go offline.

This fits nicely into our extended disruption model. In terms of technologies, the cloud is all about datacenters and computers. The new/not-new twist appears again: businesses have previously used computers in datacenters, but the cloud approach opens up cheap access to robust computing at a fraction of the price and time, creating a new market segment.³ The small business cannot only rapidly deploy and scale its computing footprint, it can also experiment with infrastructure in ways that were previously far too cost prohibitive. Want to try two large servers instead of four small ones? Deploy both sets for a couple of weeks and run a side-by-side test, tear it down when you’re done, and pay only for the time they were running. You simply can’t do that with physical hardware.

There are two value networks at work here: hardware makers and technology professionals. Both face an overwhelming force—a clear case of “get on the bus or get out of the way”:

• Hardware makers have been forced to adapt to a changing and concentrated customer base for datacenter-class infrastructure. Sellers that do adapt can benefit from simplified sales channels (fewer customers, who have larger budgets) and reduced cost. Sellers that do not adapt will be disrupted and their small-business clientele will move away.

• Technology professionals face a similar crossroads. You still need smart, talented people to lay out and support your cloud-based technology, but you will need different skill sets and likely fewer people to do so. We have both witnessed IT shops in which large teams have permitted people to specialize. Those who used to focus on capacity planning will see a shift in role to more of a just-in-time model. Those who specialized in planning physical space and arrangement of hardware in the datacenter ⁴ will need to learn different infrastructure skills or move on (perhaps to work for a cloud provider).

As a benefit, cloud-knowledgeable IT teams should enjoy expanded opportunities as well as freedom. They no longer need to drive to a datacenter at 3 A.M., which opens them up to smoother on-call rotations as well as living anywhere in the world. Independent practitioners in this space can therefore take on short projects or even long-term support arrangements with clients all over the globe, all without leaving their back porch.

Disruptor: Predictive Analytics and Machine Learning

This disruptor goes by many names: some say “data science,” or “predictive analytics,” or “machine learning.” There are others who go for the old-school term “data mining.” A few will argue that big data belongs in this list, though others will say that term is more about storage. More recently, we’ve also seen the term artificial intelligence making a comeback from its humble 1950s origins. Even more confusing is the ongoing debate as to which of these terms are peers and which are supersets of the others. For the purposes of this paper, and for brevity, we’ll use the term machine learning to encompass this wider spectrum of terms.

Whatever you call it, it’s the process of analyzing data for fun and profit: you take some raw data, tweak it, and develop a model that will provide you more insight. The business focus is often predictive in nature: “How many widgets do we expect to sell over the coming months?” “Will this flight arrive on time?” Despite the naming confusion, and the hype, the attention is well-deserved: businesses that analyze their data as such improve their chances of making more money and otherwise improving their standing.

It’s also nothing new. We got our first taste of this working in investment banks in the late 1990s and early 2000s. Traders have always used data in an attempt to predict the future—that’s how they make their money, after all—and by the time we started our careers, a special kind of trader—the quant—ruled the roost. These people blended high-end math with custom software (the combination of which we’d call machine learning today) to make money at high speed.

Machine learning disrupted trading because the quants’ models, loaded up on historical data and rules the quants put into place, would make decisions and place orders far faster than any human.⁵ Machine learning is therefore the latest evolutionary step in a trader’s job, from shouting out orders on an open-outcry floor, to typing orders into a computer, to writing the code that places orders in lieu of a human. In this latest step, the quants deploy machine learning code that reacts to market conditions and—most important—makes decisions without direct human interaction.

Machine learning has since extended beyond the trading floor. The hardware required to handle all of the data in a timely fashion, once considered high-end, has hit consumer-level prices, and it’s even cheaper if you get virtual kit through the cloud. Analyzing the data once required translating statistical models from lengthy equations on paper to code (in C or C++, no less) but today there are ready-made, free-to-use implementations that are production-ready (Python’s scikit-learn and Google’s TensorFlow are just two of many examples). Learning resources about machine learning, once the domain of university-level courses, are readily available. Tying all of those together is that companies are collecting lots of data, simply because so much activity these days occurs in electronic form.

This means that we now have cheap access to machine-driven decisions and also reasons to analyze our data. There is still a high bar for success, but with such a low barrier to entry, it’s much less expensive for people to try machine learning on their problem and see whether they get any traction.⁶

Why, though, is machine learning considered disruptive? Consider what software does for human activity: we get to free humans of certain dull, repetitive activity, and to perform that activity with both scale and speed. This also holds for ML in that we can take humans out of the loop for certain matching or prediction problems. Machine learning tools can find the patterns in mountains of data, far beyond what humans can hold in their heads, and far faster than humans can calculate. This is less a matter of replacing people (as software so often does) as it is about getting answers that are beyond one person’s abilities.

Just as with software, your barrier to entry is that you must have a problem that machine learning can handle. Even worse is that you don’t always know whether machine learning can handle a problem until you’ve tried. Even with that caveat, after you’ve developed a working solution, it’s very powerful. It’s also cheap to try.

As real-world examples, consider Netflix and Amazon. Both employ a recommendation engine that will suggest more movies or goods to you based on what you have already seen or purchased. Anyone who runs a store or similar operation knows the power of happy, repeat business, and a well-tuned recommendation engine certainly makes that possible.

Recommendations aren’t new. Before machine learning–driven recommendation engines, though, they were broad. They didn’t target an individual’s tastes, but some (perhaps somewhat arbitrarily drawn) demographic: “If you saw film Foo Part 1, you’ll love Foo Part 2!" Well, we hope that people who saw Foo Part 1 will want to see its sequel, but on what are we basing that (other than the hope that people want to see the next story in the series)? And what else would they like, if only they knew about it? From broad recommendations you can get only…additional broad recommendations.

Machine learning–based recommendations are tailored to you, based on your activity. This precision leads to a stronger chance of recommending content you’ll like, which is content you’ll consume, which is more money in the pockets of Amazon and Netflix. To do this at the scale of Amazon and Netflix—consider the size of their respective customer bases and catalogs—you’ll agree that tailored recommendations are simply not possible if done by hand.

Another example of machine learning–driven disruption is Public Good. The organization partners with news organizations to enable people to donate money to causes, based on an article they have just read. CEO Dan Ratner describes the company’s business as a form of “ambient activism,” reaching people in the moment that they would want to act. Public Good uses machine learning to classify an article in real time and tie it to a cause, so that by the time the article has loaded, a button on the page knows where to take people who wish to donate.

Public Good could certainly exist without the power of machine learning, but in that case, they would require a lot of a different kind of “ML”—manual labor—to label content and connect it to causes. They would also need people on staff around the clock, to keep an eye on the news cycle and classify articles just as they are published. Instead, thanks to machine learning and some extra code, this 10-person company can scale to any number of publishers at the drop of a hat and classify articles even while they’re all at home and asleep.

Disruptor: Multi-Sided Markets

What do dating sites, credit card companies, and stock exchanges have in common? They all help interested parties meet each other. In particular, they help pair up parties who would otherwise have trouble meeting. You might have seen this concept described as N-sided markets, or the broker model, or, the term that we use, multi-sided markets. ⁷

In a multi-sided market, there are (at least) two types of people who would like to meet, and a broker who sits in the middle. The role of the broker is to attract those people and provide some space—real or online—for them to connect. The broker’s value proposition is that it provides a single, well-known, established place for people to meet.⁸ We can also explain the value proposition by considering the alternative, which would require individuals to find and trust other individuals:

Dating sites

A person of Group A wants to forge a romantic relationship with a person of Group B (note that A and B may be the same group!). They could wander around, asking everyone of Group B whether they’d like to connect …but that would be time-consuming, inefficient, and even a little creepy.

Credit card companies

I want to buy something but don’t have cash on hand, you want to sell something to me but don’t know whether I’ll pay you back. How do we establish trust such that I get my item but you know you’ll get your money?

Stock exchanges

I have shares to sell, you want shares to buy. How do I find you, and how do I trust you’ll have the money? How do you trust that I really have the shares?

Brokers must spend money to build and maintain the interaction platform, and they must attract both parties in sufficient number. After they get there, though, they have nearly risk-free revenue because they carry no inventory and (with a well-designed platform) their ongoing maintenance costs are low.

Multi-sided markets are hardly new. David S. Evans has been writing about them since the early 2000s, and some of his sources go back even further than that. Thanks to technology, though, you can develop multi-sided markets in other realms. This is why multi-sided markets make our list of disruptive forces.

Still not convinced? Consider ride-sharing services Uber and Lyft. By mixing omnipresent smartphones with GPS technology, these companies are able to pair up People Who Own Cars and Have Spare Time with People Who Want to Get Somewhere. True to the model, these brokers don’t carry inventory⁹ and they provide some degree of trust between passenger and driver.¹⁰

Similarly, AirBnb (connecting People Who Have Spare Space with People Who Want a Place to Stay) and even eBay (connecting People Who Want to Sell Random Things with People Who Want to Buy Random Things) serve as brokers. Although technology made all of these companies possible, that was really a supporting concern to the primary goal of connecting disparate groups. These companies have found ways to disrupt their respective fields—getting around, getting a room, getting physical goods—by inserting themselves between people who wanted to meet but couldn’t find a way to do so.

Disruptor: Self-Publishing

People have wanted to share their creations for as long as they have expressed themselves. Until the last few years, this meant haggling with some sort of publishing house that served as the gatekeeper to distribution. Book publishers, record labels, radio and TV stations, and film studios (we’ll collectively refer to them as “publishing houses”) arose to address the complications and complexities of distribution.

Being a talented creator wasn’t enough; the means of creation were expensive for individuals. Cameras, recording equipment, printing presses, broadcasting, and the like all required significant investment that only established firms could do at scale. There were independent publishers that would have the occasional hit but, for the most part, a few large publishing houses controlled the means of production. In turn, and in the quest for profit margins, these central publishing houses also controlled which content would be produced and distributed.

Self-publishing is not a new concept, but it has often sat at the extremes of low-quality (using low-cost, consumer-grade gear and therefore looking unprofessional) or exclusive-and-out-of-reach (a few established creators who could afford professional gear, which effectively made them their own publishing houses). Recent advances in technology have given entry-level creators access to professional-grade creation tools and distribution networks. This has led to a radical shift in how creators reach their audience and, therefore, how they get paid.

There’s not just one company causing the disruption. A host of firms have altered the playing fields in their respective content types: Amazon, Spotify, Google (via YouTube) and Facebook/Instagram. They have all in their own way made it almost trivially easy to create content.

This is another example of how the technologies weren’t really new; they were just used in new ways. When Amazon and other booksellers launched support for self-publishing, all an author really needed was a word processor. One example of this is the highly successful Silo series by Hugh Howey, which began as a series of self-published short stories.

Other forms of media required additional technology, but the tech offerings caught up quickly. Everyone with a smartphone has a decent video camera in their pocket. Inexpensive video and audio recording and editing software had already been on the desktop for years. YouTube and Spotify have made distribution as easy as uploading a file.

It’s also tough to overlook social networking’s ties to distribution. You have friends and acquaintances with whom you share interests, so you have a ready-made network that will help spread the word on your latest creation.

Self-publishing has elevated all this free and low-cost content to become a legitimate market segment in its own right. The consumer has a multitude of alternative channels through which to discover and consume content. Consider online reviews on YouTube: if you can think of a product, you can probably find someone who has reviewed it. Even though the creators do not always directly charge for this content, there are indirect revenue opportunities: paid reviews, marketing their content for sale elsewhere, or cross-promotional opportunities with their other endeavors.

One such example is Andertons music, a family-owned music store in Guildford, Surrey in the United Kingdom that has created a video series of music gear reviews. Companies that make the equipment sometimes compensate the Andertons crew for the reviews. Furthermore, guitarist and bandleader Rob Chapman has used the network effects of these reviews to benefit his guitar company (which crowdsources features of the new guitar models they build) and market his band. The value network component is straightforward: it is the social network of consumers and distributors of this self-published content, and the app providers like Amazon and YouTube, who are incented to make this content available either via direct sales or advertising.

In terms of disruption from self-publishing, there are four main effects. Existing content is cheaper or free because it is far easier to produce and disintermediation has further reduced costs. Entirely new types or subgenres of content provide value to the creators in the form of promotion as well as entertainment even if the content is cheap or free. Incumbents will still be viable for some time but are likely to lose ground in the long term as distribution becomes increasingly independent. Finally, a small group of labels/studios/publishers no longer get to decide what content sees the light of day; the consumer wins in the form of free or inexpensive high-quality content (and yes, some not so high quality, as well). The onus will still be on individual creators to work hard to market their work and build an audience, but the leverage they have is far greater than ever before. Of course, this cuts both ways, because all the smart creators they are competing with are also afforded the same options.

Another way to think about the “anyone can be a creator” concept is as a long tail: rather than a tight concentration of content in the hands of relatively few distributors, there is a broad variety of content to meet every need.