Chapter 8

Building and Running Crowdsourcing Platforms

Regardless of the objective, every crowdsourcing venture shares similarities and necessitates the completion of certain steps. Every time you build, launch, and maintain a crowdsourcing platform, you will need to execute a specific process. This chapter details that process by providing an overview, and then describing how to choose your platform's objective and scope, analyze its target audience and media environment, design its structure and interface, build the technology behind it, market it, measure its success, and learn how to improve it.

Overview of the Process

From a technology standpoint, crowdsourcing platforms are easy to set up and launch. However, maintaining them and ensuring that they are successful are far more difficult. Due to the diffusion and low cost of social media and Web 2.0 technology, almost anyone can build a crowdsourcing platform or hire developers to do so without breaking the bank. Of course, the more complex the platform, the more technology expertise is required and the harder it gets. Still, the hardest parts about designing and maintaining platforms are the fuzzy details and factors that are difficult to make sense of and quantify. These include choosing the right participants with the appropriate access to technology, attracting them despite geopolitical or cultural issues, keeping them interested and motivated over a long period of time, ensuring their participation is proving valuable, and making sure adversaries do not ruin your platform. They also include, in the words of the accidental philosopher Donald Rumsfeld, the “unknown unknowns.” In other words, getting the technology behind every platform is a type of science that anyone can master over time; but everything else about the platform is an art form that requires patience, creativity, and intuition about human behavior.

This chapter describes a process that will help you develop and master the art of building and running crowdsourcing platforms. The following is an overview of the process:

The following sections describe each step of the process in greater detail. Chapters 9, 10, and 11 teach you how to adapt and add to the process depending on your objective and target audience, and feature case studies that walk you through the process. As you read through this and the subsequent chapters, note that you do not have to execute each step in the process in a specific sequence. You will find that you will have to do some concurrently, or go back and redo an earlier step depending on the outcome of a later step. Also, be flexible and prepared for unforeseen problems, factors, and hostile audience reactions that force you to redo steps or modify the process.

Select Objective and Scope

Based on your task order or the problem you want to solve, select the objective(s) that fits best. Refine and narrow your problem of interest as much as you can to ensure you select the most appropriate objective.

Refine Problem to Establish Clear Objective

The platforms you build for security issues will accomplish one or a combination of three objectives: collect intelligence, solve problems, and influence populations. If this is your first platform, we advise against combining objectives because it introduces complexity and difficulty. As we describe later, it is better to modify the platform slowly over time to fit new objectives. For example, if your task is to collect information from inhabitants of a rural area about a series of local murders, your objective is to collect intelligence. If your task is to create a prototype algorithm that analyzes Twitter data to identify humanitarian disasters, your objective is to solve a problem. If your task is to persuade dissidents living under a dictatorship to organize and work with your government to overthrow the dictator, your objective is to influence the population. Perhaps you also need to ask the dissidents the best way you can help them overthrow the dictator, in which case the objective is to solve a problem. In most cases involving influence platforms, you will need to combine objectives.

After selecting the objective, continue narrowing and focusing the problem until it is clear and simple or split it up into smaller, simpler problems. Doing so is essential to running a successful crowdsourcing platform. If your target audience is confused about what you want them to do, your platform will fail to generate appropriate outcomes. If you want to collect intelligence about murders in an area, you need to ask participants about specific time periods, locations, and people. If you want to solve a problem as complicated as analyzing Twitter data to identify all types of humanitarian disasters, you need to break up the problem into smaller problems and ask people to analyze Twitter data to identify one type of disaster at a time. Subsequent chapters give more details about how to narrow and define problems for each objective.

Decide the Scope of the Platform

After selecting the objective and refining the problem, define the scope of your effort, which in turn will severely impact everything about your platform. Several factors comprise the scope, some that are very specific to the problem. A few salient ones are as follows:

• Need for secrecy—Determine if you care if anyone can find out about your platform, or if you want to maintain deniability. The level of secrecy and anonymity you desire will greatly impact who your participants will be, how they will interact with you, and what your marketing campaign will look like. If you want to collect information about narcotics smuggling from individuals in areas of Colombia sympathetic to the FARC (Fuerzas Armadas Revolucionarias de Colombia), you probably do not want to openly advertise your platform and intentions. Otherwise your participants may face danger, or you will receive misinformation from FARC sympathizers. In the case of hiding participants' identities, you can allow them to adopt fake personas when interacting with others on the platform.
• Need to minimize risk—Risk in crowdsourcing platforms exists in many forms. Recognize what the risks may be and how willing you are to live with them. One of the most salient is the risk that adversarial participants present to other participants and your objective. If your participants are made up of individuals from populations known to be hostile to your government or organization, you will likely need to hire more moderators.
• Financial resources and manpower—Budget the money and labor you can commit to the platform. The constraints your resources present will greatly impact your incentive structure, how much you can minimize risk, and the technological sophistication of your platform.
• Time—Crowdsourcing usually works best over a longer period of time, especially if you are dealing with hostile or suspicious populations. It takes time to attract participants, win their trust, and convince them to fully help out. If you must accomplish your objective in a short period of time, you will need to rely on more lucrative incentives and amplify your initial marketing push. If you have a longer period of time, you can combine objectives and relax the monetary value of your incentives and the force of your marketing campaign.

Countless other factors exist. As you come across case studies and examples of crowdsourcing, identify and keep a list of factors that may affect your platform.

Analyze the Target Audience and Media Environment

After deciding what you want your platform to do, you need to decide exactly who you want to recruit to use the platform.

The target audience is the group of individuals who you expect will use your platform.

The characteristics and numbers that make up the target audience may shift over the lifetime of the platform. So far we have used the word “participants” to describe the target audience. The target audience inhabits a specific media environment.

The media environment comprises the types of communication technology available to a target audience, and how the target audience and people around them typically use the technology and influence each other.

Determining the target audience and understanding them and their media environment is essential to the success of your platform. Formulating who comprises the target audience and understanding the audience and their environment will help focus the function and features of your platform, and the marketing campaign around it. Imagine you are a company like Coca-Cola that is trying to sell a product. One of the most important steps for selling a product is identifying who will buy it, how to change it to better fit their needs, and how to advertise it to them. Instead of a product, you are essentially selling a service to a specific group of people. You will not receive money in return, but you will receive information, solutions, and influence.

Determine the Target Audience

Your objective and problem will determine which individuals and groups belong in the target audience. In other words, your objective will determine the characteristics, demographics, and behaviors that define a given individual in the target audience. Figuring out who comprises the target audience is an iterative process. With security issues, your platform likely will focus on a specific physical region such as Southeast India, or a problem affecting a specific population, such as people interested in solving math puzzles. Cast a wide net and choose potential participants based on their location and relevant attributes. For example, if you are collecting intelligence from Southeast India about violent communist groups, at first blush the target audience could be everyone in Southeast India.

Then create a list of characteristics that you would most like your ideal platform participant to have to narrow your target audience. You need people who have access to and regularly use phones or the Internet. Use characteristics on the list to then refine the constitution of your target audience. Thus, instead of everyone in Southeast India, your target audience now becomes 18–40 year olds (an age group that is familiar with communication technology) in Southeast India with phones and access to the Internet. You may determine that another characteristic you would like your target audience to have is that they live in rural areas, because that is where much of the action you are interested in takes place. Your target audience is then refined as 18–40 year olds in rural areas of Southeast India with phones and access to the Internet.

Your goal should be to select a target audience of moderate size. The target audience can be larger only if you have massive resources, the technology behind the platform is simpler, and it is geared toward collecting specific pieces of intelligence from individuals. You do not want a target audience that is too large and is defined only by one or two characteristics because it will dilute your ability to harness them effectively. You will find it difficult to moderate the audience, incentivize them effectively, and keep track of the data they are supplying. If the target audience is too small, you will not have enough diversity within the audience to get novel information and perspectives, and you will be too dependent on only a few people. If you have the resources to run a platform for a longer period of time, you can slowly ramp up the size of the target audience. The most famous social media platforms such as Twitter and Facebook do precisely that.

Another reason for containing the size of the target audience and slowly scaling it up is that expecting a large number of people to instantly join your platform is not realistic. Humans around the world tend to adopt technology and new ways of doing things at rates defined by the technology adoption life cycle. Typically, a small group of individuals in a group called the “innovators”—about 2.5 percent of a population—will be quick to try something new, including a new platform. About 13.5 percent—the “early adopters”—will eventually try it out, followed soon after by 34 percent of the population, the “early majority.” The remaining 50 percent—the “late majority” and “laggards”—will slowly come around to trying it out, if ever.¹ See Figure 8.1 for a visual representation of the adoption curve.

Figure 8.1 Technology adoption life cycle

Consider how people buy Apple's newest product. A very small proportion of individuals line up around stores and wait overnight so they can get the product before everyone else. A few (which includes us) will pre-order it on the Internet and wait two weeks or so for it to arrive, because we want to make sure the product is not a dud and we are too lazy to wait in lines outside a store. A big proportion will wait for the buzz to die down a little and buy it when it is more readily available in stores, and lots of people have given it good reviews. The rest will slowly come around to buying it, if ever. You should be happy if 10 percent of your defined target audience joins your platform and about 5 percent of your defined target audience uses it regularly.

Analyze the Target Audience

After determining what the target audience looks like, you need to analyze them to further refine them and extract insights so you can build a platform they find appealing and useful. This step is extremely important and pivotal to your platform's success. You do not want, for example, to end up creating a crowdsourcing platform accessible only through iPads for the rural populace in Papua, New Guinea who do not have them. If you expect to run your platform for a long time, you will need to routinely re-analyze your target audience and their media environment, and make changes to your platform as appropriate.

To appropriately analyze the target audience you need to uncover information about their lifestyles, media consumption, and social media use. Essentially, you need to find out if the potential target audience:

• Have the knowledge, skills, and/or resources you desire, and if so, what are they exactly?
• Can regularly use communication technology to access the platform, and if so, what is the technology?
• Are willing to interact with you and/or the platform, and if so, how and to what extent?
• Will respond to incentives and thus prove helpful, and if so, what are the incentives?

If some parts of the target audience do not fit any of the preceding criteria, you can discard them and further refine your definition of them. The criteria are fairly broad and the information you need to address them depends on your objective and your guess about the appropriate target audience. Most likely, you will need to find the following specific information about the potential audience:

• Traditional demographics to get an idea of exactly who may be in the audience. Certain age groups, genders, or economic classes may respond to incentives differently or prefer a different-looking platform. Such data includes their age, race, gender, group affiliations, literacy rate, and income. Ideally, you should try to find out the subsequent information for a few major demographic groups, such as age (teens, 20s, 30–50s) and gender (male vs. female).
• Lifestyle information to understand the audience's daily lives and how your platform can integrate into them without disturbing them. Information includes their occupations, hobbies, community participation, and level of material consumption.
• Media consumption to understand what type of media and interaction they prefer and what competition your platform may face. Media refers to traditional media (TV, radios, newspapers, and so on), the Internet (news sites, games), and social media. Relevant data includes how many times a day they consume media and from where; what genre and type of media they prefer; which age groups use it the most; if they use, consume it, or share it with others; which websites they like best; how many hours they spend on the Internet per day on average; and how much they pay for media.
• Technology adoption to see how local cultural and economic factors impact the technology adoption life cycle. Understanding their behavior with all sorts of technology will help you create metrics that appropriately measure what is a good versus poor rate of participation at certain time periods. Relevant information includes sales of new technology, how much money they spend on technology, and their use of social media.

Feel free to look for more or less and different types of information as you see fit. The goal is to choose and understand the target audience well enough to create a platform that appeals to them, and hence works for you.

Analyze the Media Environment

Analyzing the target audience's media environment will further your understanding of not only how the audience may use your platform, but how people around the audience will affect their behavior toward your platform. The people around the audience are not initially part of your audience, but they may become so after some time. The aforementioned criteria and information go a long way toward analyzing the audience's media environment; especially, identifying what sort of media and technology exists in the audience's environment and how they interact with it or use it. See Figure 8.2 for a visual representation of the media environment. The following information will fill in gaps and provide you with information about the people and technologies surrounding the audience:

• Information about the two circles of people that surround the target audience. The people and the audience provide each other with information and influence each other's behavior concerning media consumption and technology use.
  • The first circle consists of people immediately around members of the target audience, such as their family members, friends, classmates, and colleagues. Understanding the audience's first circle can help in numerous ways. For example, if you want to scale up your audience, you can encourage participants to convince their first circle to join. Also, if your platform is controversial, the first circle may steer participants away from participating. It also consists of the media technologies the target audience uses on a daily basis.
  • The second circle consists of institutions, groups, and individuals that guide the audience's behavior, including schools, local governments, popular radio stations, and celebrities. It also consists of all media available in their environment. The second circle may subtly influence the audience to stay away from or to join the platform depending on how they perceive the platform. Identify who they are so you can track what they say about your platform or competing media.
• The general state of technology in their environment to assess the audience and their culture's relationship to social media, the Internet, and their willingness to try new things. In certain areas, you may need to provide people with phones or access to the Internet in order for your platform to work. Understanding the local relationship to technology and media will help you assess whether you should try and what you should expect from your efforts.

Figure 8.2 Media environment surrounding target audience

Get the Information to Do the Analyses

You may start to feel overwhelmed by the amount of information you need to simply get started with crowdsourcing. However, do not be discouraged. A lot of the information is surprisingly easy to come by, and you can probably make do without the missing information. The following are some of the best and easiest ways to get the appropriate information:

• Hire subject matter and area experts—By experts, we mean people who have lived in the locations, know the audience, their culture, and their media environment. We do not mean “pundits” who write lots of articles or tweet all day about the relevant areas without never actually visiting the places. The best way to tell them apart is that the former usually eschew the label “expert,” whereas the latter crave it. The real experts can help you craft your platform and maintain it so it is the most appropriate for the audience. We know from personal experience that working on a platform in a remote part of the world without experts generates lots of frustration and wasted effort.
• Google—Use the Internet to look up census data, U.N. data sets, various non-governmental organizations data sets, academic research papers, and sociological articles. You will be surprised at how much stuff on the Internet ends up becoming relevant. Make sure to verify the information using some of the tips from Part II of this book.
• Social media data—You spent part of this book learning how to collect and analyze social media data, so you might as well use those skills to help build your platform. Your potential audience may talk about their media and technology habits on social media, and the fact that they are using social media provides you with critical information.
• Crowdsourcing pilot programs—Simpler crowdsourcing platforms or failed platforms can provide lots of key insights. We explain how to build a simple crowdsourcing platform that employs sending bursts or blasts of texts to collect intelligence in Chapter 9.

Design the Platform

Designing the platform entails determining exactly what the platform will look like, how it will function, how participants will access and use it, and how it will attract the participants. The information you have collected so far and your analysis of the target audience and the media environment will help you make the determinations. This step is the hardest part when it comes to working with crowdsourcing platforms, and you will likely get it wrong more times than you will get it right. It is very much an iterative process and highly dependent on your objective, target audience, resources, and various geopolitical, cultural, and economic factors.

Determine the Platform's Look and Feel

The look and feel of the platform refers to the type of platform, the features available to the participants, and the design of the user interface (UI). Types of platforms are the ones described in Chapter 7 and include standalone websites such as OpenIDEO's or forums, smartphone applications (apps), apps integrated into existing social media platforms, and SMS-based communication networks. The features are the ones detailed in Chapter 2 and include the ability for the participants to chat with each other, share pictures, and friend each other. The UI design includes everything from the colors of a website to the placement of buttons on a smartphone application. Use the findings of your target audience and an honest consideration of your resources to determine the platform's look and feel.

Choose the Correct Type of Platform

The first step is determining the type of platform you will build. Your analysis should have detailed what types of social media platforms, websites, and phone applications your target audience prefers. Unless the media environment is oversaturated with a specific type of platform, build the type of platform that the audience prefers. The audience will adopt it quicker, and it will reduce how many resources you expend trying to teach them how to use it. Thus, if the audience:

• Obsessively uses Facebook, build an app on Facebook. It will lower your development costs and make marketing easier.
• Is very web-savvy and visits lots of websites, build a new standalone website. Many websites for people in third-world countries look awful and are seizure-inducing (lots of annoying, flashing banners). A well-designed and clean website will prove very attractive.
• Uses smartphones a lot, build an app for the smartphone. Generally, the more developed the media environment, the more likely you should build a smartphone app.
• Uses dumb phones and SMS a lot, build an SMS-based network. SMS networks are also helpful when you need to discreetly collect intelligence from individuals or small groups in hostile or rural areas.
• Is made up of different types of people or uses lots of different media, build a mix of platform types. For example, if your target audience includes people from all income classes in Pakistan, you should build a standalone website that users can also access through SMS.

Your budget and the resources available to you will determine your type of platform. Generally, standalone websites require much more money and manpower to build, followed by smartphone apps, existing social media platform apps, and SMS networks. However, do not make decisions solely according to your budget. Whatever type of platform you build has to be familiar and attractive to your audience. Building an SMS network for an audience who prefers complex and feature-full smartphone apps will save you money but prove ineffectual.

Choose Your Platform's Features

Figuring out which features your platform should have is very similar to determining what type of platform you should build. Again, look through data on the target audience and media environment to see what your audience prefers. If the audience tends to prefer platforms with a certain type of feature, make sure to include that feature into your platform. Try to introduce a feature with which the audience is unfamiliar. The novelty of the feature will attract participants and entice them to try out the platform.

However, do not overburden your platform with too many features. If a platform has too many features, the audience may become overwhelmed and stop using it. Simplicity sells. Focus on a few things and do them well. If you must integrate certain features, introduce them piecemeal over a long period of time. Introducing features gradually also enables you to remove features without upsetting the audience too much. Your audience will likely speak up about what kinds of features they want. Make sure to listen to them.

Also, when choosing features, stay true to your objective and make sure the features do not contradict it. If your platform's goal is to only collect intelligence about the movement of terrorist groups in specific parts of Lebanon, you likely do not need to give your audience the ability to friend and communicate with each other. Your audience will likely want anonymity, and the last thing they would want to do is willingly speak with others about the fact that they are providing foreign governments with intelligence about people in their neighborhoods.

Design the User Interface

Your platform must have at least a decent UI to succeed. Few people will willingly use a website or app that has an awful UI. The UI is less important for SMS networks, because all people can do is send texts to each other. For other types of platforms, however, the UI is critical. Again, look through your analysis to see what your audience prefers, paying special attention to their media environment and culture. Culture and personal experience may impact how people view and manipulate objects, shapes, and colors. Study the audience's culture and environment to see what they prefer and expect, and make sure to find out if certain colors or shapes are taboo. Regardless of culture, humans do share some global similarities when accessing interfaces. Check out the references for a list of such similarities, and also differences.²

Designing a UI properly is difficult and should not be left to amateurs. Unless you have a background in web or industrial design, do not try doing this step yourself. Hire art or web development students, preferably ones familiar with the target audience's culture, to help you design the UI. As you design the UI, recruit testers to use your platform. A good guideline for UI design is that it should be minimalist, intuitive, and built with the audience in mind. Just because you prefer certain colors or functionalities does not mean your audience will as well.

Determine the Platform's Incentive Structure

The platform may look and function great, but it will be useless if you cannot incentivize your audience to participate. Incentivizing entails encouraging and motivating your audience to sign up, try the platform out, recommend it to others, and use it regularly over a long period of time. An incentive structure organizes the incentives, helps participants understand them, and ensures the participants receive them in a timely manner.

You need incentives for your platform to succeed—there is no free lunch and there is no free platform participant. Well-structured and attractive incentives can also help your platform stand apart from other platforms and gain popularity. They can also encourage your participants to compete with each other and boost their value to you. Those of you involved with human intelligence (HUMINT) collection already understand the power of incentives to recruit and maintain sources. HUMINT officials use the shorthand M.I.C.E. for describing their incentives. M.I.C.E. stands for:

• Money—Source needs money to pay off debt or improve quality of life and social status.
• Ideology—Source does not agree with her government's way of doing things, or has competing belief systems (social, religious, economic, political).
• Compromise or Coercion—Source is susceptible to blackmail or vulnerable and willing to help due to an emotional relationship. It is usually the last resort, and for crowdsourcing the very worst motivator.
• Ego or Excitement—Source does not feel valued or is unsatisfied with his reputation or life. He believes he is worth much more than he is currently valued, or thirsts after challenges and adrenaline rushes.³

Incentives on crowdsourcing platforms also follow the M.I.C.E. model, although we call them extrinsic and intrinsic incentives.

Choosing Extrinsic and Intrinsic Incentives

Your platform may offer extrinsic, intrinsic, or a combination of both incentives. Extrinsic incentives are material items and objects external to a person such as cash, phones, clothes, SMS credits, and toys. They also include services such as the provision of local crop prices and weather forecasts. They are the M in the M.I.C.E. model. The exact nature of the extrinsic incentive will depend heavily on what your target audience likes, prefers, needs, and wants. It is essential to choose incentives that your target audience values. Rewarding free MP3 downloads of the new Iron Maiden album to Tuareg nomads who do not have iPods and have never heard Western heavy metal music will only elicit confusion and disinterest. In contrast, dumb phone ringtones from a local Tuareg rock band such as Tinariwen will resonate with the target audience and leverage technology they already have.

Be creative when coming up with extrinsic incentives. Work with local governments, NGOs, companies, and/or area experts to determine appropriate extrinsic incentives. For example, throughout much of the world where literacy is low and oral traditions persist, new technologies enabling easy recording, replay, and dissemination of local materials may be how you build your platform. Your platform could connect populations (who share a common language that is in danger of dying out) to share security or travel tips across long distances in a remote region. Such a platform could not only provide hard-to-get security tips, but it could map locations, the distribution of the population, and transcribe and create dictionaries (through crowdwork or appropriate language tools). Such an effort would draw not only security-minded organizations, but academics and NGOs (such as UNESCO) as well.⁴

Generally, extrinsic incentives are easy to quantify, measure, inform the target audience about, and provide. They have agreed-upon definitions. Participants can easily understand what winning an hour's worth of credit at the local cyber cafe entails. They are also likely to entice participants in the beginning when the platform is first launched and they do not yet trust you. However, extrinsic incentives can become expensive and lose their appeal over time. People usually like novelty, so over a long period of time you will have to offer a variety of incentives. Also, people may lie or cheat when exchanging information for money and material objects depending on how they acquire the incentives and the oversight they receive.⁵ In general, when creating a platform for a long period of time, phase out or minimize the role of extrinsic incentives and replace them with intrinsic incentives.

Intrinsic incentives are internally generated motivations specific to an individual—they are the fuzzy, hard-to-describe things such as pride, a sense of community and belonging, and the feeling of contributing to a valued project and helping out. They are the I, C, and E of the M.I.C.E. model and much more. Usually, intrinsic incentives are agnostic to a person's culture. Everyone responds to similar intrinsic incentives as long as they are not psychopaths who are motivated by their own set of personal intrinsic incentives.

Intrinsic incentives have enormous potential and power. Most people will respond strongly to them and, if given the opportunity, surpass participation expectations. Because much of intrinsic incentives have to do with social factors, such as maintaining reputation and relationships, their propagation among participants will lead to the development of strong communities on the platform, which in turn will lead to sustained participation. Communities on a platform are the social networks and relationships that participants develop, much like they do in the real world. Once people become part of an online or offline community, they do not want to leave it. People also have an innate tendency to want to be useful to their community. Thus, if their community expands to include you as the moderator and owner of the platform, then even better for you. Also, extrinsic incentives can lead to corruption, but intrinsic incentives have a way of simplifying people's motivations, and ensuring that the participants remain helpful and honest. However, intrinsic incentives are much harder than extrinsic incentives to deliver. You cannot simply tell a 16-year-old Latvian programmer that she will feel wonderful if she participates in helping you to find security gaps in your IT infrastructure. Instead, you have to provide the participants with the environment where they can acquire the incentives. Creating the environment is an art form.

When discussing intrinsic incentives, some people think of the word gamification. Gamification is the process of creating a game out of an everyday process. The theory is that people like playing games, in which incentives are delivered through game mechanics such as the earning of points. The theory then continues that by gamifying an everyday process, you can boost a person's willingness to undertake the process. For example, individuals trying to lose weight are more likely to follow their diets if they get points at the end of a day based on what they ate. According to the theory, because individuals really want to earn points they will follow their diets more strictly. Some of the intrinsic incentive delivery methods we describe later have some similarities to such a gamification process. However, the term has become mangled and overused, and is frequently misapplied. It also simplifies and makes light of people's complex motivations. When looking at other crowdsourcing platforms or social media platforms for inspiration on how to create your own incentive structure, be wary of analyses of the platforms done through the lens of gamification. Gamification aficionados often see game mechanics and intrinsic incentives where none really exist. Do not approach structuring intrinsic incentives for your platform as if you were creating a game out of getting participants to complete your objective. Approach it solely as how can you incentivize individuals to do something they would probably want to do anyway. If an individual absolutely does not want to help you, you will not be able to gamify him into doing so.

Overall, both incentive types are useful, albeit for different cases. Extrinsic ones are better for attracting participants when the platform launches, and persuading participants to join and use the platform initially. They are also more appropriate and easier to implement when your objective is to simply collect intelligence and solutions for problems. Intrinsic ones are better for keeping participants interested over a long period of time, and if you need them to do amazing things. They are essential when you need to influence audiences or if you need to encourage participants to work together toward a common goal or solution. Ideally, you should combine both. The best way to combine them is where acquiring one type of incentive allows the participant to acquire the other type.

Delivering the Incentives

Making sure participants can acquire the incentives in a timely manner is essential to keeping participants happy and useful. Delivering extrinsic incentives depends on the target audience's physical, political, and cultural environment. Work with local governments, organizations, and/or area experts to determine the best way for delivering goods to participants. In some cases, discretion may be needed. Delivering services is much easier and should usually be done through the same device with which participants access the platform. Clearly communicate what the reward is, how people can get it, and when people should expect to get it.

You do not deliver intrinsic incentives, per se. Participants acquire them as a result of their behavior on the platform. You must, however, construct the environment to ensure their behaviors can translate to incentives. You have numerous ways to construct the environment, and which you choose will depend on your target audience. Some popular mechanisms for constructing the environment on other platforms include doing the following:

• Allow participants to create a community on the platform and communicate with each other by allowing them to text or direct message each other, form online groups, friend each other, and discuss issues in chat rooms and forums. Implementing this mechanism in platforms that leverage only dumb phone technology and do not exist as websites is markedly more difficult, but not impossible. Try to implement it nonetheless because the platform will then stand out and draw the target audience's attention.
• Implement a leveling (popular in video games such as Warcraft) or virtual badge system (Foursquare, Yelp) that makes participants feel they are earning improvements in knowledge, effectiveness, and status. For example, if a participant provides you with extremely valuable intelligence, reward her publicly with a rare and hard-to-earn gold badge that participants understand only goes to top performers. The boost in the rewarded participant's public status will increase her self-worth, pride, and eagerness to continue to participate in the community and platform.
• Provide participants with things they can collect. For example, for every puzzle a participant solves, reward him with virtual gold coins. Participants with larger collections will feel as if they are higher in status and more needed by the community.
• As mentioned before, combine intrinsic and extrinsic rewards in such a way that earning one type of reward boosts the ability to earn the other type. For example, let participants trade in the badges they earned for money. The participants will then work harder to regain the badges and their status in the platform's community. Conversely, publicly congratulate participants who win an extrinsic reward to boost their status and pride.
• Communicate directly with the participants through community moderators to elicit their feedback, suggestions, and questions. Participants will then feel a sense of ownership concerning the platform, which will increase their desire for the platform to succeed and thus, increase their desire to participate. We discuss the role of moderators later in this chapter.
• If appropriate, inform participants about how their efforts have helped you complete your objective and/or have helped improve their lives. Like the OpenIDEO platform, get them involved in carrying out any suggestions or solutions they offer.

Look through existing crowdsourcing platforms to identify more mechanisms. Feel free to try different ones, and expect some to fail.

Regardless of the type of incentive, always deliver on your promises. The platform owner and manager's reputation is everything, and failing to deliver incentives on time or lying to participants about what they have won will cause participants hurt feelings and anger. They might even seek retribution against you by, for example, posting about their experience with your platform online for the rest of your target audience and adversaries to see. You may have noticed that word travels extremely fast on the Internet. Also, do not make it too easy for participants to win the rewards. Make them work for it. In return, they will be more eager and thus more useful. It is the same principle as asking a girl out—if you appear desperate and too easy to please, she will either not value you or simply ignore you.

If you expect participation to be high for whatever reason, you can scale back how many rewards you provide and make their bestowment schedule irregular. An incentive or reward's bestowment schedule stipulates when participants can win a reward. For example, participants can answer a question every Friday and win something, which is an example of a regular schedule. Or they can sometimes answer questions on Tuesday to win something, or sometimes on Wednesday, which is an example of an irregular schedule. As is expected, people, like other mammals, can be conditioned to perform actions upon receiving a reward. Ironically, if people expect a reward but are not sure when they will earn it, they will be motivated to perform the action more aggressively and regularly—a phenomenon known as conditioning with a variable schedule of reinforcement. By bestowing rewards irregularly—sometimes immediately rewarding participants and sometimes holding back—you can boost overall participation exponentially.⁶ However, use this tactic carefully and only in cases where you are confident the participants trust you, like your platform, and are eager to participate.

Build the Platform

Using the technology to build a crowdsourcing platform is relatively easy, as long as you are not trying to do too much at one time. Building a platform entails creating its components, which roughly are:

• The front end—The website or app that participants and moderators see
• The back end—The servers that house the website or app and manage the data coming through the platform, and the tools that allow the platform managers to change the platform
• The data analysis tools—If applicable, that help you sift through crowdsourced data

Assuming you do not have a web development background, you have three ways to go about building a platform.

The first way is to use and modify existing crowdsourcing platform templates. Increasingly, mostly non-profit organizations are putting up software packages and code online for free so that anyone can use them to create and deploy certain types of platforms. For example, Ushahidi, the Africa-based non-profit, allows anyone to use its templates for creating and deploying crowdmaps. The crowdmaps are ideal for collecting intelligence through SMS and populating them on a map. Anyone can use the crowdmap templates because they are open source and require very little technical skill to use. Some current open source solutions are not quite templates, but rather tools that can still help with deploying and running platforms. These solutions are usually for running SMS-based platforms. FrontlineSMS and RapidSMS are two such free tools that enable anyone to send numerous bursts or mass texts to people and receive and manage their responses. More templates and tools will become available as crowdsourcing grows in popularity. As you may have noticed, the templates and tools are better suited for platforms that collect intelligence as opposed to other objectives.

The second way is to hire in-house or contract developers to build the platforms. If you want to build standalone websites or smartphone applications, you probably need to hire outside help. The developers should help you create the front end, the back end, and any data tools you need. The developers can also help you refine and extend the aforementioned open source templates and tools. We prefer working with developers from the region our target audience inhabits. Local developers will know what their neighbors will like and can steer you away from bad decisions. Also, do not simply hire developers to build the platform initially and then let them disappear. Keep them on retainer, or at least ensure that they are willing to work with you over the long term because platforms will go awry. Web technology is always riddled with bugs.

The third way is to hire or partner with third-party organizations to build and maintain customized crowdsourcing platforms. Increasingly, for-profit companies are starting to offer such services. Most, if not all, of these companies are geared toward helping companies do brand management and marketing. Be wary of such companies because very few deal with security issues or work in the areas you probably find most interesting. Defense contractors in the U.S. are starting to offer services to build and manage crowdsourcing platforms. From our experience, many are simply jumping on the social media bandwagon and promising amazing things, but have little idea of what they are doing. When considering them, make sure to ask about their experience and how many people they have on staff who have built such platforms before. The ideal company will need solid and experienced teams of behavioral experts, area experts, data analysts, and web developers. Very few companies actually have such capabilities because crowdsourcing for security is a relatively new field. Do your due diligence when considering the companies, because hiring the wrong company can lead to a disastrous platform and disastrous consequences. Partnering with experienced non-profit organizations such as OpenIDEO is better in some ways. They are fewer in number, but they have run crowdsourcing platforms before. In OpenIDEO's case, it actually advertises its willingness and preference for partnering with government and other organizations to solve tough societal problems. They will lessen your risk and burden. However, many non-profits are skeptical of people working in the defense and security field. Make sure to approach them cautiously.

Market the Platform

Your platform could be the best designed and constructed in the world, but it will not amount to anything if your target audience does not know about it. You are not Steve Jobs. Regardless of where your target audience lives, you cannot simply build something and expect people to go crazy trying to get in on it. You will have to launch advertising campaigns for your platform, just as you would for any other product or service. In this case, your customer is the target audience. Unfortunately, creating the advertising campaign will not be anything like the TV show Mad Men (although, feel free to drink at lunch).

First, decide how much you want and need to advertise your platform. Take the need for secrecy and discretion into account. As we explain in subsequent chapters, often it is better to create platforms that do not make their ultimate objectives clear. For example, if you want to collect microeconomic data to understand instability in certain areas, you may tell your audience that your objective is to create a platform for merchants and farmers to share market price data so they can improve their market strategies. In this case, you would advertise the latter objective and not your ultimate objective. Often, you may not want too many people in an area to know that you are collecting information from individuals in those areas. For example, you could incentivize workers for sex traffickers such as drivers and bodyguards to text you their locations so you can map trafficking ratlines. If word got out that there was an SMS network that was collecting information from such workers, a few sources would probably be put in danger and you may get misinformation. In such cases, crowdsourcing becomes more of a traditional clandestine HUMINT mission, albeit one that relies on social media technology and where word of the network spreads mouth to mouth. Also, take into account how many participants you want on the platform. If you only want 15 users, do not advertise too much. You will have to start turning people away, and people do not like being turned away.

Keep this need for secrecy in mind when naming your platform. Use names that your target audience will understand and knows how to pronounce and spell. Make sure your name is simple and has some indication of what you want your audience to think your platform will do for them. However, your name probably should not reveal your true objective. For example, your objective's platform may be to collect intelligence about how Al-Shabaab activity is affecting economic and thus physical stability in the areas surrounding Mogadishu. However, you may tell your audience that your objective is to collect market prices so you can supply the participants with their area's market prices to boost market transparency and help them set their crops' prices. In such case, name your platform the Somali translation of something like “Open Market” or “Farmer Help.” Use common sense and do not name it “Al-Shabaab Tracker.”

Apart from secrecy, the media environment also dictates the volume of the advertising campaign. If you are trying to launch a platform in a media-saturated place like France, you have to compete with lots of other activities and media for people's attention. Your advertising campaign will then have to be fairly substantial. If your audience lives in the Andes Mountains and only has access to dumb phones because you provided them, you do not need to advertise that heavily. You will not have to compete with other dumb phone networks because they will not exist. You may, however, have to compete with daily activities that have nothing to do with technology.

After determining the appropriate marketing push, employ a variety of campaigns and ways to advertise your platform. Your advertising campaigns will likely come in waves, depending on your platform's popularity. You will have to launch an initial advertising campaign to get the word out in the weeks leading up to the launch of the platform. After the launch of the platform, you may have to launch smaller advertising campaigns every few weeks to keep participants interested and to attract more participants. Use different methods for each campaign. The methods depend heavily on the media environment. Some examples of methods by media environment type are given in Table 8.1.

Table 8.1 Example Advertising Methods

Apart from external marketing campaigns, you can also advertise through the platform itself. Incentivize participants to recruit others.

Lastly, track mentions of your platform or advertisements in the target audience's media environment to see how much buzz you are creating. If you want lots of people on your platform, your goal should be to get a lot of people in the area of the audience to talk about it. Getting picked up by local media is a good indication that your advertising campaigns are working. Use the metrics we describe later to assess how well your platform is doing, and modulate your advertising campaign as needed.

Manage the Platform

Running a crowdsourcing platform involves dealing with lots of people, many of whom will likely see the world very differently than you. The more features a platform has, the more likely the interactions among participants will lead to disagreements, fights, rule-breaking, and chaos. You need to manage the interactions among the participants and the communities that may appear on your platform. Communities are more likely to appear on platforms with social networking and messaging features, where the objective is to influence or solve complex problems that require working together. If your platform simply collects intelligence from individuals, much of this section probably will not be as relevant. For all other cases, especially when your objective is influence, you will want to foster a sense of community for the participants because it is bound to happen after repeat interactions among participants. Creating a set of rules and hiring competent community moderators or managers to patrol the platform will significantly reduce the risk that antisocial interactions drive away participants and ruin your platform.

The first step for managing the platform is determining what you consider the ideal types of interactions on the platform. If you are running a platform where participants solve math problems, the interactions among participants should not result in political arguments. If you are running an SMS-based platform where participants text you the location of illicit actors, you do not want interactions among the participants. If you are running a platform through which you want to influence participants to denounce extremist activity, the interactions among participants cannot result in language promoting extremist activity. However, do not be too idealistic or strict with what you consider proper interactions. People everywhere frequently like to go off-topic and only sometimes does it produce harm. Providing people with space to interact is essential for creating communities, which could significantly bolster your platform's effectiveness.

Create a set of rules and policies that you want all participants to abide by. They are similar to the rules you agree to abide by but never read when you sign up for platforms like Facebook. The rules should clearly state what you would like the audience to think is the objective of the platform. It should also state what sort of language and behaviors are allowed and, just as importantly, what is not allowed. Read the terms and conditions or rule pages of websites like OpenIDEO or Facebook to get an idea. When participants sign up for the platform, make sure they get a chance to read the rules. You will likely need to work with a translator to make sure it is in language they clearly understand. Simply reading the rules does not mean participants will necessarily abide by them. However, the rules give moderators the authority and legitimacy to then patrol the community.

Community moderators are key for making sophisticated platforms with lots of social networking and messaging features work. Moderators have numerous roles, including assisting participants, keeping them engaged, securing and protecting them, and regularly updating you with the platform's status. They post on the platform like they are participants similar to the police officers who walk the streets as the people they are charged with protecting. The moderator is an example of how social media has led to the creation of a job that did not exist a few years ago. Because the occupation is new, finding experienced and competent moderators can be difficult. An ideal moderator must be familiar with the Internet and social media, pay attention to detail, have social tact and a desire to work with people, and have the ability to handle lots of stress, awkward situations, and verbal abuse.

Finding moderators is easier if your target audience is Western. If your target audience is in a foreign country or from a different way of life, you must hire and train local moderators who live among the target audience. You will need to hire anywhere from two to five moderators, depending on the sophistication of your platform, the risk you are willing to tolerate, and the number of participants. You can even recruit part-time or full-time moderators from the participants, many of whom will want to help out as the platform becomes more successful. The moderators should work in shifts so that they cover most of the target audience's waking hours. Make your most experienced or talented moderator the lead moderator and put him or her in charge of managing the other moderators and updating you with what is going on with the platform periodically. The moderators should also have access to platform administrator tools, not available to participants, that allow them to warn and ban participants, delete messages, and discuss among each other.

Several books and articles describe how to help train community moderators, a few of which we list in the references.⁷ Generally, all moderators should regularly do the following:

• Answer queries from participants—Participants will no doubt have questions about how the platform works and how they should behave. The moderators must identify when participants are having problems, acknowledge queries and problems, and provide sound guidance, as needed.
• Make participants feel important—Moderators must routinely thank and compliment participants for their ideas, information, and posts. They should also ask participants for advice on improving the platform or other issues. Essentially, the moderators need to constantly treat participants as if they are very important people without seeming too sycophantic. Participants will consider the goodwill that moderators provide them with as an intrinsic incentive that makes them feel better, and thus spur them to be more active on the platform.
• Identify and stop trolls and misguided participants—In Internet parlance, trolls are troublemakers and rule-breakers, who sometimes create havoc on social media and online communities for the sake of creating havoc or attention. Often, participants will break the rules because they do not understand or do not agree with them. Moderators must quickly identify and stop trolls and misguided participants. They should reinforce the rules of the platforms and make it clear that rule-breakers face punishments. Otherwise, the platform and the community on it will devolve into chaos. The punishment can include a one-on-one stern talking-to, or even outright banning of the users and their IP address from the platform.
• Acknowledge criticisms and suggestions—Your platform will not be perfect and will suffer from several problems. Engaged participants will voice their opinions about the problems. The moderator must listen to appropriate criticisms and suggestions, and inform you of them. Participants often know the best way to make your platform more effective and engaging.
• Spur participation—The moderator should serve as a role model for the participants, and frequently ask questions, post their opinions about topics (especially important when your objective is influence), and echo the sentiment and opinions of participants. If participants are engaged and active already, the moderators need not be as active. Ideally, you do not want moderators to act too much like participants.

Measure the Platform's Performance

Regularly measuring your platform's performance is critical to ensuring that your objectives are being met, and for knowing if and when you need to implement changes. There is no one right way to measure performance. You will likely need to create and populate numerous metrics that tell you something about each component of the platform.

Create and Choose Metrics

Traditional information operation metrics and forms are largely unsuitable for crowdsourcing efforts. Instead, you will have to create and select metrics based on your objective, your target audience, and expectations. Use the sample metrics in Table 8.2 and come up with more on your own.

Table 8.2 Sample Metrics for Measuring Crowdsourcing Performance

Set expectations of what you would like to accomplish and make it a habit to regularly update your metrics and compare them with your expectations. By regularly checking the metrics, you can make your expectations more realistic and identify parts of the platform that may need improvement. For example, if you find that participants lose interest after three weeks of using the platform, you can introduce special incentives around the three-week mark to keep the participants interested.

Collect Data to Populate Metrics

The metrics you select are only useful if you can collect data to substantiate them. Three data sources are available. The first is data tools integrated into your platform that measure everything from how many participants you have to what time of day they post. Google Analytics offers a free service that enables you to track usage statistics or how participants are using your website. Smartphone apps and SMS networks are harder to collect data about. However, ask your developers to integrate data collection tools into the app, network, and web page. Integrating data collection tools is easy and will prove very useful. The second source includes everything external to your platform. They can include how many times your platform is mentioned on Twitter or in the local newspaper. Regularly check the newspaper, radio stations, and social media platforms that your target audience frequents to collect this type of data. The third is providing surveys to participants before, during, and after the platform's life. Work with a statistician or someone with a background in quantitative research methods to create and run proper surveys. Ruining surveys with biases and leading questions is very easy and will result in errant data.

Wash, Rinse, Repeat

Do not expect to succeed on your first platform or meet all expectations. Running crowdsourcing platforms is more difficult than it appears. Each time you run one, you will gain new insights for how to run them more efficiently.

Hire outside consultants to do a post-mortem on your platform and identify specific instances where it went wrong or succeeded. Create and maintain a lessons learned chart throughout the life of the platform to ensure you have captured all key insights. However, keep in mind that some of the insights and lessons may be specific to a target audience or situation. There is a burgeoning community of well-meaning individuals who are constantly working on crowdsourcing problems and sharing the lessons they learn on websites such as http://www.crowdsourcing.org/. Consider posting your failure and lessons learned on such sites to look for advice. In an almost post-modern twist to your dilemma, crowdsource the solution to your crowdsourcing problem.

You now have a thorough understanding of the overall process for building a crowdsourcing platform. However, you need to tweak the process to build the platform that will help you meet your objective. Chapter 9 will teach you how to tweak the overall process to build a platform to collect intelligence.

Summary

• Using existing technology to build a crowdsourcing platform is relatively easy; however, ensuring the platform's success is much harder and depends on numerous factors.
• Regardless of the objective, each crowdsourcing platform necessitates the completion of certain steps. Read Chapters 9, 10, and 11 to learn how to tweak the steps according to your objective.
• The overall process for building a crowdsourcing platform is roughly as follows:
  • Select the platform's objective and scope:
    • Refine the problem to establish a clear objective or combination of objectives.
    • Decide the scope of the platform depending on factors such as the need for secrecy, and the overall budget, time, and appetite for risk.
  • Analyze the target audience and media environment:
    • Determine the target audience, who are the people you want to participate on the platform.
    • Analyze the target audience to understand how they communicate and what they prefer so your platform is attractive to them.
    • Analyze the media environment to see what your platform is up against in terms of other media and how outside forces can influence your participants.
    • Get the information to do the analyses from the Internet, social media, research reports, sociological studies, pilot crowdsourcing projects, and area experts.
  • Design the platform:
    • Determine the platform's look and feel by choosing the correct type of platform, features available for participants, and intuitive user interface.
    • Determine the platform's incentive structure by integrating a mix of extrinsic and intrinsic incentives into your platform. Doing so will attract participants and keep them active.
  • Build the platform:
    • Create a front end that participants will see, a back end that houses the server and data, and data analysis tools to analyze all the incoming data.
    • You can either use existing crowdsourcing platform templates, hire developers, or hire a for-profit crowdsourcing building and managing company.
  • Market the platform:
    • Decide how much you want to advertise your platform based on your need for secrecy and the saturation of the audience's media environment.
    • Use a variety and combination of methods including advertising it on the Internet, traditional media, and billboards; also incentivize participants to recruit others.
  • Manage the platform:
    • Create a set of rules and policies so participants know how to behave.
    • Hire local community moderators with good social skills to ensure participants abide by the rules, help and listen to participants, shut down troublemakers, and update you on the platform's status.
  • Measure the platform's performance:
    • Create and choose metrics that are relevant to your platform and objective.
    • Collect data to populate metrics through tools such as Google Analytics and data tools integrated into the platform that collect user statistics.
  • Wash, rinse, repeat:
    • Learn from mistakes you make from your first few platforms.
    • Make the appropriate changes and try again.

Notes

1. Rogers, E. (1962) Diffusion of Innovations. Free Press, New York.

2. Johnson, J. (2010) Designing with the Mind in Mind: Simple Guide to Understanding User Interface Design Rules. Morgan Kaufmann, Burlington; Norman, D. (1988) The Design of Everyday Things. Basic Books, New York; Weinschenk, S. (2011) 100 Things Every Designer Needs to Know About People. New Riders, Berkeley.

3. Wallace, R., Melton, H.K., and Schlesinger, H.R. (2008) Spycraft: The Secret History of the CIA's Spytechs from Communism to Al-Qaeda. Dutton, New York.

4. Rymer, Russ (2012) “Vanishing Voices.” Journal of the National Geographic Society. 222 (no. 1).

5. Ariely, D. (2012) The Honest Truth About Dishonesty: How We Lie to Everyone—Especially Ourselves. HarperCollins, New York.

6. Ferster, C. and Skinner, B. (1957) Schedules of Reinforcement. Appleton-Century-Crofts, New York.

7. Connor, A. (2009) 18 Rules of Community Engagement: A Guide for Building Relationships and Connecting with Customers Online. Happy About, Silicon Valley; Ng, D. (2011) Online Community Management for Dummies. Wiley, Hoboken.