Chapter 10

Crowdsourcing Solutions

Through crowdsourcing platforms, you can leverage populations worldwide to help you solve problems and complete tasks. The process for building and running crowdsourcing platforms is technologically simple, but requires the visceral ability to incentivize and foster collaboration between different types of people. The process also increases in complexity along with the complexity of the problem or task you want the crowd to solve. This chapter goes over the process by explaining what is possible through crowdsourcing, two ways of approaching crowdsourcing solutions, what is not possible, and how solution platforms differ from other types of platforms. It also walks through how to build and run platforms that leverage the crowd to translate obscure languages and to identify antagonistic actors on video footage.

Understanding the Scope of Crowdsourced Solutions

Crowdsourced solutions are solutions derived specifically through custom crowdsourcing platforms known as solution platforms. They differ from other, more traditional forms of solving problems or completing tasks in the way the problem is solved and the people who solve the problem. Traditional forms involve specifying the individuals or groups that will solve the problem, and compensating them for their work based on a predetermined rate. The specified individuals or groups may collaborate with one or two other groups. Usually, the party contracting out the work knows the identity of the workers and specifies, or at least knows, which tools and methods the workers will use. In contrast, when crowdsourcing solutions, the party contracting out the work does not usually specify the individuals or groups that will solve the problem, does not always know their identity, and does not specify or know the methods they will use. These features give rise to the creativity and flexibility that traditional forms of solving problems typically lack. Also, solution platforms differ from traditional forms in that how you formulate problems and the actual platform design can differ wildly. You have a limited number of ways to perform traditional contract work, but many different ways to crowdsource solutions. The following two sections expand on this feature.

Formulating the Right Type of Problem

Solution platforms can help you solve and complete a variety of problems and tasks. Possible problems or tasks can include translating a pidgin language that computer translators do not understand, creating complex computer algorithms for cameras so they can identify people behaving suspiciously, solving tedious but essential math problems, forecasting the likelihood that a certain event will take place, finding the right person for a job opening, and much more. In some cases, the line between solutions and intelligence is blurred. If you consider solutions to simply be unknown intelligence, solution platforms can help you solve the problem of finding that extremely elusive piece of intelligence.

When designing solution platforms, focusing on exactly how you define the problem for the crowd is key. If you do not properly formulate the problem, you will receive either wrong solutions or solutions that do not solve the problem the way you want it solved. Differentiating the problems into three types can help you think about whether you have properly formulated the problem you intend to put to the crowd.

The first type is what we call objective problems and they have only one correct answer and usually one solution. The answers are falsifiable and they have clear right and wrong answers. In some cases, they are simply intelligence collection problems. Example problems include:

• Who is the current prime minister of Zimbabwe?
• What is the American English translation of “que pase un buen dia”?

The second type is what we call subjective problems and they may have more than one answer or, in other words, more than one solution. They tend to be much more complex than objective problems and are open to interpretation. They do not necessarily have right or wrong answers, but simply answers that are better than others depending on a number of factors. Example problems include:

• What is an algorithm underlying a natural language processing tool that can detect sarcasm in a tweet?
• What is a design of a rugged hybrid-vehicle chassis that is durable and inexpensive to manufacture?

The third type is what we call mixed problems, which have features of both the objective and subjective types. Consider the types not as independent categories but as a spectrum, with objective problems taking up one end, subjective problems taking up the other end, and mixed problems somewhere in the middle. These middle, mixed problems usually have a right or wrong answer, but many ways of finding that answer. Example problems include:

• What will be the annual murder rate in Caracas for 2013?
• What is an algorithm underlying a natural language processing tool that can, in near real-time and with an 80 percent accuracy rate, detect sarcasm in a tweet?

You may have noticed that a problem's categorization depends on how exactly you formulate the problem and how exact you want the answer to be—the boundaries on the solutions. By boundaries we mean factors that define and limit what solution you want, which problem it should answer, and how it should work. They include the solution's accuracy rate, what place or time period you want the solution to consider, or the amount of resources required to create the solution. Generally, the more boundaries you put on a problem, the more you push it toward the objective part of the spectrum. Subjective problems have few boundaries and no right or wrong answer, only answers that are better than others according to certain criteria. Mixed problems are essentially subjective problems with well-defined boundaries and some answers that may be right or wrong, and some answers that are simply better than others. Objective problems are essentially subjective problems with very strict boundaries and a falsifiable, right or wrong answer. Strict boundaries tend to lead to falsifiable answers because the more restrictions you put on how someone may solve a problem, the fewer ways the person can solve it. Note that when we say there is only one way to solve the problem, we mean that there is only one solution that solves the problem. For example, many ways exist to figure out the identity of the current prime minister of Zimbabwe, but only one solution solves the problem of finding the identity of the prime minister. In contrast, many solutions or computer programs can, for example, find sarcasm in language. A subjective problem usually has countless solutions, whereas an objective problem usually has only one. Figure 10.1 graphically summarizes the relationship and differences between the different parts of the problem type spectrum.

Figure 10.1 Spectrum of types of problems

Keep the problem types in mind when designing your platform. They do not reflect hard-and-fast rules, and you are not required to typify problems. In some cases, the types will not apply or make sense. The types of problems are simply guidelines that help you formulate your problem more precisely. If you neglect to put strict boundaries on your question, the crowd will tend to see it as a more subjective question, and so the kinds of solutions you get may differ greatly from what you would like. To better appreciate this point, consider the following example.

Formulating the Right Crowdsourcing Approach

The type of problem is not the only feature that affects and defines your solution platform. The way you want the crowd to solve the problem also affects and defines it. The way, or what we call the approach, helps define the look and features of your solution platform, the identity of your target audience, and the intensity of your marketing campaign—in other words, the design of your platform. Similar to how thinking about the type of problem can help you formulate the problem, thinking about the type of approach can help you formulate the actual platform. Think of determining the approach as determining how you should fulfill some of the steps of the process that we detailed in Chapter 8. To aid understanding, we separate the approach into three types. In some cases, some approach types are better at solving certain types of problems than other approach types.¹

The first approach involves focusing more on leveraging the wisdom of the crowd and is usually best for solving problems that fall near the objective end of the spectrum. The focus is not on crowdsourcing from the “right” people, experts, or niche communities. Instead, it is on asking a large, diverse, and independent group of people what they think the answer is to a specific problem. As described in Chapter 7, leveraging the wisdom of the crowd involves averaging the responses of lots of people. Under the right conditions, the average of the responses is more accurate than the response of any given individual, including experts. However, getting the conditions right is not always easy. Utilizing the wisdom of the crowd approach requires understanding the right conditions as they relate to your platform and how you can bring them about. The following list describes the conditions and their impacts on your platform:²

• The problem must be clear and near the objective end of the spectrum. Clarifying and simplifying the problem ensures that all the platform participants, regardless of their background, understand what response you expect from them. A simple problem usually requires a simple response, which makes it easier for the responder. Generally, your question should start with phrases such as, “Who is,” “When did,” “What is the likelihood that,” and “What is the translation of.”
• The participants must be diverse. The greater the diversity, the more likely different insights and information are brought to bear on a problem, increasing the likelihood that you reach the correct answer. By diverse, we mean they should have different ways of looking at the world and different pieces of information. To achieve this type of diversity, you should recruit participants who are diverse in other ways, such as their cultural background, education level, and their political leaning. In practical terms, your target audience must be heterogeneous, numerically sizeable, and geographically distributed over a large area.
• The participants must be independent and refrain from biasing the responses of others. People inherently tend to conform and change their considerations of things to match that of their peers.³ Participants should not have their responses subjected to the whims of their peers, because then most participants will change their response to match that of others. Also, keeping participants independent actually increases the likelihood they will come up with something creative. Brainstorming and working in groups is all the rage in at least corporate America, but studies show that brainstorming and group work actually reduce creativity and effectiveness.⁴ Most people work better when they are given a problem, they can go away in a quiet spot to think about and work on it, and then come to the group with a rough draft of the solution. Do not get caught up in the pop-psychology babble about always forcing people to work as a team, and instead let people be hermits. They will be much more productive and creative. Thus, your platform should have minimal social networking and collaboration features.
• Information about the problem must be widely available. A large number of participants cannot bring information to bear on a specific problem if they cannot access that information. The ideal problems for the wisdom of the crowd approach are the ones that virtually anyone can think about. Thanks to the Internet and social media, the information is usually easy to get even if it is obscure. However, some cases of information paucity still exist. For example, there is no point in asking a large group of people on the Internet exactly which military personnel in North Korea have the most influence over Kim Jung-Un, because very few people have access to that sort of information. Thus, ensure that your problem is not asking something impossible to answer or too obscure.
• Recruit persons who are well informed and who you would traditionally go to for an answer to such a problem to help sift through the responses. Sometimes you cannot literally average out a response to the problem from all the participants' responses. In some cases, there may be a lot of prevalent misinformation that badly obscures the participants' responses. In such cases, it is always handy to have a subject matter expert who can help guide how you should structure the platform, what kinds of problems you should ask, and what sorts of responses you should expect. Do not let the expert just outright disqualify certain responses because he or she disagrees with them, and ignore any predictions he or she makes with absolute confidence. Take the expert's advice only as guidelines. Or, ask a crowd of experts and compare it with responses of other crowds. Use your judgment and intuition when using experts. Of course, it is a good idea to always keep in mind that sometimes crowds can be terribly wrong, ignorant, and dumb.⁵

Table 10.1 summarizes the conditions under which the wisdom of the crowd approach works best and how they affect your platform.

The second approach involves moving away from leveraging a very large group of people, and instead focusing on leveraging a few select niche communities or uniquely knowledgeable individuals. We call it the niche approach. Crowdsourcing is not only about farming out tasks and problems to large groups of people. It is also about reaching the right people you may never be able to employ under traditional circumstances. Some problems may be too complex or obscure and only a few individuals or groups may have the resources and information to solve them. For example, say you want to crowdsource a sophisticated computer algorithm that uses Bayesian belief networks to automatically identify the spread of a virus in a community based on the appearance of symptoms in that community. Most people will not have the knowledge, ability, or interest in solving the problem. Your efforts would be more fruitful if you focused your target audience on people in universities, healthcare consulting companies, and research hospitals, who do have the wherewithal to offer solutions. Or, you may need to translate a text from a very obscure language. You could ask everyone on the Internet to do it, but your marketing campaign would be enormous. You would be better off identifying the few people who do know the language—for example, the members of a forum dedicated to talking about obscure languages—and incentivize them to participate.

Table 10.1 Wisdom of the Crowd Approach Conditions

The niche approach has a significantly lower marketing effort than the wisdom of the crowd approach and it involves more complex and obscure problems. It also, in some cases, requires more collaboration between participants. Often, problems are so complex that certain people from all over the world need to come together to share their insights and solutions. In this case, your platform should feature social networking and collaboration tools that allow participants to come together organically to solve the complex problem. Usually, these people will have strong opinions about their efforts and may have solved a piece of the problem, and so will be less affected by the views of others. Collaboration then becomes more about stitching together different pieces of the solution.

The third approach, as you may have guessed, is the mixed approach, and one that you will use more often than not. Like the problem types, think of the approach types as a spectrum. Some problems will have components that are simple to solve and some that are very difficult to solve. The former will require a wisdom of the crowd approach and the latter will require the niche approach. In some cases you will need to re-create the wisdom of the crowd approach within a niche approach. For example, say you want to know the location of a stolen antique. Only some people in the world care about antiques, but those who do may be very knowledgeable about the location of antiques, where they might end up, who might steal them, and so on. Your target audience, then, is the niche community of people interested in antiques, but you can still use elements of the wisdom of the crowd approach to ensure you get a response that is representative of the antique crowd and, you hope, the actual location of the stolen antique. If you employ the third approach, you will need to undertake a smaller marketing campaign than you would with the wisdom of the crowd approach. However, you will need to implement some social networking and collaboration features into the platform. In summary, the wisdom of the crowd approach has a bigger and wider target audience and marketing campaign, but fewer social collaboration features in the platform. The mixed approach has a moderate size of each, with some differences depending on the case. The niche approach has a smaller and more focused target audience and marketing campaign, but more social collaboration features. Figure 10.2 graphically summarizes the relationship and differences between the different parts of the approach type spectrum.

Figure 10.2 Spectrum of approach types

Keep the approach types in mind when designing your platform. Like the problem types, the approach types do not reflect hard-and-fast rules. They are guidelines that clue you into the social features of your platform, the span of your marketing campaign, and the identity of your target audience. Also, you do not need to take our word for it. We devised these rules after reading diverse literature on the wisdom of the crowd phenomenon and creativity. Check the references for more information.

Appreciating the Limits of Crowdsourcing Solutions

Solution platforms are very powerful and cost-effective ways of completing mundane and obscure tasks, and solving seemingly impossible problems. However, they cannot help you solve all problems. In some cases, they can and will return inaccurate solutions. Understanding the limitations will help you understand where and how solution platforms can go wrong, and what you can do to mitigate complications.

Unsolvable Problem

Not every problem has a solution, and not every task can be completed. Some problems simply may be too difficult or complex for anyone, let alone crowds on the Internet, to solve. Some problems do require traditional forms of problem solving. They require special equipment and a dedication of manpower and resources that you cannot incentivize platform participants to match. Do not put problems to the crowd that require an enormous amount of resources, unless you are willing to incentivize them effectively and lucratively. If the problem does not require special resources but is simply very difficult, you might as well bring it to the crowd because you do not know which problems really are unsolvable. In some cases, the problem may have a solution, but you may not be able to reach the individual or group who can solve it. Finding the right participant involves conducting an intelligent marketing campaign and defining the target audience properly. Still, in some cases you may never be able to find the right person. But you will not know unless you try, so you might as well try.

Incorrect Solutions

A lot of the responses you receive will be junk, especially if you do not formulate your problem clearly and do not institute features in the platform that effectively pre-test the solutions. Some solutions may be correct in some areas but wrong in others. You should expect to receive incorrect solutions, and should have a plan in place to sift through the selections and pick the ones you think are the most accurate and effective. The plan depends heavily on the nature of your problem and wants.

Adversarial Participants

Participants sign on to solve problems for a variety of reasons. Most want to help solve the problem, but a few have malicious reasons or may adopt malicious reasons if they feel scorned. Adversaries can purposefully torpedo your platform using a variety of methods, although the chance of them doing so is low. The more likely threat is that of a few participants on platforms geared more toward the niche approach that aggressively criticize other participants and generally wreck any forms of collaboration. Moderators that effectively patrol the platform are the most effective defense against them.

Political Fallout

Sometimes the ends do not justify the means. Occasionally politicians, corporate heads, and public relations departments do not want their organizations using crowdsourcing to solve problems. For example, after the 9/11 terrorist attacks, the U.S. Defense Advanced Research Projects Agency (DARPA) tried to create a prediction market, a website that crowdsourced people's perception of when the next terrorist attack would occur. If people guessed right, they could win money. However, some politicians complained that it seemed the U.S. government was encouraging people to gamble on the basis of terrorist attacks and lost human lives. The platform was soon shut down. To mitigate such fallout, you should frame the problem in a more politically correct way or change the incentive structure. Instead of collecting direct intelligence, collect indirect intelligence. Similarly, collect solutions to problems that appear more politically correct, but in turn can help you solve less politically correct problems. Or, instead of paying people money, incentivize them through other ways. Tell them you are paying them to serve as patriotic analysts that predict terrorist attacks, rather than gamblers betting on terror. The U.S. government's Intelligence Advance Research Projects Agency apparently learned the lessons from DARPA's prediction market, and is now trying out a new terrorism prediction market that does not let people profit off their gambles.⁶

Lastly, sometimes an organization may prohibit you from bringing a problem to the public. A defense laboratory may not want to reveal it is working on a specific project, or a company assessing risks in a certain region of the world may not want to reveal to its competitors that it is looking to expand to that region. Keep such considerations in mind so you do not unwittingly reveal a costly secret.

Tweaking the Process for Solution Platforms

Despite the limitations, solution platforms can still greatly help you and your organization. They, of course, differ from other types of crowdsourcing platforms. Because of these differences, you need to tweak the process from Chapter 8 somewhat and adapt the approaches to them.

The most obvious is that the way you formulate problems matters immensely with solution platforms. They matter with other platforms as well, but exactly how you phrase the problem and what details you give, or how you bound the solution, can greatly affect the responses you receive. With intelligence collection platforms, if you put out a slightly misshapen problem to the participants, you can still receive useful indirect intelligence. However, a misshapen problem for a solution platform can result in useless solutions. For example, asking participants to come up with an algorithm that detects sarcasm in tweets written in the English language is very different from asking participants to come up with an algorithm that detects sarcasm in tweets, without specifying which language.

Also, the level of social networking and collaboration features you integrate into the platform affect the level of moderation you need. They depend highly on which approach you want to take. With the wisdom of the crowd approach, you should integrate fewer social collaboration features, which will require fewer moderators. With the niche approach, you should integrate more, which will require more moderators to ensure that the participants play nice. With the mixed or nested (wisdom of the crowd within niche) approach, you will need a moderate amount.

The type of approach will also affect how narrow you go with your target audience and what your marketing campaign will look like. With the wisdom of the crowd approach, you need to reach a wider and more diverse range of participants and so will need a marketing campaign that can reach a diverse number of people. With the niche approach, you need to reach niche communities and specific groups and so will need a more targeted and focused marketing campaign that reaches those people. With the mixed approach, you may need both.

Now that you know where to tweak the process, you can start building and running solution collection platforms. In the subsequent sections, we walk through how to build and run several platforms that differ in the missions they can support, and the type, resources budget, and target audience. For ease of understanding and comparison, the steps of the walkthroughs follow the structure of the process in Chapter 8.

Crowdsourcing Translations during Disaster Relief

In Chapter 9, we walked through designing a crowdsourcing platform that enabled responders to collect information during disasters from affected victims about what type of help they need, triage the information, and then respond immediately. Part of triaging the information involves translating incoming texts. By translating, we mean everything from literally translating words and phrases from one language into another and making sense of idioms, sarcasm, slang, and abbreviations.

Companies like Google and Microsoft (through its Bing service) offer a variety of translation services, each of which has strengths and weaknesses. Integrating their services, some of which are free, into a crowdmap or other crowdsourcing platforms is relatively easy because the services usually function as APIs. You simply send a phrase or word to their API and it sends back the translated version. In Chapter 6, we discussed several other services that also attempted to make sense of idioms, slang, and other elements of speech that foreigners may not understand.

The problem with these services is that they are focused on translating and making sense of popular languages such as English, Mandarin Chinese, French, and Spanish. Machine translation works well with popular languages because there is so much of it in text format, such as books to use to train the translators, and because there is a greater demand for them. Increasingly, machine translation services are starting to focus on more obscure languages, but they still have a long way to go. They are often inaccurate and unable to translate large parts of a less popular language's vocabulary. They are not ideal for translating messages from victims during disasters because they will likely fail to translate the messages properly. Poor translation leads to poor information, which in turn leads to responders being unable to find out which people need help and help them.

For now, the only option available to you for translating incoming messages during a disaster in a place like Haiti is to use people, who are very good at translating and making sense of language. This walkthrough explains how to create a capability that uses existing tools and platforms to crowdsource translation. Crowdsourcing translation is a form of crowd labor that is much more efficient and relevant for this situation than hiring full-time translators. Hiring full-time or even part-time translators takes resources and immense planning. You have to guess well before the disaster how many translators you will need and when you will need them. You also have to pay them a salary. In contrast, crowd labor enables you to quickly surge the number of translators, and in many cases you do not have to pay them. Through crowd labor, you can also get help from native communities that know the language and vernacular very well, as opposed to traditional, non-native translators who only know a more artificial and academic form of the language. The following sections walk through the process of creating such a capability.

Define Objective and Scope

Continuing with the disaster relief walkthrough in Chapter 9, imagine you have set up and deployed a crowdmap to collect and manage text messages from victims of a disaster in Haiti. (We got the idea for this walkthrough example from a real crowdsourcing effort that we reference again later. Read the corresponding reference in the note to learn more.)⁷ You will need to instantaneously translate and make sense of the incoming texts and push them to responders, some of whom speak only English. Basic research tells you that Haitians primarily speak French and Haitian Creole, which is an amalgam of French and various African languages. Also, because participants will be communicating through text messages, which limit texts to 160 characters, you expect the texts to contain many abbreviations and linguistic shortcuts. You also expect them to use slang and idioms. Lastly, because they will be texting at a very stressful time while possibly facing danger, you expect the texts to have many misspellings and errors.

No existing natural-language processing tool or machine translator can help you make sense of the incoming texts in an automatic manner. The best they can do is identify the language of the text, but integrating those tools into the crowdmap requires technical development. Ideally, you would like to use real people to translate and make sense of the texts, but you have already used up your manpower resources. You do have a number of moderators, some of whom can help you translate the texts, but you need more help.

With these considerations in mind, you can formulate your objective. You need to create a capability that can take the texts flowing into the crowdmap and send them to a crowdsourcing platform where participants can translate the texts. The participants then send the texts back to the crowdmap, where it is displayed for the responders to use. Additionally, you need to be able to stand up this capability quickly. You also want this process to work smoothly and quickly, and you do not have much financial or manpower resources to dedicate to it.

Analyze Target Audience and Media Environment

Because your requirements for an ideal platform participant are important, you should use them to figure out who belongs in your target audience. The target audience for the translating capability is not the Haitians suffering from the disaster, but rather people away from the disaster who can translate French and Haitian Creole to English. You also need people who understand the nuances, idioms, and slangs of the Haitian language. The target audience also needs to have access to the Internet so they can download texts from the crowdmap and send the translated ones back.

One possible group is Haitians living in parts of Haiti that are not affected by the disaster. However, your previous target audience analysis shows that most Haitians do not have access to the Internet. You could get them to help using SMS, but then you have to pay the cost of the SMS and it slows down the process. Also, if victims are sending texts full of abbreviations because they do not have space left, the person translating it over SMS will also use the abbreviation to save space. Due to the complications, you eliminate this group from your target audience considerations.

Another group is the Haitian diaspora living in the United States, which consists of at least one million people. Much of the diaspora has extensive contacts with relatives living in Haiti, so they are caught up with the culture and language. Also, many in the diaspora likely have access to the Internet and will likely be willing to help their relatives in Haiti during a disaster. The media environment in the United States is, of course, over-saturated. The media environment for the Haitian diaspora is somewhat limited by their preferences but still very similar to that of other Americans. Young members of the diaspora use Facebook and Twitter, whereas older members watch television and read newspapers. A cursory search on Facebook shows that many people in the diaspora form social media groups with the intention of connecting diaspora members in the United States. They also form groups dedicated to helping Haiti. Indeed, this target audience has helped past crowdmap deployments in Haiti with translating Haitian text messages, so the chance they will do so again is fairly high.⁷

In summary, the target audience and media environment analyses reveal the following key insights:

• The Haitian diaspora in the United States and similar countries such as Canada are the ideal target audience. They understand the language, have regular access to the Internet, and are willing to help.
• Many of them are active on social media such as Facebook, but also consume traditional media.
• The target audience is inherently incentivized to participate because they want to help their relatives in Haiti.

Keep these insights in mind while designing the crowdsourcing capability.

Design the Platform

The first part of designing the platform involves determining exactly what the platform should look like. The second part involves designing the incentive structure, which, in turn, can affect the look of the platform.

Determine the Platform's Look and Feel

The target audience analysis reveals that the crowdsourcing platform should be based on social media technologies because the target audience actively uses social media. The platform should enable crowdmap moderators to submit texts that need translation to participants. The participants should then translate the texts and send them back to the moderators. Figure 10.3 shows a graphical overview of the platform. Note that the problem type falls near the objective part of the spectrum and the approach type falls between the mixed and niche part of the spectrum.

Figure 10.3 Crowd translation platform overview

The platform really has only one component. It is the piece of virtual real estate where the participants will receive the texts, translate them, and send them back out. In all, the platform is relatively simple and has the following features:

• Messaging—Obviously, participants need to receive and send the texts that need translation.
• Task Tracker—The platform should have a capability that enables you or your moderator to keep track of which texts have been translated and who is doing the translation.

You do not need to design the interface for the platform because platforms with your desired features already exist. Ideally, you should create your own platform, but for now you can simply use one of the existing platforms. By using existing platforms, you will save resources because you will not need to build a new platform from scratch. You will also save time because you can put an existing platform to use much quicker than you can create a new one. Amazon's Mechanical Turk and Facebook are good existing platforms to use. Other platforms provide further customization options, but you do not have time to fiddle with those options now. Mechanical Turk is better suited for when the incoming flow of texts is not too great and you are willing to dedicate manpower and a little bit of time to be organized. Facebook is more ideal for when you desperately need help and are willing to sacrifice order for a quick turnaround. When you have more time you can also create tools that automatically use Mechanical Turk or Facebook to recruit participants and get the translations done. But for now, you will need to settle with using moderators to do most of the unglamorous coordination work.

Determine the Platform's Incentive Structure

The target audience is very involved with Haiti and has a history of helping their relatives. Parts of the diaspora will be willing to participate due to their intrinsic motivation to help people from their homeland. You need only to make it clear to the participants that their efforts are going toward helping Haitians. The great thing about crowdmaps is that you can share them on the Internet. If needed, you can show the participants the crowdmap filling up with reports in real time, and you can also show the crowdmap receiving updates from responders saying they are taking care of victims.

Some parts of the target audience and others who you may not have initially considered as part of the target audience but still have the language skills to qualify may need extrinsic incentives. If the rate of participation is low, you can introduce small amounts of cash as incentives for participants to translate texts for you. Most likely, though, you will not need the extrinsic incentives. Based on past disasters, diaspora and others are willing to spare some time and resources to help out disaster victims as long as they know that their help is needed.

Build the Platform

For the crowd translation capability, building the platform entails using existing social media platforms. The platforms are Facebook and Mechanical Turk, and they differ greatly in how you use them. In essence, Facebook enables you to crowdsource translation of individual texts more quickly without using extrinsic incentives, whereas Mechanical Turk enables you to translate texts in batches, but more slowly and usually by using extrinsic incentives. You should use both platforms just in case you find more willing participants on one than on the other. Usually, you will find more participants for such general crowd labor on Facebook because more people know about Facebook. We walk through how to use each platform. The following steps describe how to use Facebook:

You now have a simple way of getting people on Facebook to translate the texts for you. As you set up the Facebook capability, you should also set up the Mechanical Turk capability when you need to translate many texts at a time. You can also use Mechanical Turk to translate only one text at a time, although it is not as efficient. The following steps describe how to use Mechanical Turk to translate a one-time batch of texts:

We have showed you only a very simple capability to translate the texts. When you actually do deploy this sort of capability, you will want to create the capability for text to download automatically from the crowdmap and upload to Mechanical Turk or post to Facebook. Developers can easily help you implement it. Explore the Mechanical Turk platform to see how else you can use it. Go through the section that says Applications where third-party organizations can help you to easily implement and deploy more sophisticated translation capabilities.

Market the Platform

Marketing the platform to the target audience is not difficult because of the nature of the message and the rich media environment in which the target audience lives. You are not convincing people to buy a new product or take part in a risky and suspicious endeavor. You are simply asking them to help you help their relatives, friends, and countrymen. Previous pleas for help, everything from crowd-translating Haiti texts to donating money to the Red Cross, elicit massive amounts of participation.

The media environment consists of all sorts of media technologies, and you should use all of them in different phases of the marketing campaigns. Before there is a disaster, you need to spread awareness of what you intend to do with a low-key but somewhat consistent marketing effort. Advertise on Facebook and in newspapers and magazines that the Haitian diaspora reads. In the advertisements, explain what it is you intend to do and how you would like them to help. Tell them specifically that they need to sign on to Facebook and join a certain group, or sign on to Mechanical Turk and expect to find a certain task. Because you can create a group or page on Facebook whenever you like, getting people to join the group is a preferable option. On the group page, you can then advertise Mechanical Turk to them. You can make sure your initial marketing push is working by counting how many people join the group and when.

During hurricane season or in the run up to a disaster, increase your marketing campaign with a few bursts. Place ads on social media and news sites to once again remind people that their help may be needed. If you are part of a government, you do not need to pass the legitimacy test in people's eyes. If the U.S. government came to someone for help, the person would likely comply. However, if you are a company or a non-profit, perhaps working on behalf of a government agency, you should look to partnering with Non-governmental organizations that most people trust and admire, such as Doctors Without Borders or the Red Cross. Partners can also help you spread the word about your efforts.

When a disaster does happen, you will need to once again ramp up your marketing campaign. Much of the Haitian diaspora in the United States live in New York and Florida. Take out television advertisements for those markets urging people to sign on and help. Only take out such advertisements if your marketing efforts so far have not been very fruitful. If you already have ten thousand people signed on to your Facebook group, all you need to do is send them a message through Facebook at the time of the disaster. You do not need that many participants to help unless you are running an extremely massive crowdmap operation. In such a case, you will need to implement tools that automatically process data instead of only relying on moderators to transfer messages between the crowdmap and Facebook and Mechanical Turk. Finally, after the disaster passes, make sure to send a message through Facebook or Mechanical Turk thanking people for their efforts. Give them a few anecdotes about how their efforts helped. If Haiti's history with disasters is any guess, you will need their participation again.

Manage the Platform

Depending on the number of participants and the size of the crowdmap operation, you may need anywhere from five to ten moderators for both the crowd translation and the crowdmap. Make sure to train the moderators and run them through drills. During the disaster, they should not be asking you for the password to the Facebook account. Also, give them the opportunity to play around on Facebook and Mechanical Turk and become familiar with the numerous options both sites offer. The moderators need not speak French and Haitian Creole, but it is beneficial if they do. They can then double-check some randomly selected messages to make sure participants are translating them correctly. They do need to be social-media savvy and comfortable with multitasking.

Not including their duties with the crowdmap, which we explained in Chapter 9, the moderators will have the following duties concerning the crowd-translation platform:

• Identify messages on the crowdmap that need translation.
• Ensure that the Facebook and Mechanical Turk platforms are working and updated frequently.
• Post messages that need translation on Facebook and Mechanical Turk.
• Monitor the platforms to ensure translation is being done and at an acceptable standard. If not, educate the participants about what they need to do differently.
• Thank the participants for their help regularly.
• On Facebook, message members of related groups to remind them to participate.
• Make sure the fund with which to pay Mechanical Turk is full. Keep track of the participation rate on Mechanical Turk, and if it is low, modify the payment structure to incentivize more participants.
• Update the crowdmap with the translated messages.

If you implement development tools into Mechanical Turk, Facebook, and the crowdmap, your moderators' duties may change. For example, they may not need to post messages from and to the crowdmap, but they will have to ensure that any automated message-posting tools are working correctly. They will need to open up and maintain lines of communication with developers in case of problems.

Measure the Platform's Performance

Facebook and Mechanical Turk enable you to collect performance data easily during and after the disaster. Depending on privacy settings on your page or the page of another group that you are using, you can see all the posts and comments ever posted on the page. You can then go back and see who translated which messages. Mechanical Turk provides several tools and a dashboard with which you can track which participants completed which translation tasks, and other relevant data. Use such tools and the moderators' overall assessments to measure the platform's performance. Example metrics include:

• Number of messages successfully translated
• Number of messages translated incorrectly
• Number of participants who signed on to a group
• Number of participants who actually translated a message
• Peak time when participants were most active
• Average time to translate a posted message

Wash, Rinse, Repeat

The process we detailed so far is not very sophisticated, and for good reason. One benefit of using social media data and technologies is to do things that you can do traditionally, such as translating, through much more inexpensive methods. Also, social media enables you to improve and execute spontaneously, and modulate the scale of your effort as it transpires. If you need more translators and you need to boost participation, you can post tasks on the Mechanical Turk platform and pay people to complete them. However, you will find that simply leveraging people through Facebook is enough.

In some cases, though, you will need a much more sophisticated effort. If your crowdmap is receiving hundreds of text messages an hour and they all need translating, your moderators cannot keep posting messages back and forth between the crowdmap and Facebook. You will need to hire developers to build you relatively simple tools that post the messages automatically, and sift through the messages automatically looking for ones that need translation. Many of these tools are language-agnostic and so you can use them for other crowdmaps in very different places. You will still have to find and incentivize the target audience, but your job will become slightly easier. Also, you need to implement more formal tools and processes to verify the text messages and their translations.

Another more sophisticated tool you can implement is machine-learning language processing tools that essentially learn Haitian Creole and other languages over time. They basically look at how people read and translate certain words and phrases, and then they learn from those people and change how they read and translate those same words and phrases. Eventually, the machine-learning language processing tools can replace your need to crowd-translate entirely. Crowd-translation platforms then become tools to teach computers how to translate less popular languages. So the next time a disaster comes to Haiti, you will not need to recruit and incentivize as many participants. The machine will pick up the slack. Finally, think about eventually building your own crowd-translation platform. With your own platform, you can better control privacy settings and limit who sees what information and for how long. With your own platform or not, and the machine-learning translation tools, you can then translate other types of texts including other types of intelligence.

Crowdsource Tools to Identify Antagonistic Actors in Video Feeds

Solution platforms are ideal for crowdsourcing responses to complicated, scientific problems. In Chapter 7, we described how governments have used solution platforms to crowdsource the solutions to everything from designing computer programs that make sense of shredded documents to unmanned aerial vehicle schematics. The sponsoring organizations then use the crowdsourced solutions as starting points for developing much more robust and operational tools. The organizations also receive kudos from their colleagues and the public for using a relatively inexpensive method for coming up with a solution that would have cost much more if done through traditional means.

Like many other technologies, social media not only enables the solving of complex problems, it also creates new problems to solve. One problem is the need for governments, analysts, and security officials to make sense of the gargantuan amount of video that is now available on the Internet. The amount increases when you include videos from other sources such as closed-circuit television, security cameras, and cameras belonging to apprehended individuals. The majority of the videos serve no purpose; however, some video clips offer revealing insights about security events and suspects. Specifically, some video clips such as the ones taken at the London Tube before and during the July 2005 attacks in London show the suspects in action.⁸ Also, YouTube videos and news video feeds during the Arab Spring show the actions of pro-Mubarak forces infiltrating the anti-Mubarak protests to cause chaos and incite violence.⁹ Suspects and other antagonistic actors, who intend to cause harm or infiltrate peaceful crowds, behave differently than peaceful people around them. For example, their position or gait may be different from everyone else around them or they may make many physically imposing movements. Some tools can identify and discriminate between different types of behavior on the individual and group level, so some tools can also theoretically identify antagonistic behavior and thus individuals.¹⁰

Identifying antagonistic individuals who are behaving strangely in video feeds of otherwise peaceful settings can help security officials preempt the antagonistic individuals before they commit harm. Officials can then keep an eye on them to make sure they do not cause problems or to make sure they are not persons already thought to be suspicious. However, few if any tools can detect antagonistic individuals and behavior on video quickly and with great accuracy. Also, some tools that can do so cannot process the large amounts of video that are now available. Solution platforms provide the ability to crowdsource processes and tools that can process large amounts of video feeds to detect antagonistic individuals quickly and accurately. The following sections walk through the process of creating such a platform.

Define Objective and Scope

Imagine that a domestic law enforcement force tasks you with creating a tool that can detect antagonistic individuals on real-time video feeds and immediately notify security officials. You have a moderate-sized budget but a lack of ideas and a need to get started quickly. Overall, you expect that coming up with a robust and operational solution will take time. However, you need to jumpstart the process and need to crowdsource ideas and prototype tools off of which you can build.

You have done some research on your task and have acquired a decent-sized archive of videos of antagonistic individuals and normal, peaceful individuals in numerous contexts and settings. You can crowdsource tools and test them against this archive of videos to see which ones look promising. By testing the tool on different types of videos, you can also ensure that the tool can work against any other video, even ones that are not in your archive. You realize this problem is a difficult one and so you are open to all considerations and ideas. Because the project is at such an early phase, you also do not have to abide by security restrictions and can accept solutions from all over the world, barring a few special exceptions.

With these considerations in mind, you can formulate your objective. You need to create a platform that crowdsources the capability to identify different antagonistic individuals in different videos quickly and accurately. The capability must work on numerous videos subsequently. After identifying antagonistic individuals, the capability or tool should then somehow tag the individual on the video and send a notification. Note that because you are not wedded to any solution set and are open to any idea, the objective does not state exactly how the capability, or in other words, the tool, needs to work. The boundaries to the solution for this problem are limited to the capability's performance and not its makeup.

Analyze Target Audience and Media Environment

Because you are not bound by idea type, need for secrecy, and region, almost anyone can make up part of your target audience. The problem you are putting out to the crowd is a very difficult one that requires expertise in a number of disciplines. A group trying to solve the problem will need some background in at least behavioral science, video processing, and computer programming.

Anyone could become part of your target audience; however, not everyone will. You need to use some factors to limit your target audience, because advertising to and accommodating everyone is not realistic or cost-effective. Most people around the world will either not care about solving the problem, or will not have the resources, expertise, and time to do it. At a minimum, your target audience needs to have access to the Internet to participate in the platform. They also need to have some level of formal or informal technical expertise or education so they can begin tackling the problem. Also, only a small subset of the world's population will be interested in the problem and dedicate their free time to figuring out how to detect strange-behaving people in videos. Based on examining the types of people who talk about and participate in similar crowdsourcing platforms, you can stereotype a little and expect that most of your participants will tend to be geeks, scientists, entrepreneurs, young people interested in such topics, and older people with disposable time. Looking at other platforms also tells you that they incentivize such people through cash, their interest in solving a problem that they find appealing, and even bragging rights. Thus, you can refine your target audience to include people globally interested in behavioral, security, and video technology topics who have disposable time and regular access to computers and the Internet.

It is easy to imagine the media environment for such a target audience; at least it is for us, because we fit into it. Your target audience's media environment is rich and over-saturated. They likely use Facebook and Twitter regularly, and frequent websites dedicated to scientific and technology topics such as WIRED and Scientific American. They also likely frequent sites, blogs, and forums that specifically talk about video technology and human behavior.

In summary, the target audience and media environment analyses reveal the following key insights:

• The target audience is made up of well-connected and tech-savvy people from around the world.
• Their media environment is very rich and skews toward new media content, such as Facebook and websites that talk about technology and science.
• Based on their participation in similar platforms, the target audience is incentivized through extrinsic items such as cash, and intrinsic items such as their interest in the problem and willingness to compete against like-minded individuals.

Keep these insights in mind while designing the crowdsourcing capability.

Design the Platform

The first part of designing the platform involves determining exactly what the platform should look like, which will matter a great deal in this platform because you will design it from scratch. The second part involves designing the incentive structure, which, in turn, can affect the look of the platform.

Determine the Platform's Look and Feel

The crowdsourcing platform should be a website for a number of reasons. One, everyone around the world can access a website easily and without the need of special devices such as smartphones. Two, websites provide you the space and tools to feature a variety of content including the videos you want your participants to process. Three, websites are easy to advertise and link to from other sites and platforms.

The platform will have a simple overall process that is made up of many component steps. Initially, participants will register on the platform either as individual participants or as teams. You will then post ten or so short, different looking videos from different environments, most with antagonistic individuals in them and some without. These videos are known as the training videos. You will know who the antagonistic individuals are in all the videos concerned. The participants will then develop a video processing capability and test it on the training videos. After a certain time period, so that participants have the time to develop a tool, you will then release a completely new video and invite participants to quickly process it and send you a note identifying the antagonistic individuals in the video, and saying so if none are in the video. You will score the participants based on how quickly and accurately they identified and notified you of the antagonistic individuals. This completely new video is known as the test video. Over the lifetime of the platform, at, say, a specific time each week, you will release several new test videos and aggregate the participants' scores over all the videos. Figure 10.4 graphically represents the platform overview.

Figure 10.4 Video processing platform overview

Clearly state exactly what you want the participants to do and fully describe how you will score their tools. On the website, create a page dedicated to explaining instructions, providing samples, and answering frequently asked questions. Note that the problem type falls near the subjective part of the spectrum with a few boundaries, and the approach type falls near the mixed part of the spectrum.

The platform is made up of three components. The first is the website that participants will access to interface with you and your videos. The second is the data management system and servers that will host the videos and the website, and store participant information. The third is the back-end web interface that enables you to keep track of the participants and their scores, and upload videos. In all, the platform will have the following features:

• Messages—Participants can send messages to each other on the platform through a forum interface so they can collaborate on ideas and find potential partners. Participants and platform moderators should also be able to message each other with submissions, questions, and comments.
• Video archive—Participants can download the training videos and the test video easily from the platform. The servers hosting the videos must be able to take on a substantial amount of traffic.
• User accounts—Participants can create accounts for themselves (including their teams). Participants must specify a team name and have the option of submitting demographic information about themselves. To safeguard privacy, encourage the participants to pick names for themselves or their teams that mask their true identity.
• Notifications—Moderators should be able to send notifications to the participants by posting on the platform and by sending e-mails to the participants about the posting of new test videos. The notifications should come far in advance of the posting of the test video so participants have the time to prepare and fix their schedule.

The website interface has no set look. Check out existing government platforms using the links we provided in Chapter 7 to see examples of what your interface should look like. Overall, the website should not be overly flashy or have lots of colors and banners. You want the participants to navigate the website easily, not get seizures. The most important thing about the website is making sure that participants can access the videos easily and submit responses to you. The back-end interface that you and your moderators access should provide the following features:

• Messages—You and the moderators should be able to easily send messages, e-mails, and notifications to the participants.
• Tracking system—Keeping track of the participants' scores is essential to the platform. A digital tracking system integrated into the platform will make sure you do not make any mistakes. If need be, it also enables you to then show the participants their scores.

You do not need to create a completely separate interface for your moderator controls. You can simply integrate them into the website that participants use, but make sure you integrate passwords to protect participants and outsiders from accessing the moderators' tools.

Determine the Platform's Incentive Structure

The first step in creating an incentive structure is to examine the incentive structure of other similar platforms. Our research shows that the majority of the platforms, at least the most successful ones, tend to offer moderate-sized cash prizes. Smaller challenges feature cash rewards worth somewhere in the hundreds or thousands of U.S. dollars, whereas bigger, more comprehensive challenges feature rewards worth tens of thousands of dollars. Cash rewards are an ideal incentive to offer target audiences that are wide and made up of people with different backgrounds. However, the size of the cash rewards may change depending on the region in which your target audience mostly resides. Generally, the more money a member of the target audience has in general, the more with which you need to incentivize them. Past platform participation information suggests that most of our target audience resides in the West, where cash reward sizes are generally larger than they are in less developed areas. Almost everyone wants money and will expend effort to get it. Also, providing the cash reward is easy because you can simply mail out checks. Overall, consider providing the participants with the top three aggregate scores at the end of the platform with a major cash prize worth tens of thousands of U.S. dollars. Depending on the tools you receive, you can also hand out smaller cash prizes at your discretion to participants who you feel were the most creative or innovative.

Some past platforms also use intrinsic incentives. Participants like to compete with each other, especially when it comes to solving a complex problem. Successful participants feel accomplished and can brag to others about their skill. You can intensify the drive to compete by letting participants know how their colleagues are doing and how they are faring in comparison. You will have to reconsider your design so far to integrate such a feature.

Some incentives combine extrinsic and intrinsic motivators. For example, you can offer the participants with the highest score at the end of the platform the opportunity to continue working on their tools with others in a more traditional contractor setting. The winning participants will receive compensation for continuing work on the tool and perhaps eventually converting it into a business, and they will feel good about building something tangible and long-lasting from their initial contribution that goes on to help secure the world.

Assuming you will provide the large and small cash rewards and the opportunity for the participant with the highest score to work long-term on the project, clearly state on the website what the prizes are and how participants can win and collect them.

Determine the Platform's Look and Feel, Redux

To provide participants with an incentive to compete with each other and boost their efforts, you need to make it obvious to the participants how their efforts compare with others. A leaderboard is a simple feature that fulfills the delivery of the intrinsic incentive and that you can integrate into the platform easily. A leaderboard is a ranked list of perhaps the top five or the top ten participants at a given time in the competition. Remember that you are scoring the participants based on the accuracy and speed of their tools in regard to the test videos. You are also keeping track of the scores and updating them regularly after you verify the participants' response with what you know to be the correct answer. You can then reveal the top five or ten scores and the names of the participants (teams) with the top scores at all times through a leaderboard. Some of the participants with the top scores will likely take to the forums to brag about their video processing prowess, which in turn will motivate others to try harder. See Figure 10.5 for a graphical example of a leaderboard.

Figure 10.5 Leaderboard example

Build the Platform

The video processing platform is relatively easy to build and deploy but requires some developmental expertise. Do not use existing website templates such as WordPress to build the website because you need to account for significant security and privacy concerns, especially because you are sending out cash rewards to certain participants. The fewer features you implement, such as the forums and leaderboard, the easier it is to build and maintain the website. Work with a development team that can help you design, build, deploy, and maintain the website.

Also, ask the developers to integrate tools into the platform that enable you and the moderators to upload videos quickly and easily, and download and track participant responses. The website should also update the leaderboard automatically with the rankings at all times.

Market the Platform

The marketing effort should consist of an initial marketing push that starts a few weeks before the launch of the platform, and depending on the participation rate, a few marketing bursts soon after the platform's launch. Unlike for other platforms, you will want to focus most of your marketing energy on the initial campaign. The platform has a definite end point and participants win a cash prize depending on the scores they aggregate over the lifetime of the platform. Participants who engage with the platform well after the release of a few test videos will be at a disadvantage in terms of aggregating scores and winning the major cash prize and the opportunity to work on their tool long-term. Thus, you should concentrate most of your efforts to making sure participants sign up at the beginning. However, if your participation rate is low, you may need to advertise a little after the platform's launch. Check out past platform participation rates to determine a realistic participation rate.

Because the target audience is tech-savvy and spread around the world, you will need to focus your initial marketing campaign at media that reflect the target audience's key interests. Of course, their key interests are aligned with your platform's topic, as it should be for all target audiences you pick. A few weeks before the platform launches, start advertising on sites, such as WIRED and Scientific American, that the target audience tends to frequent. Contact news organizations and urge them to write about the platform. Also, advertise on social media, and target the ads at Facebook users who list technology topics as their interests and on Twitter hashtags that have to do with technology and science. Additionally, find a few forums or websites dedicated to video processing and visual behavioral recognition and advertise there. Ideally, you will want to have a presence on such forums well before you start your advertising campaigns. You will want to post regularly on such forums and show you care about the topic to gain the forum community's trust. Eventually, you will want to subtly state information about the platform without overdoing it. No one likes being spammed at a site that they visit frequently. The strength of the marketing campaign should increase in intensity as the platform launch date nears to create buzz and get the participants excited. Also, ramping up the advertising will remind the participants who already intend to participate that the platform will launch soon and that they need to clear up time in their schedule to work on the tool.

If participation rates are low, release an infrequent burst of ads at the aforementioned sites. As participants sign up on the platform, try to collect demographic information about them. You can then understand exactly who your target audience is and where you should focus your marketing. If you have already released half of your test videos and participation rates are low, do not bother advertising any more. It is too late for participants to join, create a tool, and then start aggregating scores. Your platform may be a failure; however, you can always deploy it again.

Lastly, make sure to e-mail the participants regularly about the release of upcoming test videos. You will especially want to e-mail them during the time period between the training and test videos when participants are developing their tools. You will want to make sure that they do not forget, and remain engaged at that critical time.

Manage the Platform

The platform will have a few social messaging features, such as the forum, that will require moderation. However, because such features are not the focus of the platform, you will not need too many moderators. Expect to hire one to two moderators depending on the number of participants and the extent to which the posting of videos and aggregating of participants' responses is automated. The moderators will have the following roles and duties:

• Monitor the forum to make sure participants are abiding by the rules and refraining from picking fights with each other.
• Answer all participant questions about the platform and rules.
• Explain the rules to participants.
• Make sure participants' responses are collected and scored properly.
• Ensure that the platform is working properly, including the uploading of videos and the leaderboard.
• Inform participants that they have won and make sure they get their rewards in a timely manner.

Measure the Platform's Performance

Use the tools integrated into the platform and other web analytical services such as Google Analytics to measure the platform's performance, which includes the participation levels on the platform and the success of the participants in regards to creating a useful tool. Example metrics include:

• Number of participants registered on the platform
• The scores of the participants and how close they are to perfect scores
• Time between releasing test videos and receiving responses
• Number of participants per team
• Number of page visits
• Number of video downloads

Wash, Rinse, Repeat

If any parts of your platform fail or cause confusion, it will be the parts that deal with the scoring and the nature of the participants' responses. You will find that what you thought of as intuitive, the participants might think of as confusing—hence, the need for moderators to interact with the participants and learn their insights. You may want to consider releasing the test videos on an irregular schedule. As we discussed in Chapter 8, an irregular schedule can increase the participants' interest and eagerness to participate. However, keep in mind that an irregular schedule could backfire because it may upset the participants' lives too much.

Now that you know how to design and deploy crowdsourcing platforms to collect intelligence and solve problems, you can start creating crowdsourcing platforms to achieve the relatively harder objective of influencing populations. Chapter 11 will describe how to build crowdsourcing platforms to influence populations.

Summary

• Crowdsourced solutions are the solutions to problems and tasks you collect from crowdsourcing platforms known as solution platforms.
• A limited number of ways to traditionally contract out work exist, but you have many different ways to crowdsource solutions.
• Thinking about the type of problem you want solved can help you formulate the problem. Solution platforms can help you solve three types of problems:
  • Objective problems that have a right or wrong answer and usually only one method of coming up with the answer
  • Subjective problems that have no right or wrong answer and numerous methods of coming up with the answer, with some being better than others according to some criteria
  • Mixed problems that may have a right or wrong answer and numerous methods of coming up with the answer, with some being better than others according to some criteria
• Thinking about which approach you want to use or how you want to solve the problem can help you define your platform's look and features, the identity of your target audience, and the intensity and focus of your marketing campaign. Solution platforms employ three types of approaches:
  • The wisdom of the crowd approach involves asking a large, heterogeneous target audience the answer to objective-type problems, and thus requires a large marketing campaign but a platform with few social networking features.
  • The niche approach involves asking a small, specialized target audience the answer to mixed- or subjective-type problems, and thus requires a focused marketing campaign and a platform with social networking features.
  • The mixed or nested approach involves asking both types of target audience or a heterogeneous version of a specialized audience the answer to mixed- or subjective-type problems, and thus requires a moderately sized and focused marketing campaign and a platform with some social networking features, depending on the case.
• Solution platforms suffer from a few limitations:
  • Some problems are too difficult for crowds to solve.
  • Participants may provide incorrect solutions.
  • Adversarial participants may bias other participants and wreck collaboration.
  • Problems stated politically incorrectly may engender scorn from politicians, corporate heads, and public relations departments.
• Compared to other types of platforms, everything about a solution platform is highly dependent on the type of problem you want to solve and the approach you want to take to do it. Also, the success of every solution platform is completely dependent on how clearly you state the problem to the crowd.
• You can use solution platforms to crowd-translate text messages from affected victims during disasters.
• You can use solution platforms to crowdsource tools to identify antagonistic actors in video feeds.

Notes

1. Lanier, J. (2010) You Are Not a Gadget: A Manifesto. Knopf, New York; Leonhardt, D. (2012) “When the Crowd Isn't Wise.” New York Times. Accessed: 27 August 2012. http://www.nytimes.com/2012/07/08/ sunday-review/when-the-crowd-isnt-wise.html

2. Surowiecki, J. (2005) The Wisdom of Crowds. Anchor, New York.

3. Asch, S.E. (1995) “Opinions and Social Pressure.” Scientific American, 193, pg. 31-35.

4. Cain, S. (2012) “The Rise of the New Groupthink.” New York Times. Accessed: 27 August 2012. http://www.nytimes.com/2012/01/15/opinion/sunday/the-rise-of-the-new-groupthink.html?pagewanted=all

5. McKay, C. (2009) Extraordinary Popular Delusions and the Madness of Crowds. Wilder, Radford.

6. Dilanian, K. (2012) “US Intelligence Tests Crowd-Sourcing Against Its Experts.” Stars and Stripes. Accessed: 27 August 2012. http://www.stripes.com/news/us/ us-intelligence-tests-crowd-sourcing-against-its-experts-1.186464

7. Meier, P. (2010) “Ushahidi and the Unprecedented Role of SMS in Disaster Response.” Ushahidi Blog. Accessed: 27 August 2012. http://blog.ushahidi.com/index.php/2010/02/23/ushahidi-the-unprecedented-role-of-sms-in-disaster-response/

8. (2008) “Jury Sees 7/7 Bombing CCTV Images.” BBC News. Accessed: 27 August 2012. http://news.bbc.co.uk/2/hi/uk_news/7377649.stm

9. Kirkpatrick, D. and Fahim, K. (2011) “Mubarak's Allies and Foes Clash in Egypt.” New York Times. Accessed: 27 August 2012. http://www.nytimes.com/ 2011/02/03/world/middleeast/03egypt.html?_r=1&pagewanted=1&hp

10. For examples of such tools, check out the New York University Movement Lab at movement.nyu.edu.