Chapter 7

Introduction to Crowdsourcing

Although crowdsourcing is an old technique, social media technology enables you to crowdsource cheaper, faster, and better. This chapter lays the foundation of Part III of the book, which is dedicated to detailing how to build and run a variety of crowdsourcing platforms for different purposes. The chapter introduces and explores the concept of crowdsourcing and its relationship to social media, details the advantages of doing it, provides a few brief examples of relevant crowdsourcing applications, and explains when and when not to do it.

What Is Crowdsourcing?

So far the book has taught you how to take a more passive approach with regards to social media, namely to consume and analyze social media data. In this chapter you learn how to adopt a more active role, and build and run social media platforms for various purposes through the technique of crowdsourcing. Crowdsourcing via social media will help you do analysis, and the analysis you conduct will in turn help you crowdsource more effectively.

Crowdsourcing will soon replace social media as the hot new frontier of technology, because it offers tremendous, game-changing opportunities. Understanding crowdsourcing and how to do it will not only place you at the forefront of technology applications, but provide you with the tools to change how you accomplish a variety of missions, from getting community members to help local police identify thieves and murderers to encouraging Afghans to text you the locations of enemy combatants. Through crowdsourcing, you can accomplish previously unattainable objectives and accomplish existing objectives in a more efficient and inexpensive way. Also, building and running crowdsourcing platforms will help you bypass complicated ethical and privacy concerns surrounding using data collected on third-party social media platforms like Twitter. Instead, you can directly engage with and gain permission from the data source to use their data. Although crowdsourcing is difficult, doing it effectively can give you enormous leverage, insight, and influence.

Defining Crowdsourcing and Its Relevance

Chapter 2 briefly defined and introduced crowdsourcing as follows:

Crowdsourcing is the act of influencing, incentivizing, and leveraging crowds through social media to provide you with information and help you solve problems.

Expanding on the definition, crowdsourcing essentially involves taking a complex problem that is difficult and expensive to solve, splitting it up into smaller tasks, and then incentivizing people to solve the smaller tasks and, consequently, the larger complex problem. Crowdsourcing is similar to outsourcing but they differ somewhat. When you outsource something, you typically hand over the problem to an external entity that then coordinates its execution. When you crowdsource something, you hand over the problem to thousands of individuals and take the lead in managing and coordinating its execution. Outsourcing is also a lot more formal, and is backed up by contracts, whereas crowdsourcing is more informal and you may never even know the identity of the person who completes your task. You can use crowdsourcing to accomplish a variety of objectives, but in this book we concentrate on the three objectives described in Table 7.1 and further detailed in Chapters 9, 10, and 11. Note that you cannot crowdsource influence per se. However, you can use crowdsourcing platforms to foster engagement and interaction with populations, which can lead to influencing populations. The word “influence” here has the same meaning as it did in Chapter 5. Influence connotes delicately persuading people over a period of time to adopt your point of view or participate in a behavior that you desire. It does not connote brainwashing or spreading obvious misinformation. See Figure 7.1 for a visual summary of the objectives of crowdsourcing.

Table 7.1 Crowdsourcing Objectives

Figure 7.1 Crowdsourcing for Intelligence, solutions, and influence

Bolstering Crowdsourcing with Social Media

Crowdsourcing is not a new technique, but social media and the Internet have changed how it is done and amplified its advantages. Law enforcement and governments have used crowdsourcing to collect intelligence, solve problems, and influence people for decades. For example, police often place pictures of missing children on milk cartons, or have local TV networks show them on the news to crowdsource intelligence about the children. They then encourage thousands of local citizens to call in with tips, with extrinsic motivators like cash rewards or intrinsic motivators like the good feeling people get when they are helpful.

Many agencies and offices in the U.S. government contract out work to private entities and individuals, and ask the public to submit their opinions and recommendations concerning controversial policy—a way to crowdsource solutions. In the former, money is the clear motivator, whereas in the latter, it is the sense of civic duty. Government crowdsourcing efforts are also subtly influencing people. In 2010, the U.S. State Department, in partnership with several embassies, technology groups, and funding partners launched the “Apps4Africa: Civic Challenge.” The contest allowed individuals from all over the world and especially from within East Africa, to submit ideas for technological solutions to everyday problems faced by Africans continent-wide. The problems spanned issues of governance, healthcare, education, and transparency. The contest offered prizes ranging from simply recognition to $15,000 for the best idea, but the real driving factor for many was the chance for individuals to vote on the problems they felt most necessary and feasible to address through technological solutions. For those voting and judging there was no reward other than civic duty. The challenge also implicitly encouraged populations in East Africa and elsewhere to use technology and entrepreneurship to solve their communities' problems.¹

Social media is the ideal vehicle for crowdsourcing and has taken it to the next level. Due to social media, crowdsourcing has taken off in recent years and will only become more popular. Crowdsourcing via social media involves both building and maintaining dedicated websites and virtual platforms with social media feature integration such as the ability to talk to other people, and using existing social media platforms to advertise dedicated platforms or crowdsource tasks. Contemporary dedicated crowdsourcing platforms can include standalone websites, smartphone applications, and SMS-based communication networks. They generally fall into eight categories, many of them overlapping, which Table 7.2 describes. The crowdsourcing platforms you will learn to build and run fall into several of the categories. Explore the crowdsourcing efforts in each category to come up with ways you can use crowdsourcing to accomplish objectives we do not specifically address.

Table 7.2 Crowdsourcing Application Categories

Social media significantly bolsters crowdsourcing primarily because of three reasons. One, social media enables you to reach more people. As explored later, crowdsourcing is more effective when you can reach a greater amount of people or a greater amount of types of people. Because social media use is nearly ubiquitous around the world, it assures you that more people will hear about your crowdsourcing effort and likely participate. Two, social media makes it easier for people to communicate and collaborate with each other. Many crowdsourcing efforts often require people who have never met to work together to solve complex tasks. Social media technologies integrated into dedicated crowdsourcing websites or existing social media platforms, such as hi5, provide numerous collaboration and communication tools that also make it easier for you to oversee and talk to the people to whom you are crowdsourcing. Three, social media can handle lots of multimedia and thus is more engaging and fun. No one wants to participate in something that is boring and dull. Through social media, you can use videos, pictures, audio, and lots of other ways of communications to entice and excite potential crowdsourcing users. The more interesting and fun your effort looks, the more people will likely participate.

Why Use Crowdsourcing?

Crowdsourcing via social media not only bolsters the act of crowdsourcing, but also your ability to accomplish the three aforementioned objectives. Specifically, it helps you solve more problems and get more information, more quickly, more discreetly, and for less money.

Solve More Problems and Get More Information

Solving more problems, completing more tasks, and solving seemingly unsolvable problems is a function of getting more people to help, trying out new creative approaches, and leveraging the wisdom of the crowds. Crowdsourcing via social media enables you to engage with more people than ever before, and more types of people than ever before. This greater reach increases the likelihood that you will engage with the person who has the solution to your problem or the piece of information you need. Often, complex problems are made up of so many diverse smaller technical and scientific niche problems that it is difficult to assemble all the experts who can solve the larger complex problem. Crowdsourcing enables you to tap into experts in niche areas living around the world that otherwise you may not be able to find or incentivize using traditional outsourcing or employment methods. For example, Amazon's Mechanical Turk platform enables you to hire individuals who can translate texts in obscure languages such as Somali at relatively cheap rates.² Additionally, researchers who are sitting in their offices thousands of miles away from the country they are researching may not have the same insights into what is happening on the ground as locals, despite how good the researchers are at using Google. If you want an accurate picture about local conditions, sourcing intelligence from local inhabitants is the best way. Finally, to influence and co-opt populations in denied areas, you have to engage with the population. They will react more favorably to you and your efforts if you talk with them on the Internet through crowdsourcing platforms than by dropping flyers on their head.

Tapping into more people and different types of people also increases the overall creativity involved and provides alternative perspectives with which to approach and solve problems. You may find it shocking to learn that the Washington D.C. metro area hosts security analysts and policy-makers who have largely gone to the same schools and read the same blogs. Thus, they largely share the same ways of approaching and solving problems. Such groupthink corrodes the ability to get things done competently and intelligently. People alien to D.C. and similar cities and organizations can bring to bear new perspectives and help develop effective and creative solutions to problems. They can also provide new viewpoints with which to acquire and process intelligence. If they are the populations you are interested in co-opting and engaging with, they also are probably the best at evaluating and correcting your efforts.

Apart from niche experts and fresh perspectives, crowdsourcing also enables you to solve more problems through the wisdom of crowds. James Surowiecki popularized the term, which posits that the average of the responses of many people to a question is often more accurate than the response of a few experts.³ If you want to guess the weight of an object, it is better to ask a hundred people what they think and then average their responses than ask the alleged weight-guessing champion of the world. The phenomenon is valid partly because more people means more perspectives are included, and also that more people bring to bear more information and insights to a problem that an expert may have missed. Keep in mind that in some cases the wisdom of the crowd will fail you—specifically, when the problem requires extremely specialized and technical knowledge, or when the crowd is not diverse enough. Crowdsourcing via social media is perfect for leveraging the wisdom of the crowd, because on the Internet you can tap into lots of people from all sorts of backgrounds, including very specialized and technical ones.

Work Quickly

Solving a problem traditionally is arduous work. You have to hire employees or contractors, and hope they are as good as they claim. Doing so usually involves lots of paperwork and bureaucracy. When unforeseen problems appear or more nuanced information and expertise is required, projects can go into a tailspin and fall behind schedule. Crowdsourcing can bypass bureaucracy and make it easy for experts to find you, instead of you finding them. Lots of people doing small tasks or giving you little bits of information simultaneously means your project gets done faster, and that you are never lacking for manpower. As long as you set up your crowdsourcing effort properly, you can let people on the Internet and social media take care of most of the problems you would face traditionally. They free up your time for other things, and their constant interaction with you and each other means things get done at a constant, quick rate.

Work Discreetly

Organizations in the security field often need to work discreetly. For instance, they need to collect intelligence from denied areas and hostile populations who are suspicious of them or hesitant to talk to them for fear of reprisal. Social media allows people to communicate with each other while hiding their identity, and even hide the fact that they are communicating at all. A person can create anonymous profiles and fake identities, or private message or text others and then delete the messages. Crowdsourcing platforms are ideal for collecting intelligence without compromising the identity and lives of the people who are providing you with information. You can also hide your identity and intentions when creating a platform, although as you learn in Chapter 8, that is not always a good idea.

Organizations conducting information operations on populations also need to work discreetly and subtly. People are more likely to agree with you and adopt your way of thinking if you can make them believe that they came up with the idea or perspective. Doing so requires subtlety and discretion, and not making it obvious you are trying to convince someone of something. Crowdsourcing platforms enable you to subtly communicate and encourage people to adopt your ideas and influence, while maintaining deniability and face. Through incentives on crowdsourcing platforms, you can discreetly nudge populations to think and do how you wish them to, and make it seem as if they are convincing each other.

Save Money

Crowdsourcing can provide enormous cash savings. Setting up a crowdsourcing platform and then running it with monetary incentives is much cheaper than contracting out work or hiring employees. Creating a web page, smartphone application, or mass texting people is cheap because the technology is widespread. After your project is finished or changes in focus, you can easily shut down and adapt your platform to other needs. You cannot as easily get rid of subpar employees or retrain them. Threadless (www.threadless.com) is a pioneering crowdsourcing company that encapsulates how crowdsourcing lowers costs. Threadless is a website that crowdsources T-shirt designs. Every week users submit T-shirt designs, and vote and comment on each other's designs. At the end of the week, Threadless founders pick the design with the most votes and positive feedback, and print T-shirts with the design. The user with the chosen design gets about $2000 as a reward. Threadless then sells the T-shirts for a lot more money. Threadless does not reveal its annual revenue but it is estimated to be in the tens of millions of U.S. dollars. Threadless makes so much money from just printing T-shirts because its profit margins are large and its costs are low. Instead of hiring T-shirt designers, it crowdsources designs and provides cash rewards that are small when you consider the company's revenues, but large for amateur T-shirt designers.⁴ Due to crowdsourcing, Threadless saves a lot of money and, thus, makes a lot of money.

Depending on the population and your skills, you can even motivate people to provide you with information and solutions without providing monetary incentives. You will be surprised at how many people will help you out for free or in return for things such as market information. We provide examples of non-monetary incentives in Chapter 8 and in the subsequent examples.

Relevant Examples of Crowdsourcing

To illustrate how crowdsourcing is relevant to the problems you face, this section briefly reviews four popular crowdsourcing platforms. The platforms span solving collecting intelligence, humanitarian and military problems, and influencing populations. In practice, all social media platforms are influence platforms because they promote communication between people. The greater the communication between people, the more likely they are to influence each other. Lots more examples are available throughout Part III.

OpenIDEO

IDEO, a California-based design consulting firm, founded OpenIDEO (www.openideo.com) as a way of galvanizing organizations and people globally to come together and solve humanitarian challenges. OpenIDEO is a website where governments, corporations, individuals, and large non-profits work with OpenIDEO to come up with challenges that usually focus on solving a health or environmental problem in developing countries. See Table 7.3 for a list of relevant challenges. OpenIDEO then posts the challenges on its websites and invites users from around the world, who sign up for free, to come up with solutions to the challenges. The users often collaborate on solutions using social networking and messaging tools built into the OpenIDEO website. Users then vote on solutions and incrementally improve them as the challenge continues. At the end of the challenge period, the organization sponsoring the challenge may then adopt and implement one of the solutions, sometimes with the users who came up with it.⁵

Table 7.3 OpenIDEO Challenge Examples

What makes OpenIDEO unique is that OpenIDEO users do not receive any type of concrete reward such as cash or prizes. The users participate because they want to work with others, play a role in solving an important societal challenge, be helpful, or simply have fun working on something that is not their normal job. Keep OpenIDEO in mind when considering how to incentivize participants to join and stay with your crowdsourcing efforts.

DARPA Shredder Challenge

Apart from development, health, and environmental problems, crowdsourcing can also help solve military and intelligence problems. The U.S. Defense Advanced Research Projects Agency (DARPA) frequently launches crowdsourcing challenges. In late 2011, DARPA launched the Shredder Challenge with the objective of acquiring computer algorithms that could reassemble and make sense of shredded documents. While collecting intelligence, law enforcement, intelligence, and military agencies frequently come by shredded documents that hold valuable information. An algorithm that could help reassemble the documents would significantly help intelligence collection. With the goal of creating such an algorithm, DARPA launched a crowdsourcing website where it hosted pictures of five shredded documents and then invited teams and individuals from all over the world to use the pictures to reassemble the documents. DARPA pledged a cash prize of $50,000 to the first team that could reassemble the documents. It used social media to advertise the challenge, which also got coverage in popular science magazines and websites. In only 33 days, the challenge was solved. Nearly 9000 teams competed, and the “All Your Shreds Are Belong to U.S.” team, based in San Francisco and made up of computer programmers, was the first to succeed and win. Lots of experts were skeptical that the challenge would work, but they were clearly proven wrong.⁶

By crowdsourcing, DARPA saved time and money because it did not have to contract the work out. It also tapped into the creativity and different perspectives of people on the Internet that led to an ingenious solution in record time. DARPA will likely build on the algorithms and solutions it received through more traditional means and move forward toward completing its objective. Crowdsourcing how to reassemble five shredded documents gave DARPA an immense head start to creating an algorithm that could reassemble hundreds of shredded documents. Expect DARPA and other U.S. government agencies to launch more crowdsourcing challenges, and keep track of them at www.challenge.gov.

If you research DARPA's history of crowdsourcing, you will likely come across one of its most popular crowdsourcing projects known as the “Red Balloon Challenge” or the “Network Challenge.” In the challenge, DARPA deployed ten red weather balloons throughout the United States. They then asked teams to identify the physical location of each balloon. The first team to submit correct location information for each balloon would receive a cash reward of $40,000. A team from MIT solved the challenge by incentivizing people on social media platforms to help them identify and locate the balloons. In other words, the MIT team crowdsourced the location of the balloons. Essentially, the DARPA crowdsourcing challenge was solved by a crowdsourcing technique. The Red Balloon Challenge illustrated the power of crowdsourcing and specifically of social media networks to coalesce people to collect intelligence and solve seemingly complicated problems.⁷

GCHQ Spy Recruitment Challenge

U.S. governments are in no way the only governments experimenting with crowdsourcing. In some ways, British authorities are leading the way with crowdsourcing. In late 2011, the British intelligence service, the Government Communications Headquarters (GCHQ), launched a crowdsourcing platform with the objective of recruiting potential employees who are good at cracking codes and thus ideal for analyzing signal intelligence.

For the challenge, GCHQ posted a code at www.canyoucrackit.co.uk and invited British citizens to solve it. The code consisted of 160 letters and numbers arranged in a rectangular display. Those who decrypted the code would find a keyword that, if entered, would lead to another website. The website congratulated the solvers and asked if they were interested in applying their skills to combating terrorism and cyberthreats. Interested solvers could then submit a job application for 35 open jobs. At least 50 people have since solved the code.⁸

GCHQ employed a relatively simple and straightforward crowdsourcing approach. The website itself was interactive and allowed for two-way communication, and popular social media platforms such as Facebook only played a role in marketing the effort. Still, the GCHQ example illustrates how crowdsourcing through the Internet can even be used for something as banal as hiring intelligence analysts. Subsequent chapters reveal how the British government and the London Metropolitan Police are using other crowdsourcing efforts to solve security problems, especially those that are most relevant to city police officers.

M-Farm

Crowdsourcing can also help intelligence collection via the more straightforward task of incentivizing people to submit data about their lives and environment through social media. M-Farm is an organization based in Kenya that is dedicated to providing farmers in East Africa with information about crop prices in their area. It hopes that farmers will use the price information to make more informed decisions about how they should set their own crop prices, thereby increasing market transparency and profits in East Africa.

Crop buyers and sellers in Kenya register with M-Farm and, in return, get access to a truncated phone number known as a shortcode. They then send a free text to the shortcode providing the price of a commodity in a certain location. M-Farm agents at local markets also text the price of various commodities. M-Farm's central database then aggregates and analyzes the price information. Crop buyers and sellers can also send a free text to the shortcode asking for the price of a commodity in a certain location. M-Farm's database responds with the desired information through a text. M-Farm has now evolved to helping farmers sell their crops using SMS and the Internet. It has also apparently realized its data is valuable and has started selling it in various formats.⁹ Because Kenya is one of the few African countries with a stock exchange, the future implications for such a service could be huge in driving country, region, or global prices based on micro-level inputs from the crowd.

M-Farm illustrates how an organization can collect critical economic and other (security, health) information from areas and populations in seemingly denied areas using dumb phones. M-Farm created a virtual crowdsourcing platform on the SMS communication network and incentivized people to participate simply by providing them with information. Because more information increased the ability of the farmers to make better decisions, they in turn became more likely to volunteer information. Also, because correct information is critical, the farmers are also more likely to volunteer correct information. Note that the overhead costs for running such a crowdsourcing platform are very low because dumb phones and SMS are relatively inexpensive. They are much cheaper than hiring and deploying numerous in-field data gatherers.

Knowing When to Crowdsource

The preceding examples illustrate only a few potential applications of crowdsourcing via social media. We review several more examples and applications in the subsequent chapters, and we hope you recognize that the applications are numerous. However, numerous applications do not result in ubiquitous applications. Some cases are appropriate for crowdsourcing and a few are not.

When to Crowdsource

When you should crowdsource depends on what you are trying to accomplish. If your objective is to collect intelligence, you should crowdsource when you need to:

• Be independent of existing but uncooperative social media platforms—Some social media platforms share data but some do not for various reasons, including because they do not like you. Or, the country you are interested in has banned its populace from using Western-based social media platforms, which are usually more open about sharing data. Launching and maintaining your own crowdsourcing platform ensures that you do not have to deal with unfriendly platforms or get around bans of popular platforms. Countries are unlikely to ban platforms that are used by a small portion of their population or are built under benign auspices, such as providing economic data to farmers.
• Collect intelligence discreetly from certain areas—Some population somewhere distrusts you and will not provide you with intelligence. However, a few individuals in that population may if you can protect their identity and compensate them. To communicate with such individuals, you can build discreet crowdsourcing platforms, most likely through SMS. The platforms would allow sympathetic individuals to provide you with intelligence without compromising their safety.
• Control the type and rate of data— Popular social media platforms are currently willing to share data, but that may not always be the case. Also, social media platforms typically limit what kind of data you can get and how much of it you can get in a time period. Maintaining your own platform frees you of dependency and limitations on platforms, which understandably are hesitant about sharing too much data about their users.
• Create and maintain an exclusive data source—Many of you likely come across information that you would rather other governments and organizations do not acquire. Limiting whom Facebook or M-Farm shares their data with is difficult. If you have your platform that is collecting the type of data you need, you can control who else gets access to the data.

If your objective is to solve problems, you should crowdsource when you need to:

• Solve problems that require expertise in lots of areas—As discussed, crowdsourcing enables you to quickly reach and employ experts in all sorts of niche subjects.
• Solve problems cheaply—In an age of dwindling budgets, crowdsourcing done well can significantly lessen the cost of solving some problems.
• Solve tedious tasks efficiently—Some problems are easy to solve, but hiring the people to solve them can be difficult and arduous. Crowdsourcing platforms enable you to quickly farm out simple tasks and get them solved without the hassle of dealing with a slow bureaucracy.
• Generate creativity and new insights—Allowing people from around the world to work on your problem will broaden the information and viewpoints brought to bear on the problem. A dose of new data and perspective usually leads to creative solutions.

If your objective is to influence populations, you should crowdsource when you need to:

• Influence people subtly—The worst way to convince people of believing or doing something is making it obvious that you are trying to convince them. Crowdsourcing platforms enable you to subtly engage with populations and seed ideas. You can then work with the target population to develop the idea so they feel a sense of ownership about the idea, and come around to your perspective.
• Be discreet about your objective—Obvious convincing and co-opting tactics will likely draw the attention of adversaries. Virtual platforms are usually safer and more discreet for everyone involved.

When Not to Crowdsource

Regardless of your objective, you should not crowdsource when:

• Confidentiality and secrecy is very important—Crowdsourcing requires putting things out on social media and the Internet, which anyone can access. Obviously, if you think a piece of information or a problem is confidential and valuable, you should not post it on the Internet.
• Your tolerance for risk is very low—Anytime you engage with populations over social media, you are giving up some control to the population. Loosening the grips on what the population can say and do is essential to fostering participation and creativity. If you are terrified of what a population may do or say on your website, you should not run certain types of platforms where open interaction is allowed. We teach you how to manage the population and manage the risk, but some risk will always be there. The risk is what makes the Internet and social media so great.

Now that you are familiar with crowdsourcing and how it has proven useful in the security world, you can begin crafting and deploying your own crowdsourcing platforms. Chapter 8 will show you the ingredients that go into building a crowdsourcing platform, regardless of your ultimate objective.

Summary

• Crowdsourcing represents the frontier of social media technology application.
• Crowdsourcing involves harnessing people on the Internet and social media to solve a series of simple tasks that result in the completion of a complex problem.
• Use crowdsourcing to collect intelligence from denied areas, solve various problems, and influence populations globally.
• Governments and law enforcement have used traditional modes of crowdsourcing for decades. Crowdsourcing is not a new technique; social media has simply bolstered it and amplified its abilities. Crowdsourcing through social media enables you to reach more people, foster communication and collaboration between participants, and use multimedia to attract participants.
• Today, crowdsourcing on the Internet is done through standalone websites, smartphone applications, SMS-based networks, and existing social media platforms.
• Through crowdsourcing, you can solve more problems and get more information. Also, you can do it all much more quickly, discreetly, and inexpensively.
• Several examples exist of crowdsourcing efforts that are relevant to security and development issues. Some include OpenIDEO, the DARPA Shredder Challenge, the U.K. GCHQ Spy Recruitment Challenge, and M-Farm.
• Crowdsourcing is most effective and appropriate for collecting intelligence when you need to:
  • Be independent of existing but uncooperative social media platforms.
  • Collect intelligence discreetly from certain areas.
  • Control the type and rate of data.
  • Create and maintain an exclusive data source.
• Crowdsourcing is most effective and appropriate for solving problems when you need to:
  • Solve problems that require expertise in lots of areas.
  • Solve problems cheaply.
  • Solve tedious tasks efficiently.
  • Generate creativity and new insights.
• Crowdsourcing is most effective and appropriate for influencing populations when you need to:
  • Influence people subtly.
  • Be discreet about your objective.
• Do not crowdsource when:
  • Confidentiality and secrecy is very important.
  • Your tolerance for risk is very low.

Notes

1. U.S. Department of State (2012) “Apps4Africa:About.” Accessed: 16 July 2012. http://apps4africa.org

2. Amazon MTurk (2012) “Introduction.” Accessed: 15 July 2012. https://www.mturk.com/mturk/welcome

3. Surowiecki, J. (2005) The Wisdom of Crowds. Anchor, New York.

4. Chafkin, M. (2008) The Customer is the Company. Inc. Magazine. Accessed: 11 June 2012. http://www.inc.com/magazine/20080601/the-customer-is-the-company.html

5. OpenIDEO (2012) “How It Works.” Accessed: 11 June 2012. http://www.openideo.com/faq

6. DARPA (2011) “DARPA's Shredder Challenge Solved.” Accessed: 11 June 2012. http://www.darpa.mil/NewsEvents/Releases/2011/12/02_.aspx

7. Greenemeier, L. (2009) “Inflated Expectations: Crowd-Sourcing Comes of Age in the DARPA Network Challenge.” Scientific American. Accessed: 24 September 2012. http://www.scientificamerican.com/article.cfm?id=darpa-network-challenge-results

8. Burns, J. (2011) “Go Online, Beat a Puzzle and Become a British Spy.” New York Times. Accessed: 11 June 2012. http://www.nytimes.com/2011/12/03/world/europe/britains-gchq-uses-online-puzzle-to-recruit-hackers.html?_r=1

9. M-Farm (2012) “M-Farm: About.” Accessed: 11 June 2012. http://mfarm.co.ke/about