How Algorithms Can Diversify the Startup Pool
Data-driven approaches can help venture capital firms limit gender bias and make better, fairer investment decisions.
When pitching startups, men and women tend to have very different experiences in being evaluated for funding.1 Consider these questions that a venture capital investor might pose to aspiring business owners:
To a male entrepreneur: “Tell us about your vision for this venture.”
To a female entrepreneur: “Tell us about your track record for this type of venture.”
Research shows that men are more likely to receive promotion-focused (risk-loving) questions from investors; for women, prevention-focused (risk-averse) inquiries are the norm.2 Investors also tend to disfavor stereotypically female behaviors, such as being soft-spoken and nurturing (versus bold and assertive), whether those behaviors are exhibited by men or women.3 But even when ventures are pitched in the same way, investors significantly prefer pitches made by men over those made by women.4
One possible explanation for these biases is the so-called cupcake stigma — the perception of women as less serious in their business ventures than the typical male entrepreneur.5 This stigma is reinforced by venture capital funding decisions, which are made mostly by men and thus based primarily on heuristics derived by men. Indeed, less than 10% of decision makers at VC firms are women and 74% of U.S. VC firms have no female investors.6 Despite evidence that suggests companies with female owners and leaders tend to outperform male-owned startups,7 the opportunities for female founders during the past decade have expanded from 1% to only 2.2% of VC funding.8 This scarcity of women in tech is exacerbated by perceptual biases related to gendered social norms and by the persistent structural challenges women face in fields related to science, technology, engineering, and math.
The Analysis
- Over roughly two months, the authors interviewed seven venture capital investors (four women, three men) in a range of roles — most of them involved in seed-stage and Series A funding decisions — from six small to midsize firms.
- They then analyzed 10 VC firms’ use of data-driven decision-making tools, including scripts for mathematical calculations and evidence-based forecasting formulas.
- The authors also reviewed the literature on decision-making, cognitive biases, and algorithms.
Some VC firms are starting to pay attention to how bias can affect funding decisions. After all, bias can have real negative financial consequences. For example, the typical small group of established funds — which share the same well-known fund managers (estimated by some as 99% male VCs9) — actually underperform newer funds, smaller funds, and women-led companies. Therefore, an investor’s hesitation to step outside his comfort zone (known as “familiarity bias”) can lead to suboptimal portfolios and a greater risk of losses. As the firm Venture Science acknowledges, “Cognitive biases are toxic when it comes to making investment decisions.”10
In an attempt to keep their biases in check, VC firms are embracing algorithms, artificial intelligence, predictive analytics, and other quantitative, data-driven approaches to making funding decisions. The popular press has heralded the potential of these de-biasing tools, but their effectiveness so far remains an open question.11
So we have set out to explore whether data-driven technologies really do help to level the VC playing field for female entrepreneurs. As part of that effort, we are examining the extent to which bias is shaping investment decisions and VC investors’ perceptions of bias in their own decisions and in the industry at large.
On the basis of our emerging findings, we describe below how biases (related to gender and other demographic factors) tend to creep into VC decision-making, some of the data-driven approaches to tease out those biases, and how algorithmic methods can help to offset them. We also offer concrete recommendations that VC firms can use to mitigate bias in the profiling of entrepreneurs who seek capital for startups. Our goal: to help VC firms make less biased, more quantitative investment decisions that serve both the firms themselves and the entrepreneurs who need their funding.
How Investors Size Up Their Prospects
Early-stage investors often lack quantifiable data and therefore face great uncertainty in deciding which entrepreneurial ventures to fund.12 For that reason, many VC firms rely on cues from the founder that might predict future success. That’s when bias can insinuate itself into decision-making.
Fit and likability. Perceived “fit” was a criterion weighted heavily by the investors we interviewed. For example, one venture capitalist discussed investing in a company whose two millennial female founders sought to market their product to millennial women. “It’s clear that the company should be built by them,” the investor said. “Had it been two dudes from Stanford who were 22 years old, we probably would be like … [there is] no founder-market fit.” The assumption was that the gender and age of the entrepreneurs had to match those of the startup’s target customers.
It’s easy to imagine how apprehension about gender incongruity between aspiring entrepreneurs and target customers could blind a VC firm to other aspects of fit. The entrepreneurs might possess, for example, a core commitment to a social cause that potential customers also value. Let’s say that women who are environmental activists want to launch a company that makes men’s products from recyclable materials. Establishing fit (and overriding investors’ assumptions about gender) would take some work. The founders would need to emphasize the “values” connection with customers in their pitch. Indeed, research has shown that social impact framing — or telling the startup’s story in a way that highlights social or environmental benefits — can lessen the perceived threat of gender incongruity in VC funding decisions.13
Perceived fit, whether merited or stemming from bias, is often tallied on a CEO scorecard, a simple data-driven tool that many of our interviewees favor. So are entrepreneurs’ “likability” and “passion.” All subjective judgments are converted into quantitative measures.
One venture capitalist acknowledged that she made the bulk of her funding decisions based on whether she expected to enjoy being “in a relationship with these people” for years. Another admitted that venture deals are not always made with a strict business calculus: “I’m not going to invest in the business if I don’t feel like I want to spend the next seven to 10 years working on it.”
Although some of our interviewees acknowledged that interpersonal connection is vulnerable to bias (“touchy, feely, fuzzy stuff that’s hard to quantify,” as one put it), all emphasized that how much they liked the founder could predict future business success in a useful way. One told us, “Some founders, you talk to them and everyone says, ‘This is an amazing person, and I want to work with them.’ When people have that quality, they tend to be more successful with their company.”
Overreliance on gut instinct. Being thesis-driven (VC-speak for having a clearly defined area of investment focus), incorporating data and CEO scorecards, forging consensus, and relying on gut instinct all were decision-making approaches used by venture capitalists in our sample. A common goal was “balancing intuition with data.” Investors reported that both are essential.
This combination of data analysis and pattern recognition, derived in large part from founders’ past successes and failures, cognitively and emotionally reframes risk around investment decisions.14 Gut feelings are akin to, as one investor described it, “a smart algorithm [an experienced VC] has built up over time of monitoring human behavior. … Some investors are good at reading people, and that’s their superpower.” Gut-level decision-making, according to the folks in our sample, “lets you go outside of the rules,” “allows you to be faster,” and “keeps investors from overthinking deals,” what one researcher calls analysis paralysis.15
All of the investors we spoke with complemented hard data with gut feelings to some extent, but not all acknowledged that bias can color how they interpret a founder’s interpersonal signals. Decision makers who lack that awareness may trust their gut too much, leaving themselves even more susceptible to flawed judgment. For example, their perceptions can be readily skewed by gender stereotypes: Without realizing it, investors might be drawn to competent, passionate male entrepreneurs but put off by women who exhibit the same levels of competence and passion.16
Some investors were more attuned to this risk than others. One interviewee, a South Asian woman, expressed a keen desire to mitigate it: “I worry that a lot of the gut feeling or pattern recognition that we talk about could just be prejudice. And it could be that there are no prominent, unicorn, leading black women founders — and therefore the pattern doesn’t match if a black woman founder pitches you. I have gut feelings, and I want to try to dig into my brain and figure out where they came from.” Others sounded a bit more resigned. One observed, “There’s a lot of bias in the system because of [gut decision-making]. It’s unfortunate, because there are founders from all kinds of backgrounds that could be successful, but they don’t get the shot they deserve.” Even so, another remarked, “Only in VC do people proudly say, ‘I get a gut feeling on someone.’ More often than not, that’s as likely to mislead you as it is to get you to the right answer.”
Passively wishing for diverse candidates. Many of our interviewees lamented the lack of female- and minority-led businesses in which VC firms can invest. One remarked, “We get so few female founders. … A lot of that has to do with the pipeline coming through — so few are women.”
Some investors mentioned trying to diversify the pool of prospective entrepreneurs by looking outside their own networks for startups to fund. Most, however, did not describe actively seeking out new prospects. Their ideas focused primarily on mitigating the bias that creeps into decision-making processes, given the existing pool of candidates. Suggestions included making feelings more objective by writing them down, adding diverse decision makers to their firms, and standardizing investment screenings — all useful methods, but still constrained to sizing up the usual suspects.
Current Data-Driven Approaches
Our sample of investors recognized that one potential way out of the bias morass was to seek objective data when they find it lacking. One said, “The more data-driven I can be, the happier I am.” Acquiring and then using good data was widely seen as a means toward making better, fairer decisions about which ventures to fund.
Outside the startup world, algorithms guide a substantial portion of the decisions we make — in retail, media consumption, hiring and job seeking, even dating. VC firms have begun to use algorithms to scout for potential investments,17 screen ventures for viability,18 and now reduce bias in decision-making.
EQT Ventures, a Stockholm-based VC firm, has created an AI machine-learning tool called Motherbrain that tracks roughly 8 million startups and flags those that have promise.19 A U.S. firm, Correlation Ventures, has compiled a massive proprietary database that uses predictive analytics to inform investment decisions.20
For later-stage investments in startups that already have users, global VC firm Follow[the]Seed relies on its RavingFans algorithm, which uses reverse problem-solving to identify pain points and solutions and to assess “customer obsessions.” Then, on the basis of these inputs, the firm decides whether to invest in a venture. According to partner Eliav Alaluf, the algorithm “is interested in the potential … not in the hair color, gender, religion, and CV design of their founders.”21
Beyond enhancing efficiency, algorithmic aids can help investors become aware of, and potentially overcome, biases in decision-making.22 For example, Venture Science uses a quantitative investment strategy that incorporates AI and decision theory to compute the risk associated with a variety of decision-making categories — from vision and team completeness to geographic proximity to tech centers to market size and sales funnels.23 A team of the firm’s analysts identifies decision-making parameters and weights each one according to its importance. Then a numerical value, qualitative scale, or utility function is assigned to a given parameter in order to yield an overall framework. Once this framework is established by the team, individual members make independent assessments of each criterion. Their aims are to avoid biases that often arise in group decision-making (such as anchoring or the availability heuristic) and to “illuminate controversy” by making people conscious of their assumptions, facilitating debate, and fostering compromise.
Social Capital, a firm that invests in entrepreneurs at all stages, has pushed to prioritize data in the VC process in order to actively work against bias. Through the company’s online platform, founders self-select for funding consideration and submit their transaction data to an “automated diligence engine” (called Capital as a Service) that can output funding decisions in a matter of hours. The firm’s efforts have yielded a much-higher-than-typical ratio of underrepresented founders: Of the startups selected for funding, 42% were owned by women, more than half the founders were nonwhite, and they represented 12 countries.
Other non-VC organizations in the entrepreneurial ecosystem are also using data-driven approaches to narrow the gender gap in funding of entrepreneurs. The nonprofit Female Founders Faster Forward (F4) Capital is using data analytics as it aims for its goal of getting 20% of VC funding to female-founded startups by 2020.24 F4 Capital is in the process of developing the Startup Investment Model Index, which would give founders an objective score measuring startup maturity, opportunity, and risk to help focus VC funding.
Alice — an AI platform created in partnership with Dell, accelerator Circular Board, and software company Pivotal — was designed to help connect female and minority entrepreneurs with necessary resources to scale their startups. Dubbed the “Siri for female entrepreneurs,” Alice was built on the success of another platform its founders created: Circular Board. A virtual accelerator, Circular Board served almost 300 female entrepreneurs on six continents, who collectively raised more than $65 million for their ventures.25
These examples seem like promising applications of data-driven decision-making, but in some cases it is still unclear to what extent they have actually enabled investors to limit biases in the decision-making process. As VC firms and researchers figure out how to measure these new tools’ concrete effects, it is crucial to develop a nuanced understanding of the key challenges and opportunities that algorithm-based decision-making presents in a VC context.
Algorithmic Decisions: The Challenges
Research on algorithm-based decisions across various disciplines suggests they could substantially narrow the gender gap in VC funding, in part by making decision processes more transparent and reducing the bias that creeps into the process. The literature specifically shows that compared with humans, algorithms are typically less biased and more accurate.26
Still, algorithm aversion — people’s reluctance to trust and use algorithms for making decisions — is a real problem. Even though (on average) they outperform humans, when algorithms make mistakes, people lose confidence in them more quickly than they do in humans who err.27 That’s largely because folks falsely assume that human decision makers improve with experience and that algorithms cannot incorporate qualitative data.28
Some people also believe that algorithms are dehumanizing or, for some important decisions, ethically inappropriate.29 Multiple experiments have shown that algorithmic decisions are perceived as less fair than human decisions when the content is evaluative and people-related.30 This phenomenon is driven, according to the research, by the belief that algorithms can’t make holistic decisions about humans because they reduce a complex individual to a mere set of numbers.
In a VC context, algorithm aversion surfaces when investors assume that human decision makers are better at identifying team dynamics and unearthing information through personal connections with founder teams.31 One of our interviewees noted, “It doesn’t take long for me to realize if there are issues between the founders, little things they do and say. I’ve had entrepreneurs bark at each other and forget I’m sitting there. Those things eventually come out.” Investors value such interpersonal cues heavily, for better (such as when they portend good relationship potential) or worse (when they perpetuate gender bias) — cues that might be difficult for an algorithm to incorporate.
Scholars have theorized that people’s aversion to algorithms stems from intolerance for algorithmic error.32 But researchers have found that giving people control, even just a bit, over the algorithms they use reduces algorithm aversion — specifically, that people were more likely to use imperfect algorithms for forecasting when they could modify them.33
Recommendation: Enable decision makers to exert some control over the algorithmic decision process. One way to give VC investors control over a mostly algorithmic approach might be to complement it with a tool such as the CEO scorecards mentioned by our interviewees. The CEO scores would be numeric values for subjective qualities such as perceived passion, how well founder teams get along, and commitment to the problem. Such inputs might help investors feel more in control and, therefore, more likely to trust model outputs.
Recommendation: Give investors the chance to embrace algorithmic advice during the decision-making process. Recent experimental evidence highlights people’s preference for algorithmic advice, which guides decision-making but leaves it to humans to make final judgment calls. Such feedback might be conceived as “something to consider as you’re weighing your options” rather than “a choice you should or must make” — a gentler concept to make algorithmic models more palatable to people who are skeptical of them.
Research also suggests that when making quantitative decisions, laypeople are actually more likely to adhere to advice from algorithms than from an external adviser’s estimates, whereas experts are more likely than laypeople to be averse to feedback from algorithms.34 Experts may resist using algorithmic advice out of fear that their jobs will become obsolete. After all, algorithms provide instant, inexpensive forecasts. Framing what algorithms offer as a complement to, rather than a replacement for, expertise can help to assuage such fears.
Algorithmic Decisions: The Opportunities
Despite the challenges that algorithms present, most notably in gaining people’s acceptance, clear successes have been achieved in a wide range of settings beyond the startup world. In the mortgage-lending industry, for example, automated underwriting algorithms have predicted defaults more accurately than have manual underwriters, allowing homebuyers from traditionally marginalized groups to successfully qualify for mortgages.35 Similarly, in a recent study, when a software company used algorithms for decision-making in hiring, algorithms were more likely than human decision makers to avoid bias against women and people of color, actually favoring those candidates.36
Decisions about which startups show the greatest promise can also benefit from such approaches. Even when algorithms incorporate historical data, they have the potential to reduce (though not eliminate) bias and give underrepresented groups a fairer shake.
Recommendation: Develop algorithms to increase transparency and identify potential instances of discrimination. Despite their seeming opaqueness, algorithms have the potential to increase transparency by formally identifying and weighting the unconscious factors that constitute gut instinct. That assistance can help investors develop a fairer, more consistent, approach to decision-making.
Researchers argue, in a working paper from the National Bureau of Economic Research, that human decision-making is fraught with ambiguity and that algorithms, designed with appropriate safeguards, can allow people to weigh trade-offs among competing values, closely interrogate the entire decision process, and determine whether and when discrimination has occurred.37
This is not to say that algorithms automatically yield full transparency. Take, for example, Follow[the]Seed’s analysis of more than 200 data points — provided by founders — to determine how “obsessed” users are with a product or service before deciding whether to fund the venture. The algorithm focuses on three primary data categories: a critical mass of users, continuous growth in number of users, and higher-than-average retention rates compared with the industry standard. Although the algorithm is described on the Follow[the]Seed website, it does not clarify, for instance, exactly how a RavingFans value is assigned and how founder data is fact-checked.
Recommendation: Release the data on performance impact. Some of our understanding of algorithms’ usefulness is based on VC self-reporting. For example, one of Social Capital’s investing partners said that an early experiment using its data-driven approach “resulted in a much higher ratio of underrepresented founders, evidence that the traditional VC process is perpetuating bias.”38 But as we all know, self-reporting can be unreliable; when we want to see success, we’re likely to find some semblance of it. Sharing the data on how algorithmic decision-making has affected performance outcomes that matter to stakeholders imposes some accountability. It’s an opportunity to identify and fix problems in a transparent fashion, so that future uses of the algorithm may offer more value.
Algorithm-based decision-making, for all its advantages, does not eradicate bias and subjectivity because, after all, algorithms are human creations. Researchers and other experts have begun to tackle this problem by explicitly countering biases in training data sets, enhancing transparency during the design phase, and calling for more auditing before deploying algorithms.39 However, in the VC domain, there is not yet systematic, empirical research on addressing algorithmic bias against minority founders, including women. Conducting that research represents the next frontier of opportunity in limiting bias in VC funding.
The optimal balance between human and algorithm remains elusive: Both have flaws, but each is crucial for making decisions that are more effective and less prone to discrimination against women and other underrepresented groups. In the VC domain, algorithmic decision-making is still in its infancy, but studies conducted so far have raised crucial guiding questions:
- Can algorithms become flexible and adaptive enough to account for rapid changes in technology, customer demographics, project pipelines, and VC interests?
- How can algorithms, given their imperfections, best enable transparency in VC funding decisions?
- What are the most useful ways for human decision makers to complement algorithm-based decisions — and, specifically, how should such partnerships be structured?
Carefully answering these questions through research on — and input from — a diverse array of individuals and firms across the VC ecosystem is clearly an investment worth making.
Morela Hernandez is an associate professor of business administration at the Darden School of Business at the University of Virginia. Roshni Raveendhran is an assistant professor of business administration at the Darden School of Business at the University of Virginia. Elizabeth Weingarten (@elizabethw723) is a senior associate at ideas42, a nonprofit behavioral science design firm, and the managing editor at The Behavioral Scientist. Michaela Barnett is a doctoral student at University of Virginia’s Convergent Behavioral Science Initiative.
References
1. V. Yadav and J. Unni, “Women Entrepreneurship: Research Review and Future Directions,” Journal of Global Entrepreneurship Research 6, no. 12 (2016): 1-18; and K.A. Eddleston, J.J. Ladge, C. Mitteness, et al., “Do You See What I See? Signaling Effects of Gender and Firm Characteristics on Financing Entrepreneurial Ventures,” Entrepreneurship Theory and Practice 40, no. 3 (2016): 489-514.
2. D. Kanze, L. Huang, M.A. Conley, et al., “Male and Female Entrepreneurs Get Asked Different Questions by VCs — and It Affects How Much Funding They Get,” June 27, 2017, https://hbr.org.
3. L. Balachandra, T. Briggs, K. Eddleston, et al., “Don’t Pitch Like a Girl!: How Gender Stereotypes Influence Investor Decisions,” Entrepreneurship Theory and Practice 43, no. 1 (2019): 116-137.
4. A.W. Brooks, L. Huang, S.W. Kearney, et al., “Investors Prefer Entrepreneurial Ventures Pitched by Attractive Men,” Proceedings of the National Academy of Sciences 111, no. 12 (March 2014): 4427-4431.
5. V.K. Gupta, D.B. Turban, S.A. Wasti, et al., “The Role of Gender Stereotypes in Perceptions of Entrepreneurs and Intentions to Become an Entrepreneur,” Entrepreneurship Theory and Practice 33, no. 2 (2009): 397-417; and J.E. Jennings and C.G. Brush, “Research on Women Entrepreneurs: Challenges to (and From) the Broader Entrepreneurship Literature?” Academy of Management Annals 7, no. 1 (2013): 663-715.
6. K. Clark, “Female Founders Have Brought In Just 2.2% of U.S. VC This Year (Yes, Again),” Nov. 4, 2018, http://techcrunch.com.
7. N. Hashimzade and Y. Rodionova, “Gender Bias in Access to Finance, Occupational Choice, and Business Performance,” Economics & Management Discussion Papers, em-dp2013-01, Henley Business School, University of Reading, U.K., 2013.
8. V. Zarya, “Female Founders Got 2% of Venture Capital Dollars in 2017,” Fortune, Jan. 31, 2018; and Clark, “Female Founders.”
9. S. Brand, “How to Finally Fix the Gender Gap in VC,” Nov. 21, 2017, www.forbes.com.
10. “Palo Alto Venture Science: Company Details,” www.wallstreetoasis.com, accessed May 14, 2019.
11. B. Schiller, “This AI Engine Takes Common Biases Out of the Venture Capital Process,” March 28, 2016, www.fastcompany.com; A. Mirhaydari and K. Clark, “Data-Driven Investing: Why ‘Gut Feel’ May No Longer Be Good Enough,” March 15, 2018, http://pitchbook.com; and F. Corea, “Artificial Intelligence and Venture Capital,” July 18, 2018, http://medium.com.
12. L. Huang, “The Role of Investor Gut Feel in Managing Complexity and Extreme Risk,” Academy of Management Journal 61, no. 5 (October 2018): 1821-1847.
13. M. Lee and L. Huang, “Gender Bias, Social Impact Framing, and Evaluation of Entrepreneurial Ventures,” Organization Science 29, no. 1 (January-February 2018): 1-16.
14. L. Huang and J.L. Pearce, “Managing the Unknowable: The Effectiveness of Early-Stage Investor Gut Feel in Entrepreneurial Investment Decisions,” Administrative Science Quarterly 60, no. 4 (2015): 634-670.
15. Huang, “The Role of Investor Gut Feel.”
16. Gupta et al., “The Role of Gender Stereotypes”; and Lee and Huang, “Gender Bias.”
17. M. Palmer, “Artificial Intelligence Is Guiding Venture Capital to Startups,” Financial Times, Dec. 11, 2017.
18. Corea, “Artificial Intelligence and Venture Capital.”
19. A. Heathman, “Motherbrain: How AI Is Helping This VC Firm to Pick the Next Big Startup,” April 18, 2019, www.standard.co.uk.
20. Correlation Ventures, “Our Selection Model,” https://correlationvc.com, accessed July 8, 2019.
21. E. Alaluf, “How Does Follow[the]Seed Examine Investments?” April 20, 2017, http://followtheseed.vc.
22. D. Coats, “Too Many VC Cooks in the Kitchen?” March 13, 2018, http://medium.com; and Alaluf, “How Does Follow[the]Seed Examine Investments?”
23. F. Corea, “Data-Driven VCs: Who Is Using AI to Be a Better (and Smarter) Investor,” May 2, 2019, www.forbes.com.
24. F4 Capital, “Tomorrow’s Promise,” www.f4capital.org.
25. K. Hannon and Next Avenue, “Meet Alice, the Siri for Female Entrepreneurs,” June 4, 2017, www.forbes.com; and “Frequently Asked Questions: What Is Circular Board?” http://helloalice.com.
26. B. Cowgill, “Bias and Productivity in Humans and Machines,” working paper, Columbia University, New York City, Jan. 11, 2019; and A.P. Miller, “Want Less-Biased Decisions? Use Algorithms,” July 26, 2018, https://hbr.org.
27. B.J. Dietvorst, J.P. Simmons, and C. Massey, “Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err,” Journal of Experimental Psychology: General 144, no. 1 (February 2015): 114-126.
28. S. Highhouse, “Stubborn Reliance on Intuition and Subjectivity in Employee Selection,” Industrial and Organizational Psychology 1, no. 3 (September 2008): 333-342; W.M. Grove and P.E. Meehl, “Comparative Efficiency of Informal (Subjective, Impressionistic) and Formal (Mechanical, Algorithmic) Prediction Procedures,” Psychology, Public Policy, and Law 2, no. 2 (June 1996): 293-323; and D. Newman, N.J. Fast, and D.J. Harmon, “Algorithms and Fairness,” working paper, University of Southern California, Los Angeles, 2019.
29. R.M. Dawes, “The Robust Beauty of Improper Linear Models in Decision-Making,” American Psychologist 34, no. 7 (1979): 571-582; Grove and Meehl, “Comparative Efficiency”; and Y.E. Bigman and K. Gray, “People Are Averse to Machines Making Moral Decisions,” Cognition 181 (December 2018): 21-34.
30. Newman, Fast, and Harmon, “Algorithms and Fairness.”
31. Mirhaydari and Clark, “Data-Driven Investing.”
32. H.J. Einhorn, “Accepting Error to Make Less Error,” Journal of Personality Assessment 50, no. 3 (1986): 387-395; and Highhouse, “Stubborn Reliance.”
33. Dietvorst, et al., “Algorithm Aversion.”
34. J. Logg, J. Minson, and D.A. Moore, “Algorithm Appreciation: People Prefer Algorithmic to Human Judgment,” NOM Unit working paper 17-086, Harvard Business School, Cambridge, Massachusetts, April 24, 2019.
35. S.W. Gates, V.G. Perry, and P.M. Zorn, “Automated Underwriting in Mortgage Lending: Good News for the Underserved?” Housing Policy Debate 13, no. 2 (2002): 369-391.
36. Cowgill, “Bias and Productivity.”
37. J. Kleinberg, J. Ludwig, S. Mullainathan, et al., “Discrimination in the Age of Algorithms,” working paper 25548, National Bureau of Economic Research, Cambridge, Massachusetts, February 2019.
38. Mirhaydari and Clark, “Data-Driven Investing.”
39. N. Shadowen, “How to Prevent Bias in Machine Learning,” Jan. 29, 2018, https://becominghuman.ai; K. Hosanagar and V. Jair, “We Need Transparency in Algorithms, But Too Much Can Backfire,” July 23, 2018, https://hbr.org; and A. Campolo, M. Sanfilippo, M. Whittaker, et al., AI Now 2017 Report (New York: AI Now Institute at New York University, 2017).
Reprint 61104.
For ordering information, visit our FAQ page. Copyright © Massachusetts Institute of Technology, 2019. All rights reserved.