Office Hours: Jennifer Logg on the Psychology of Big Data and Perceptions of Algorithms

Historically, managers in organizations have informed their decisions using advice from other people. However, the rise of big data has increased both the availability and accuracy of a new source of advice: algorithms. These scripts for mathematical calculations sort through massive amounts of data to produce insights that can improve decision-making. Jennifer Logg, assistant professor of management at Georgetown University’s McDonough School of Business, examines how individuals can assess themselves and the world more accurately by using advice and feedback produced by algorithms. She calls her primary line of research, “Theory of Machine,” which examines how people expect algorithmic and human judgment to differ.

Why does data analytics need psychology?

While organizations are swimming in data and investing in algorithms, many are trying to understand how to maximize the benefits of algorithmic advice. And while companies focus on producing more analytical insights, it is not clear how well those insights are utilized, which leaves us asking:

1. What happens when algorithmic advice lands in the hands of managers and other decision makers?
2. When do they listen to it?
3. When do they disregard it?

It is important to understand how people respond to algorithms because they have the potential to improve human judgment and decision-making; but only if people are willing to listen. There’s a lot of research dating back to the 1950s that shows even the simplest algorithms, linear regression models, often make more accurate predictions than even experts such as doctors, loan officers, and meteorologists. If employees, managers, and customers ignore algorithmically produced advice, the organization’s time and money invested into data analytics will go to waste. 

While the field of data analytics (the systematic computation of data, most commonly using algorithms) progresses quickly, many companies overlook this important connection between producing and utilizing insights. This gap is referred to as the “last mile” problem and psychology can help solve it.

Can you explain the “last mile” problem?

The “last mile” concept is commonly used to describe how goods in a supply chain are transported from a centralized hub to the final end user. In data analytics, the “last mile” problem describes the issue of producing analytical insights but failing to communicate them well, if at all. This problem is often exacerbated by siloes within organizations. In fact, many analytics teams (focused on producing insights) do not interact much with the managers who are making decisions within their company (who could benefit from utilizing those insights and acting on them).

People on the data analytics side have asked me: “How can I present my results so that people will a) understand them and b) listen to them?” On the other hand, managers often ask me: a) “How can our organization obtain more buy-in for investment in data analytics?” and b) “What questions should I ask my data analytics team?” The answer to these questions is context-dependent and may vary from decision to decision. But these questions highlight the communication gap between producing and applying insights. Failing to communicate analytical results prevents decision-makers from acting on them. One of my favorite quotes on this topic is from Schrage (2016), “Effectively communicating and sharing analytic insights is as important as finding them.”

What is your new idea “Theory of Machine” about?

I created a framework, “Theory of Machine,” to build a program of research that systematically examines how people expect algorithmic and human judgment to differ. People’s expectations for each likely influence how they respond to information generated by both. “Theory of Machine” is a twist on the classic “Theory of Mind” from philosophy and psychology. Philosophical work on “Theory of Mind” examines how people infer other people’s beliefs and goals. “Theory of Machine” similarly examines people’s expectations about sources of information they receive, but extends that to algorithms.

In contrast to “Theory of Mind,” “Theory of Machine’ considers how people expect algorithmic judgment and human judgment to differ from each other in their input (the information used), process (how the same information is utilized), and output (the predictions, advice, and feedback that are produced). As people receive more and more information from algorithms, both in their jobs and personal lives, science needs to understand how people develop their theory of machine.

Right now, you might be thinking what and algorithm even is. What matters when studying how people respond to algorithmic advice is how everyday people define the term themselves. When participants were asked, 42% said it was math, an equation, or calculation; 26% said a step-by-step procedure; 14% said logic or a formula; and the remaining fell into a category mentioning computers. Mathematicians and computer scientists would be ok with these answers. People have a good idea of what an algorithm is.

What happens when algorithmic advice lands in the hands of managers and other decision makers; when do they listen to it and when do they disregard it? 

Many organizations are already leveraging the power of algorithms for important decisions such as hiring promising applicants (e.g., Amazon), predicting performance of current employees (e.g., Navy Seals, National Football League teams, and Premier League soccer teams), and identifying individuals who are likely to leave in order to improve retention (e.g., Johnson & Johnson). While organizations are swimming in data and investing in algorithms, many are trying to understand how to maximize the benefits of algorithmic advice. And while companies focus on producing more analytical insights, it is not clear how well those insights are utilized. Without psychology, data analytics may continue investing time and resources into producing worthwhile insights that never find an audience. 

Logg’s latest book chapter The Psychology of Big Data: Developing a “Theory of Machine” to Examine Perceptions of Algorithms was recently released on SSRN and will be released in the book American Psychological Association Handbook of Psychology of Technology (Editor Matz, S.) in 2022.

Learn more about Logg’s research and current focus on her website.