Big Data is watching you. Part two

 Introduction.

In the first part of this article, I briefly surveyed two phases of surveillance capitalism: (a) personal data extraction practices and (b) their normalization, while here (c) behavioral prediction and (d) manipulation will be considered. In the following paragraphs, we will see how the ceaselessly extracted data is used by surveillance capitalists to predict human behaviors and manipulate them to earn revenues. On the one hand, governments or companies may acquire data to prevent crimes or evaluate the credit worthiness of mortgage applicants, for instance. On the other hand, predictions can shape the (digital) context where the decision-making process of individuals takes place. Both manipulation strategies have unsettling ethical drawbacks.

1.Prediction

After data is collected it is used to build sophisticated algorithms that predict human behavior. Indeed, data is employed “to produce virtual representations – data doubles – optimized for modulating human behavior systematically [which] correlate to identifiable, flesh-and-blood human beings” (1). Social quantification (2) (the mathematical transcription of the whole human social existence) is based on a precise socio-anthropological framework. Indeed, “executives in the social quantification sector [surveillance capitalists], whether or not they planned to, are becoming social theorists, or at least trying to sound like them” (3). The social theory they adopt mirrors the pillars of the research of MIT professor Alex Pentland (4), whose essay Social physics (2014) argues that human behaviors and social interactions replicate mathematical patterns that, once recognized, allow for behavioral prediction. In Pentland’s words, “social physics is a quantitative social science that describes reliable, mathematical connections between information and idea flow on the one hand and people’s behavior on the other” (5). Behavioral predictions are thus enabled by the mapping of the idea flow (propagation of information) within a group of people, which implies that human behavior can be predicted through the observation of the information each individual is exposed to. Pentland grounds these conclusions on complex mathematical reasoning that involve stochastic processes probability. His model, he holds, predicts roughly 40% of the variance in behavior adoption (6).

A practical example helps clarify these conclusions. In the Social Evolution study, Pentland and colleagues ascertained a strong correlation between students’ development of specific habits (like weight change) or preferences (like political orientations) and their exposure to other people’s corresponding behavior (7). Interestingly, indirect or incidental exposure was more effective in shaping individuals’ behavior than explicit conversations with close friends. Using a special data extraction software installed on the students’ smartphones, Pentland’s team could detect the students’ exposure to specific behaviors (as well as the development of new habits and preferences that such exposure caused). Thus, Pentland and colleagues reconstructed the ideas/information flow in which each student was entangled and correlated it to the particular behaviors they developed. The statistical regularities they found in the extracted data enabled them to infer the likelihood of the development of a habit or preference even in the absence of its direct observation. This explains why, since the accuracy of predictions depends on the quality and quantity of the extracted data (more data, better predictions), surveillance capitalists are enticed to invest in more pervasive extractive practices.

2. Manipulation

It has been proved that “deviations from our regular social patterns occur only a few percent of the time” (8). But Pentland’s Social physics seems exclusively interested in detecting these patterns rather than investigating their origin. This approach neutralizes all “the personal, the social, and the political meanings” (9), and sustains a scenario where, “based on algorithmic reasoning, it is a real person who gets sent to jail, refused health insurance, or offered a favorable price in the supermarket or an opportunity for social housing” (10).

Data brokerage companies, indeed, predict and score citizens’ social and medical reputation (e.g. credit worthiness, propensity to crime, life expectancy etc…) following social physics’ tenets. They extract and collect data from different sources linked to the same individual (like social networks posts, credit card purchases, health records and google searches histories), and elaborate it through algorithmic procedures that they are unwilling to disclose. In this way, data brokers detect concealed patterns that, once unveiled, act as reliable predictors of future behavior. As a result, just as Pentland and his team predicted the students’ weight variations and voting preferences, data brokers anticipate citizens’ likelihood to repay their debt or to develop diseases (11). This information is then sold to banks, insurance companies and governments, who base their business decisions on it, denying health insurance to patients or adjusting the interest rates on mortgages. No wonder why the global data broker market was valued at US$257.16 billion in 2021 and is expected to reach US$365.71 billion in 2029. 

Other companies offer services that are not implemented by private companies only. Palantir’s facial recognition capabilities and back-end processes are used by the USA Department of Homeland Security “to create a framework on which to base any arrest or deportation” (12). Police activities are shifting from law enforcement (that takes place only after the crime has been committed) to predictive intelligence (that identifies potential criminals before the crime has actually occurred) (13). Let alone unnoticed biases that may enter the algorithmic processing of data, the risk of “double jeopardy” arises in all these circumstances. Recognized patterns are considered reliable proxies to predict future events, but prevent more complex social explanations from taking place (14). Less affluent people struggling to repay their debt are offered - if any - mortgage contracts with higher interest rates. Patients facing severe medical conditions pay higher health or life insurance premiums. Citizens living in disadvantaged situations may be identified as a threat to social security and subjected to even greater police surveillance. This form of manipulation affects disproportionately the least affluent and most vulnerable substrate of our society, worsening economic divides. 

But now, let’s step back to Pentland. Not only he argued that the mapping of the information flow allows for behavioral predictions, but he also saw its chances for a stealthier and obtrusive manipulation. Once the structure of social interactions is known and visualized, it can be reshaped to change the behavior of its members. This is not science fiction, but the theoretical conclusion of his eToro experiment (15). The latter is a stock exchange platform where users can interact through the social network embedded in it, exchanging information on investment strategies or mimicking the decisions of other users. By tuning and reshaping the information flow of eToro’s social network, Pentland’s team could “increase the profitability of all the social traders by more than 6 percent, thus doubling their profitability” (16). Even though Pentland insists that this process may achieve beneficial outcomes, almost ten years after the publication of Social physics we are witnessing a quite different evolution of the ideas contained in that book. Social physics’ principles have informed the business plan of companies like Cambridge Analytica that manipulated the voting behavior of 230 million citizens in the USA, perhaps even in the UK and Russia. Furthermore, Facebook seems disconcertingly uninterested in “tuning the idea flow” on its African servers, where the spread of fake news is seriously undermining democracy. 

Recommender systems are used to shape and personalize cyber spaces by filtering, ranking and organizing the information in every digital scenario. Search engines exploit these devices to give “each of us a perfect little world of our own, a world tailored so exquisitely to our individual interests and preferences that it is different from the world as seen by anyone else” (17). However, being unable to choose the options among which we decide and their order of display in the (digital) world that appears to us, our autonomy is reduced (18). Cyberlaw scholar K. Yeung coined the term hypernudge to describe the threat that recommender systems pose to human autonomy (19), where the term nudge refers to the “gentle push” exerted by the configuration of the decisional context (e.g. the positioning of salad in front of lasagna in a diner to endorse the consumption of healthier food). Indeed, in virtual worlds, exploiting Big Data and predictive algorithms, nudging techniques are much more pervasive, personalized and thus effective than in the physical world.

Now, we surely need some criteria to “tame the information overload” (20) in google queries, social networks or streaming platforms. However, most of the time, we have no insight into the cogs of recommender systems and into the interests they respond to. This is problematic especially when we consider that numerous aspects of our lives are being transferred from the physical environment to algorithmically shaped scenarios. For instance, according to a research conducted in the USA in 2017, around 40% (but the trend is increasing) of heterosexual couples met online using dating apps (21), while “social media platforms show more highly paid job advertisements to men than to women” (22). 

3. Conclusion. 

Even though it is extremely difficult to prove that “AI systems may erode human self-determination bringing to unwanted and unintended changes in human behaviors” (23), what is surely happening is an algorithmic translation and shaping of every facet of our existence. The existential choices we make (like the romantic partner we elect, the job we land or the university we apply to) may not yet be entirely driven by algorithmic elaboration. Nevertheless, we are building the technologies and ideologies to make this scenario possible. A new social governance framework is rising before our eyes and lurching in our society through stealthy techniques. The least we can do is to educate our sight to recognize it.

Sources

1.  J. E. Cohen, Between truth and power, Oxford University Press, Oxford,  2019, p. 67.

2.  The expression comes from N. Couldry and U. A. Mejias, The cost of connection, Stanford University Press, Stanford, 2019, pp. 131-150.

3.  N. Couldry and U. A. Mejias, The cost, cit., p. 137.

4.  Cfr. N. Couldry and U. A. Mejias, The cost, cit., pp. 138-139; S. Zuboff, The age of surveillance capitalism, Public Affairs, New York, 2019, pp. 433-460.

5.  A. Pentland, Social physics. How good ideas spread (ebook edition), The Penguin Press, New York, 2014, chapter 1.

6.  A. Pentland, Social physics, cit. appendix 4.

7.  A. Pentland, Social physics, cit., chapter 2.

8.  A. Pentland, Social physics, cit., chapter 10.

9.  K. Crawford, Atlas of AI, Yale University Press, New Haven-London, 2021, p. 93.

10.  N. Couldry and U. A. Mejias, The cost, cit., p: 131

11.  F. Pasquale, The black box society. The secret algorithms that control money and information, Harvard University Press, Cambridge (MA)-London, 2015, pp. 19-58.

12.  K. Crawford, Atlas, cit., p. 195.

13.  S. Brayne, Big Data surveillance: the case of policing, American Sociological Review, 82, no. 5 (2017): 977–1008. https://doi.org/10.1177 /0003122417725865.

14.  “An insurer assumes that driving after midnight equates to choosing to take more risk even if a person’s working conditions require it” (N. Couldry and U. A. Mejias, The cost, cit., p.: 147).

15.  A. Pentland, Social physics, cit., chapter 1.

16.  A. Pentland, Social physics, cit., chapter 1.

17.  F. Pasquale, The black box, cit., p. 60.

18.  “With so much discriminatory processing controlling how the world even appears to each of us, this new social knowledge gives human actors few opportunities to exercise choice about the choices offered to them. There is something disturbing about this for human freedom and autonomy and for the human life environment that is being built here” (N. Couldry and U. A. Mejias, The cost, cit., p. 129).

19.  K. Yeung, ‘Hypernudge’: Big Data as a Mode of Regulation by Design (May 2, 2016). Information, Communication & Society (2016) 1,19, TLI Think! Paper 28/2016, Available at SSRN: https://ssrn.com/abstract=2807574.

20.  F. Pasquale, The black box, cit., p. 61.

21.  M. J. Rosenfeld, R. J. Thomas and S. Hausen, (2019). Disintermediating Your Friends: How Online Dating in the United States Displaces Other Ways of Meeting Affiliations, Proceedings of the National Academy of Science of the United States of America, 116, 17753-17758. https://doi.org/10.1073/pnas.1908630116.

22.  K. Crawford, Atlas, cit., p. 128.

23.  L. Floridi, Etica dell’intelligenza artificiale, Raffaello Cortina Editore, Milano, 2022, p. 286.