Abstract
Xenophobia is one of the key drivers of marginalisation, discrimination, and conflict, yet many prominent machine learning fairness frameworks fail to comprehensively measure or mitigate the resulting xenophobic harms. Here we aim to bridge this conceptual gap and help facilitate safe and ethical design of artificial intelligence (AI) solutions. We ground our analysis of the impact of xenophobia by first identifying distinct types of xenophobic harms, and then applying this framework across a number of prominent AI application domains, reviewing the potential interplay between AI and xenophobia on social media and recommendation systems, healthcare, immigration, employment, as well as biases in large pre-trained models. These help inform our recommendations towards an inclusive, xenophilic design of future AI systems.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
The increasing scope of the field of AI ethics—and corresponding analyses of algorithmic fairness—reflect the ubiquitous nature of AI deployment across many spheres of daily life. A major consequence of this is an increasing need to operationalize standards of algorithmic fairness widely, in order to minimise the risks involved with the use of technology and ensure an equitable distribution of its benefits. To date, much attention has focused on issues of fairness and discrimination associated with the legally protected categories of race and gender. However, research has shown that these frameworks tend to be ill-suited for unobserved or less frequently-recorded characteristics such as sexual orientation and gender identity (Tomasev et al. 2021). Building upon this insight, the present paper focuses on a prominent and widespread form of discrimination which has received comparatively little attention in ML fairness research—xenophobia, which is discrimination directed at the other or at those who are perceived to be foreign.
Xenophobia shapes the political landscape of the modern world in a number of ways (Bowman 2021; Arredondo 2018; Kopyciok and Silver 2021; Yakushko 2018; Akinola and Klimowich 2018; Chenzi 2021)—as can be seen in the recent growth of authoritarian populist movements and anti-immigration political platforms (Jörke 2019; Chatterjee 2021). Moreover, the Covid-19 pandemic led to a sharp rise in xenophobia (Lee and Li 2020), and in particular to a rise in anti-Asian sentiment in the United States (Nguyen et al. 2020). There is, consequently, a moral imperative to develop a better understanding of the role AI technology plays in amplifying or mitigating xenophobia in society. Indeed, as David Haekwon Kim and Ronald R. Sundstrom observe, by ignoring xenophobic discrimination—as distinct from racial, sexual or gender discrimination—we risk neglecting specific patterns of disadvantage and harm associated with conceptions of “foreignness”. This omission then leads to a situation in which “a normative loophole” is created, one that holds that “when thinking about social, national justice we need not care about foreign distant others.” (Kim and Ronald 2014)
From a technical standpoint, existing bias-mitigation strategies in the domain of ML fairness tend to focus on ensuring equitable outcomes for legally protected groups. However, efforts to combat xenophobia in AI need to go further, as many groups that are deemed to be foreign lack such legal protection. An additional challenge lies in the need to understand the decisions people make when distinguishing between us and them, between those who are familiar and those who are not. Ultimately, political institutions often help define and impose such categorizations, with the history of the US census providing an example of how changes in data collection may reflect and entrench changing notions of identity over time.
In the context of AI research, the detection of “hate speech” or “dangerous speech” has served as a focal point for efforts to address xenophobia, fuelling the development of content moderation systems to tackle the spread of hateful and dangerous speech in social media (Fortuna and Nunes 2018; Cao et al. 2020; Awal et al. 2021; Röttger et al. 2020). Dangerous speech is an element of a path of escalation that may lead to conflict (Bowman 2021) and even mass atrocities (Benesch and Leader 2015; Fink 2018)—while early detection of dangerous speech can provide opportunities for timely deescalation. However, accurate hate speech detection is not without challenges, as early systems have been shown to incorporate racial bias (Xia et al. 2020), as well as over-reporting queer speech as inappropriate (Gomes et al. 2019). We believe that the existing focus on social media platforms needs to be complemented by deeper analysis of xenophobia, spanning the full range of AI use cases. Such analysis needs to account for, and be rooted in, recognition of structural barriers facing foreign nationals and immigrants. This is further exacerbated by wider issues of cultural assimilation, abandonment and erasure, and a lack of recognition of stateless persons (Mansouri 2023).
To the best of our knowledge, this paper is the first comprehensive review of the interplay between xenophobia and AI. In total we look at its impact across five domains: social media, healthcare, immigration, employment, as well as via large pre-trained models. We proceed as follows. First, in Sect. 2, we define xenophobia, distinguishing it from other forms of discriminatory attitudes and practices such as racism and sexism, and specifying its conceptual relevance to AI systems. We then, in Sect. 3 discuss how xenophobia may manifest across a number of AI application domains. We proceed, in Sect. 4 to make a moral argument towards a design of inclusive, xenophilic systems, coupled with a set of technical considerations and recommendations informed by the use cases reviewed in Sect. 3. We conclude by outlining key issues for future research.
2 On xenophobia
Xenophobia is commonly understood as a kind of hostility or prejudice directed towards foreigners, immigrants or, more broadly, those construed as “others”. It can manifest as a fear, dislike or hate towards people who are perceived to be different, including their culture and customs, and has been associated with a range of social and psychological roots, (Banton 1996; Sanchez-Mazas and Licata 2015; Wimmer 1997; Yakushko 2009) including misassociations, stereotyping and cognitive bias (Rydgren 2004), projections onto the other, as well as active processes of othering groups of people (Wimmer 1997). Moving beyond the cognitivist conception of xenophobia—which focuses primarily upon mental states and attitudes—it is important to recognize that xenophobia also has institutional and structural expressions. In these cases discriminatory practices or outcomes are sustained by rules, procedures, and prior distributions of material and symbolic resources which target specific groups of people. Xenophobic discrimination may involve both phenomena, typically involving a mix of attitudinal and institutional effects (Kim and Ronald 2014).
This conceptual core, however, leaves two important questions unanswered: the first concerns the distinction between xenophobia and other forms of intergroup prejudice, notably racism (the Distinctiveness Problem); and the second centers upon defining the threshold of what counts as xenophobia, given the ubiquitous differences in how rights and responsibilities are allocated to citizens and non-citizens in contemporary nation-states (the Threshold Problem).
Much scholarship on discrimination and disadvantage ignores the first problem, by adopting, as Kim and Sundstrom observe, “the general assumption, alive in popular culture and some segments of the academy, that racism has subsumed nativism and xenophobia” (Kim and Ronald 2014). This, they point out, is problematic, since it “merges racism, xenophobia, and nativism into one hyper-concept of prejudice and exclusion” and ignores “important distinctions to be made here between prejudice against racial outsiders, civic outsiders, and the pursuit of chauvinistic ethics and racial group-interests based on claims of indigenousness.” (Kim and Ronald 2014) While the history of xenophobia can not be decoupled from the artifacts of the colonial past and racism, xenophobia is conceptually distinct from racism, and the strategies needed to mitigate racism and xenophobia may differ (Bernasconi 2014).
We adopt a modified version of Kim’s and Sundstrom’s solution to this problem: namely, orientating our understanding of xenophobia around the notion of “civic ostracism”. Specifically, we distinguish xenophobia from other forms of prejudice and discrimination by defining it as a discriminatory orientation (whether of individuals or institutions) that penalises individuals on the basis of their perceived foreignness, understood here as the purported lack of full membership of the civic community. Such non-membership may correlate with actual legal status, but it need not do so: individuals who are in fact citizens but who are prejudicially deemed foreign (perhaps because they were originally born outside the civic community, or simply because of assumptions associated with accent, name, practices, beliefs or race) may still be subject to xenophobia in this conceptualisation.
A related set of points hold for the complicated relationship between xenophobic and racial discrimination. For while the category of "race" is socially constructed, contested, and multifaceted (Hanna et al. 2020; Benjamin 2019), it seems clear that a person can encounter racial discrimination without also necessarily having their membership of the civic community called into question. The reverse is also true: a person can be judged to be "foreign", and encounter the range of obstacles detailed in this paper, without encountering co-extensive racial discrimination. By way of illustration, consider discrimination against Irish Americans—which spanned several centuries—or against the former residents of East Germany after national re-unification (Zehring and Domahidi 2022; von and West 2022). Discrimination aimed at immigrants or refugees who are ostensibly of the same race as the host community (Gellner 2015; Anderson 2020) also represent an example of this phenomenon – for example, discrimination against Syrian refugees in Lebanon (Janmyr 2016). Yet, this picture is complicated in a number of ways. First, the term "racism" can, in ordinary language, be used ways that muddy these waters, encompassing discrimination on the basis of nationality—a move that expands the scope of what might be appropriately deemed "racist"—and bringing the concepts of racism and xenophobia closer together. This folk conception of racism is quite different from the one most often operationalised in AI bias research, which has tended to investigate discrimination based on skin color (Buolamwini and Gebru 2018) and to draw heavily upon the "protected characteristics" framework used in anti-discrimination law (Solon and Selbst Andrew 2016; Mehrabi et al. 2021). Second, racial discrimination and xenophobia, even when understood as conceptually distinct still often occur together and compound one another in practice. Third, and as a contributing factor to this trend, racial discrimination is often buttressed—on the part of people who hold these views—by propaganda or conspiracy theories that stress the "foreignness" of the group that is the target of discrimination (Mamdani 2002). Equally, those who hold xenophobic attitudes often try to amplify these views by reference to spurious notions of "race" and "racial purity" (Jardina 2019). Hence, both at the conceptual and the causal level xenophobia and racism often intertwine closely.
Kim and Sundstrom do not address, however, the second problem - which goes persistently unaddressed in most scholarship on xenophobia. By their very nature, nation-states distinguish between citizens and non-citizens, and in doing so, allocate rights, responsibilities and resources differentially between these two categories. Adherents to “cosmopolitan” ethics may lament this institutional reality of contemporary politics (Caney 2008; Pogge 2002; Benhabib 2008), yet the ethical significance of civic national groups that differentiate between citizens and non-citizens has cogent defenders, and the abandonment of civic nations is extremely unlikely for the foreseeable future. A framework which deems any differential outcomes between citizens and non-citizens to be xenophobic is, as such, unlikely to be accepted as offering relevant ethical guidance for AI systems. At the same time, differential outcomes for non-citizens can indeed arise from xenophobia, and may extend up to persecution, ethnic cleansing, and genocide (Mann 2005; Straus 2015; Brubaker and Laitin 1998). The recognition of such systemic and structural xenophobia must play a central role in understanding the potential xenophobic impact of AI systems.
Our proposed solution to these ambiguities, when assessing xenophobia in technical systems, is to distinguish between two different ways of operationalising the concept of xenophobia. In general, we think designers of AI systems will be interested, in the first instance, with what we term immanent assessments of xenophobia, by which we mean the determination of whether an AI system creates disadvantages on “foreigners” that are discriminatory according to the societal standards. Put another way, if AI systems generate disadvantages that go beyond the officially sanctioned differences in the rights, responsibilities and resources that separate citizens from non-citizens, the system is immanently xenophobic, violating the standards formally adhered to by its user. Yet those standards may, themselves, be subject to ethical critique as part of a more transcendent assessment of xenophobia - i.e. they violate ethical principles which, even if the actor or organization does not formally recognise or accept them, ought to be upheld. While we think transcendent assessments of xenophobia are crucial to ethical debate in global affairs, much of the time AI-systems may be rendered substantially fairer simply by ensuring that immanently understood xenophobia is avoided.
Whether we are assessing xenophobia immanently or transcendentally, we identify three main kinds of harms that may flow from a xenophobic AI system. First, a xenophobic system may result in discriminatory material disadvantages—allocating material resources in ways that systematically and unfairly penalise certain individuals on the basis of their actual or presumed foreignness. For example, an AI system may end up prioritising medical benefits to citizens over non-citizens, as well as those with names that are often associated with foreign heritage. Second, a xenophobic system may deny individuals proper ethical recognition, by formally or informally engaging with them in ways that communicate effective civic ostracization. An example of this would be an AI system that generates descriptions or images of citizens with a certain heritage that support negative stereotypes about their socio-economic status or purported "willingness to work". Third, a xenophobic system may restrict the effective exercise of individuals’ rights, for example, by exposing them to prejudicial treatment by the police, unfairly allocating access to forms of legal action, or giving unjustified prominence to certain groups over others in platforms for expressing speech. Such restriction of rights could include exposing individuals material harm via physical harm or violence if, for example, law enforcement officers rely on systems that contain xenophobic bias. However, what separates this category of harm from the first kind, is the additional and necessary threat to a person’s rights—which limits their ability to seek fair recourse and can compound the feeling of civic precarity. Moreover, while each of these harms may overlap with discrimination based on other attributes, such as race and gender, they are not subsumed by these categories. More obviously, for example, xenophobic discrimination may be directed at immigrant and refugee populations, even when such groups are not clearly perceived of as ‘racially’ distinct.
3 Practical considerations
We now focus on several important AI application domains to establish different ways in which xenophobic discrimination may potentially manifest, and where countervailing measures may need to be adopted. Since our aim here is to draw attention to an aspect of fairness—xenophobia—which has thus far rarely been addressed by AI researchers, and for which there is very little known testing, the evidence we draw upon is necessarily suggestive and somewhat indirect, drawing heavily upon analogies and examples from the wider sociotechnical literature. Our contention is simply that such evidence gives sufficient reason to think that xenophobia is likely to to manifest in AI systems, and that there should be more urgency in developing frameworks for evaluating potential xenophobic bias and downstream harms. Tangible algorithmic harms have been previously observed across these and other domains (O’Neil 2016) for various marginalised groups, and we believe that these established mechanisms of how bias may translate into harm set a precedent for the discussion presented here, as they imply that the entrenched xenophobic biases would similarly lead to adverse consequences for the affected groups and individuals.
3.1 Social media
Social media may amplify xenophobia (Daniels 2018), especially in communities with a certain amount of pre-existent xenophobic sentiment (Bursztyn et al. 2019; Yamaguchi 2013). Low barriers to entry, reliance on user-generated content, difficulties in moderation, and the emergence of filter bubbles (Baldauf et al. 2019; KhosraviNik 2017; Bruns 2019; Zuiderveen et al. 2016) present difficulties that may lead to the easier spread of fake news (Albright 2017; Schäfer and Schadauer 2018) and potentially dangerous and hateful speech (Benesch and Leader 2015) directed at those who are deemed foreign. At the same time, these platforms could also potentially provide mechanisms to enable more inclusive discourse, expressions of support and positive views towards marginalised groups, serving as a medium for communicating information relevant to the effective exercise of individual rights. This speaks to a dual role social media platforms occupy in shaping culture: both as a source of greater cross-cultural understanding, and also as a mechanism that can entrench social divisions, sometimes leading to abuse that reflects wider structural and cultural violence (Haunschild et al. 2022). In particular, it has been shown that there is an important link between social media and hate crimes, as the prevalence of anti-refugee sentiment on social media platforms is a strong predictor of crimes against refugees across municipalities (Müller and Schwarz 2021).
The dynamics of discourse and content sharing on social media are ultimately influenced both by the societal context as well as the design of the platform, its implemented algorithmic solutions, and the enforcement of content moderation policies. Carefully designed platforms may present safe spaces for information sharing and help facilitate collective immigrant action (Zepeda-Millán 2016). In general, given an ever-increasing proportion of time spent on social media, its overall impact on the amount and the intensity of xenophobia in society can be profound (Postill 2018).
3.1.1 Risks
All three types of xenophobic harms that we consider in this study may occur in the context of social media applications. To begin with, discrepancies across personalised social media feeds often incorporate harmful stereotypes and make implicit assumptions about people that are problematic from the standpoint of representation. Indirectly, these cultural tropes may shape the public opinion in ways that result in xenophobic discrimination offline, exacerbating the propensity to exclude individuals based on their perceived “foreignness”. Indeed, xenophobic harms pertaining to civic ostracism may follow, for example, from online public shaming (Aitchison and Meckled-Garcia 2021) and amplified hate speech aimed at marginalised communities. Finally, indirect xenophobic harm may result indirectly from the shaping of public opinion, as well as directly by silencing minority views (Oliva et al. 2021; Haimson et al. 2021) and weaponising content moderation against foreign individuals, resulting in their effective exclusion from public forums and an inability to influence future decisions. To address these concerns, greater transparency and contestability need to be incorporated in content moderation system design (Vaccaro et al. 2020), to ensure fair processes and help mitigate the consequences of potential algorithmic biases.
Machine learning plays an important role in emerging patterns of social media content consumption. Personalized news feeds may give rise to feedback loops (Jiang et al. 2019), and amplify pre-existing opinions and biases, resulting in autopropaganda (Joe et al. 2021) Moreover, repeated exposure to content that others out-groups and reinforces harmful stereotypes may lead to radicalisation (Alfano et al. 2018; O’Callaghan et al. 2015; Ribeiro et al. 2020), suggesting that a careful redesign of content consumption pathways is needed (Fabbri et al. 2022). Deepfakes (Westerlund 2019) present a recent manifestation of the risk of AI-enhanced spread of misinformation, either by malicious individuals and organisations, or as a part of state propaganda (Pavlíková et al. 2021). Emergence of deepfake detection technology (Güera and Delp 2018; Dolhansky et al. 2019; Zhao et al. 2021; Wang et al. 2021; Fagni et al. 2021) may help address this challenge, at least until the deepfake technology becomes advanced enough to be indistinguishable from authentic images, video, and audio streams. Adversarial robustness of deepfake detection models plays a crucial role in this context (Neekhara et al. 2021), given the known vulnerabilities of neural computer vision systems to adversarial perturbations. Yet, technological mitigations for deepfake dissemination also need to be accompanied by an investment in media literacy (Hwang et al. 2021) and user awareness. Disinformation and deception have been shown to play a key role in forming xenophobic narratives around otherness, aiming to associate foreigners with acts of violence, over-reliance of financial aid, and illegal immigration (José et al. 2021). has mostly involved reinterpreting existing images with inaccurate associated textual descriptions, creating fictitious references and taking imagery out of context. Yet, there is a risk that the rise of deepfake technology could amplify these trends by making it easier to create such disinformation at scale.
3.1.2 Promise
Advances in natural language processing may help develop more reliable hate speech detection (Badjatiya et al. 2017; Aluru et al. 2020; Gambäck and Sikdar 2017; Rizos et al. 2019; Sutejo et al. 2018; Plaza-Del-Arco et al. 2020; Khan et al. 2020; Alatawi et al. 2021) and sentiment prediction systems (Jin et al. 2020; Seo et al. 2020), as well as improved automated topic analysis (Gui et al. 2019; Wang et al. 2019) in social media posts, across different modalities (Peng et al. 2019). This improved monitoring may also help facilitate targeted interventions (e.g. counter-speech (Sonntag 2019; Tekiroglu et al. 2022)) to mitigate xenophobic sentiment, safeguard marginalised communities, and minimise the risk of escalation towards potential violent conflict. Complementary efforts in semi-automatic detection of fake news (Popat et al. 2018; Wang et al. 2020; Kaliyar et al. 2021), as well as bots and troll accounts (Luceri et al. 2020) are imperative in establishing safe online spaces. Human-in-the-loop (Strickland 2018; Demartini et al. 2020) filtering processes and explainability in AI-driven content moderation may be necessary to address biases in language systems.
Advances in diversification of recommender systems (Kunaver and Požrl 2017; Helberger et al. 2018; Mansoury et al. 2020) and improvements in user fairness (Leonhardt et al. 2018; Nandy et al. 2022) may help avoid the formation of filter bubbles and echo chambers and empower communities to confront and challenge xenophobic narratives. Content diversity needs to be accompanied by additional counter-measures to be effective (Stray 2021) and more research is needed to identify the best contextual approach to use. In particular, neural machine translation (Bahdanau et al. 2014; Chen et al. 2018; Aharoni et al. 2019; Pan et al. 2021) approaches may prove to be an invaluable tool in breaking down barriers in communication across groups, increasing exposure to contrasting views, and help establish multicultural online spaces. To achieve this, it is important to improve the accessibility of natural language understanding, in particular in terms of improving the performance of existing systems in low-resource languages (Gu et al. 2018b, a; Karakanta et al. 2018; Sennrich and Zhang 2019; Siddhant et al. 2020), through participatory action and community engagement. Finally, ML also offers opportunities for developing tools for monitoring large-scale shifts in social behaviour (Papakyriakopoulos 2020; Patti et al. 2017), which may help further the development of theories that aim to understand the root causes behind an increase in xenophobic sentiment. These theories may be driven by observations in retrospective traces of social media usage (Frías-Vázquez and Arcila 2019; Wahlström et al. 2021), or in simulation (Yao et al. 2021b; Lucherini et al. 2021), with controllable parameters so as to isolate individual effects.
3.2 Immigration
Perhaps the most obvious context in which AI systems may exacerbate or mitigate xenophobic harms is in the management of migration flows. The complexity of immigration policies reflects a tension between internal security and the liberal frame of humanitarianism (Lavenex 2001), generating a need for involvement of stakeholders outside of established political processes. Migration between culturally distinct countries and regions is often accompanied by a growing resentment of immigrants and refugees (Hadžić 2020). Meanwhile, denialism and complicity with populist narratives hamper the implementation of remedial measures—including the development of a coordinated strategy, spanning international, national and regional efforts, which is needed given the magnitude of the problem (Crush and Ramachandran 2010). Moreover, states have already been quick to employ new digital and AI technologies in an effort to more tightly and effectively control migration (Nalbandian 2022). As an official from the South African National Defence Force observes: “The European Union is trying to fight the issue (of immigration) with data\(\ldots\) In the modern digital world, you cannot exist without a digital presence. Why is it not feasible to control all of this immigration by tracking the behavior of this data, within the database?” (Longo 2017). While digital technology may promise significant gains in efficiency in the management of human movement across borders, this opens a vast realm of potential bias and harm created by AI systems that restrict rights of movement based on assumptions of risk or threat associated with certain data profiles. The nature of problems faced by immigrants is necessarily intersectional, as immigrants from marginalised communities face additional barriers when attempting to integrate into society (Bhagat 2018), and may employ identity concealment tactics to render themselves invisible in the face of xenophobic attitudes (Makoni 2020; Tewolde 2021).
3.2.1 Risks
Numerous technological solutions incorporating AI have been proposed for border control and migrant assessment. The collection and utilisation of data from refugees, asylum-seekers and migrants—including by obtaining access to their private social media accounts (Andreassen 2021)—poses serious risk of misuse(Ahmad 2020). AI systems have also been proposed for automatic assessment of immigration forms (Chun 2007), and the deployment of facial recognition technology is also a feature of this context (Carlos-Roca et al. 2018). This risks perpetuating the historical, national, ethnic, and racial stereotypes, and is especially concerning since these types of systems have already been demonstrated to be harmful due to disparate performance for racial minorities (Birhane 2022). Indeed, many governments have been quite explicit in their efforts to define migrants in terms of an algorithmic computation of ’risks’—with migration processing dictated by the resulting assessment of risk levels. For example, officials in the Australian Department of Immigration and Citizenship have publicly defended the employment of an electronic travel card system on the grounds that it enables a “risk-tiered approach to identity \(\ldots\) where we have a range of concentric circles around Australia and we are pushing (the border) further and further out, (and) are doing more and more checks before the person hits our shores” (Longo 2017). Biases and harms in AI systems aimed at risk profiling, have also been fairly well documented in a different use case—recidivism prediction in the criminal justice system (Angwin et al. 2022; Angwin and Larson 2022).
While the management of migration risks (SeyedSoroosh and Kiana 2020) may be legitimate in principle, such an approach creates enormous potential for harmful and unfair outcomes to individuals based on associative assumptions about risk that are built into AI assessment systems. These harms can potentially manifest in different walks of life - for example, systems for the assessment of the likelihood of the H1B visa approval in the US immigration process (Swain et al. 2018) can bias potential employers against certain immigrant groups. There is also an important distinction to be made here between immanent and transcendental standards in assessing xenophobic harms in immigration, where some practices may be legally permissible, yet palpably unjust. This raises important questions about migrants’ human rights in face of the use of AI technology in immigration enforcement (Giannakau 2021).
3.2.2 Promise
If xenophobic bias can be avoided, AI systems offer the potential for fairer, faster and more efficient management of migration flows with benefits not only for governments, in encouraging welcome forms of migration, but also for migrants in avoiding complex, lengthy and expensive migration processes. Indeed, AI systems have been proposed as a mechanism for encouraging states to more effectively and expansively house refugee populations, by speedily “matching” government and individual preferences and needs in the context of large refugee flows (Jones and Teytelboym 2017). Nonetheless, improved auditing of such systems is required to reduce the risk of unjustly excluding people from safe refuge. Large language models have recently been used to identify and analyse the inconsistencies in the immigration law applications for stays of removal at the Canadian Federal Court, showing that computational legal research may be able to help improve the established processes (Rehaag 2023). Advances in modelling and understanding trends in human migration (Robinson and Dilkina 2018) can also help with anticipatory measures and planning towards building sustainable and inclusive communities. Modelling refugee flows (Mead 2020) may prove helpful in expediting a humanitarian response.
3.3 Healthcare
Xenophobia has recently been identified as an important determinant of health (Suleman et al. 2018), with a multitude of adverse effects being reported for both individual and community-based health metrics. Moreover, policies that restrict the range of health services available to foreigners sometimes exceed the costs they purport to be saving, creating a hostile environment both for patients and migrant workers alike (Shahvisi 2019). Exposure to repeated xenophobic prejudice in a healthcare context may erode trust and adversely affect care-seeking patient behaviours (Earnshaw et al. 2019).
Rather than being an exclusive artefact of misguided public health policy, medical xenophobia can manifest via negative attitudes and practices by health workers, and has been shown to be deeply entrenched in numerous public health systems (Crush and Tawodzera 2014; Loganathan et al. 2019). Discriminatory practices are further amplified in case of particularly vulnerable migrant populations, like refugees (Zihindula et al. 2017; Munyaneza and Euphemia 2019) and undocumented migrants (Richter 2015). Moreover, even prior to engaging with health services, xenophobia and racism may impact health via early exposure to adverse childhood experiences (Nguyen-Truong et al. 2021). In Adja et al. (2020), the authors point out that the World Health Organization (WHO) defines health as “a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity“ (World Health Organization 1946) and that social well-being is often overlooked in conversations around health. A holistic approach for mitigating the effects of medical xenophobia must therefore incorporate notions of social well-being and account for the impact of the social determinants of health and the disparities faced by foreigners outside of the boundaries of direct medical care.
Medical xenophobia has also historically played an important role in shaping epidemiological perception related to the spread of infectious diseases. More recently, this could be seen in the overwhelmingly xenophobic response to the spread of COVID-19 (Reny Tyler and Barreto Matt 2020; Le et al. 2020) and the resulting health disparities (Hooper et al. 2020), but it is by no means a recent phenomenon. As highlighted in (Sarbu et al. 2014; Galassi and Varotto 2020), the act of blaming foreigners for the spread of infectious disease could be readily seen in the accounts related to the devastating epidemic of venereal syphilis in Europe in the 15th and 16th century. The English, Germans and the Italians blamed the French for the epidemic, referring to syphilis as morbus Gallicus (the ‘French disease”). In turn, the French accused the Germans, the Poles and the Neapolitans, the Poles blamed the Germans, the Russians blamed the Poles, etc (Sarbu et al. 2014; Galassi and Varotto 2020). Another example of historical medical xenophobia can be seen in the anti-Asian bias during the epidemic of Bubonic Plague in San Francisco’s Chinatown in 1900 (Parfett et al. 2021). Paleopathology can help alert us to past instances of medical xenophobia, raise public awareness and reduce future xenophobic discrimination in times of public health emergency. As articulated by Muscat et al. (Natasha et al. 2017): “Public health is not a neutral scientific activity, but a political activity informed by science. Public health has a moral mandate that complements its scientific mandate.“
3.3.1 Risks
Xenophobic medical AI harms include all three types of xenophobic harm that we consider in this study. Clinical AI systems developed from retrospective clinical data may encode and perpetuate harmful biases towards migrants and ethnic minorities, potentially leading to discriminatory material disadvantages—misdiagnosis, suboptimal treatment decisions and resource allocation, and worse health outcomes (Suleman et al. 2018). Such disparities have already been identified in relation to structural racism and algorithmic clinical decision-making (Norris et al. 2021; Yates et al. 2021), making their examination in the context of xenophobia imperative. Design choices in setting up the critical health data infrastructure play a role in this, in ways in which they encode identities and represent demographics (Yi et al. 2022). AI systems that take shortcuts (DeGrave et al. 2021; Brown et al. 2023), through proxies of sensitive attributes, may further ostracize communities seen as foreign and perpetuate narratives linking foreignness to disease (Yi et al. 2022). Exclusion of ethnic groups may potentially also follow from non-inclusive designs of systems that may not be suited for safe application beyond the majority demographics in the country of deployment, resulting in disproportionate rates of deferral to alternative pathways, increased waiting times, and a deep misunderstanding of patient needs. Finally, clinical AI systems may further alienate patients and erode trust, as well as potentially reinforce harmful xenophobic stereotypes in repeated interactions with systems designed to reason based on historical data.
3.3.2 Promise
There is potential for AI technologies in healthcare to play a transformative role in improving access to care and the quality of its delivery. Advances in deep learning have led to a number of promising early research prototypes for improved diagnostics in medical imaging (De Fauw et al. 2018; McKinney et al. 2020; Baltruschat et al. 2019), diagnostics from wearables (Ravi et al. 2016), early detection of adverse events from electronic health records (Tomašev et al. 2019, 2021; Futoma et al. 2017), and others. The maturity of this technology is reflected in ongoing prospective studies and deployment (Qian et al. 2021; Dembrower et al. 2023). By way of illustration, on-device screening apps may help with earlier detection of health problems in marginalised groups, including migrants and foreign workers with limited access to health insurance and health services. For example, AI-based diabetic retinopathy detection from smartphone-based fundus photography was shown to have very high sensitivity (Rajalakshmi et al. 2018) and would potentially be applicable for mass screening of at-risk populations. Similarly, advances in AI-based dermatology screening applications (Göçeri 2020) could yield similar benefit in the future.
Yet, there are still practical obstacles for these types of systems to be safely deployed at scale (Seneviratne et al. 2020). Participatory approaches play a key role in medicine (Abma and Broerse 2010; Angel and Frederiksen 2015; Falk et al. 2019; Schinkel et al. 2018), where engaging with patient focus groups helps the clinicians align on the desired outcomes and improve the overall patient experience. Participatory approaches to developing AI solutions have similarly been identified as a key ingredient in safe and inclusive AI system design (Bondi et al. 2021), though their implementation is not without challenges (Smith et al. 2021). The outcomes of such practices hinges on the inclusion of all marginalised communities and especially those that may be facing difficulties accessing the basic healthcare services, or facing language barriers (Um and Kim 2018; Nickell et al. 2019; Hannah et al. 2019).
3.4 Employment
AI systems are increasingly used in decision-making around employment (Upadhyay and Khandelwal 2018; Sánchez-Monedero et al. 2020), in helping to identify, filter, assess and prioritise potential job candidates. This has led to the development of AI ethics frameworks for use in employment applications (Chan 2022; Evgeni et al. 2023). Given that differential outcomes in employment translate to direct material harms, it is crucial to understand how xenophobia manifests in this sector overall, and how it may impact the development of AI-driven decision-support systems (Raghavan et al. 2020).
Unequal access to employment opportunities and unfair compensation are among the key drivers of social inequality as differences in socioeconomic status tend to have a knock-on effect on community health, mental health, access to housing and physical safety. Labour market attachment is also critical to the livelihood and identity formation of individuals and groups, especially when it comes to those that have been historically marginalised (Teelucksingh et al. 2007). There are a number of different ways in which employment discrimination manifests for foreign workers (Kosny et al. 2017) at different stages in the employment process, depending upon their citizenship and immigration status. Both citizens and non-citizens can suffer xenophobic harms in the employment process, although there are commonalities and differences in their respective experiences (Bell et al. 2010). Historically, the worst jobs, with the hardest working conditions and the least pay had been reserved for immigrants, leading to the subjugation of immigrant bodies and population control, fueled by the xenophobic sentiment—which persists to this day (Longhi 2012).
Immigrants play an important role in the labour market, due to their diverse skill sets, life experience, educational profiles, and background. This diversity necessitates an intersectional analysis of employment outcomes, as it has been shown that different immigrant groups experience different levels of discrimination. For example, a study in Switzerland identified highly competitive immigrant groups from neighboring countries as being subjected to most workplace incivility (Krings et al. 2014), presenting a skills paradox (Dietz et al. 2015). Such discrimination is not restricted to first-generation immigrants, rather being rooted in assumptions about heritage, names (Midtbøen 2014; Obenauer 2023) or appearance (Fibbi et al. 2006). In France, both immigrants and their descendants face worse employment outcomes (Duguet et al. 2010; Meurs 2017). Meanwhile, in Sweden, replacing Swedish with Middle-Eastern-sounding names in CVs (Carlsson and Rooth 2007) was shown to reduce the rate of call-backs by a large margin, with the rate of discrimination being amplified by male recruiters in particular. While labour-market integration in Sweden remains straightforward for Western immigrants, immigrants from Africa, Asia and South America (Grand and Szulkin 2002) face far greater obstacles, showcasing the negative impact of xenophobia. Similar trends have been observed in the United Kingdom (Anderson et al. 2006), where employers confessed to ethnic stereotyping in their job candidate preferences. Female job-seekers (Jureidini 2005; Dlamini et al. 2012) may experience further obstacles, necessitating an intersectional analysis.
Finally, efforts to assimilate successfully play key role in modulating xenophobic employment discrimination (Kee 1994) and mitigating the resulting pay gap (Nielsen et al. 2004), though out-groups may instead experience further alienation due to xenophobic attitudes (Mayadas and Elliott 1992). Language barriers, cultural differences, lack of networks, and social capital (Behtoui and Neergaard 2010) present challenges to securing employment (Hakak et al. 2010). This lack of integration also affects foreign entrepreneurs, facing obstacles in securing capital for their businesses (Teixeira et al. 2007). A lack of elasticity contributes to the wage gap between the immigrant and native job seekers, and is reflected in monopsonistic discrimination against immigrants. Findings from a study in Germany suggest not only that the employers hold more power over immigrant workers as a result of search friction and lower flexibility, but also imply that employers profit from discriminating against immigrants (Hirsch and Jahn 2015).
Existing legal frameworks are often insufficient in protecting the rights of foreign employees and addressing the problem of xenophobia in the workplace (Handayani et al. 2021; Mubangizi 2021). This insufficiency underscores the need for a holistic approach to ensuring fair employment outcomes. Technology should aim to complement and assist numerous organisational initiatives, collective action, legislative and legal measures, administrative measures, political and educational action and international standards (Taran et al. 2004).
3.4.1 Risks
Predicting prospective employee performance from retrospective data (Mahmoud et al. 2019) carries the risk of perpetuating historical discrimination against out-groups (Kim 2018; Mujtaba et al. 2019), under the veneer of alleged objectivity. Such issues have already been identified in a number of deployed AI systems (Dastin 2018; Hsu 2020). Discriminatory outcomes may arise even in absence of direct information on ethnic and national background, via proxy features (Prince Anya and Schwarcz 2019). Additionally, there is a pressing need for establishing firm guidelines and policies governing the fair usage of employment decision support tech (Kim and Bodie 2021). Xenophobic discrimination poses unique challenges in this context, as natives and immigrant workers are often not granted the same rights and protections. This is in part due to the specific requirements for work permits in terms of skills and training, corporate sponsorship, minimum compensation etc. These requirements may differ between immigrant workers of different national and ethnic backgrounds. Polices aimed against xenophobic discrimination in employment, and the fairness on AI systems in employment, need to account for such legal distinctions. The overall impact of AI on the job market spans beyond its immediate utilisation in employment decisions, and should be seen in relation to how technology is reshaping the future of work—including job quality—across industries and regions.
3.4.2 Promise
Assistive technologies have the potential to improve the overall candidate experience as well as help overcome human biases and help improve diversity and inclusion (Daugherty et al. 2019). Preliminary studies suggest that the use of AI has in some cases already greatly increased the diversity of top selected job applicants (Avery et al. 2023). ML models may also be seen as a powerful lens into existing hiring practices (van den et al. 2020), helping audit (Liem et al. 2018) and improve such processes. Recently developed frameworks for ML explainability (Bhatt et al. 2020) and fairness (Cabrera et al. 2019) may be utilised towards this end. Simulations may also prove helpful for policy and process design (Hu and Chen 2017; Bower et al. 2017; Hu and Chen 2018; Cohen et al. 2019; Schumann et al. 2020). Based on the prevalence of xenophobic discrimination in historical hiring practice, as well as language technologies that underpin employment AI systems, we believe that there is a pressing need for improved transparency and explicit checks for fairness in decision recommendations for candidates of different national and ethnic background, as well as immigration status. If employment AI tech is ever to deliver on its promise, it needs to demonstrate a commitment to fight deeply ingrained manifestations of xenophobia in the workplace.
3.5 Stereotypes in large pretrained models
Recent trends in AI development involve systems that are increasingly general in the scope of their application. Large language models (LLM) (Li et al. 2021; Chowdhery et al. 2022), for example, can be fine-tuned (Wei et al. 2021) and adapted to a range of applications and use cases, becoming components of even larger and more complex AI systems. These systems are sometimes referred to as ”foundation models” (Bommasani et al. 2021), or ”base models”, in recognition of their emerging role as building blocks (Zeng et al. 2022) in such systems. The development of such models usually involves large-scale pre-training on increasingly large datasets which are increasingly hard to curate. This poses unique safety and ethics concerns, due to a difficulty of anticipating the totality of harmful biases and stereotypes, as well as the variety of use cases stemming from their wide applicability (Ganguli et al. 2022). Large language models may exhibit varying degrees of positive or negative sentiment across national (Rae et al. 2021; Venkit et al. 2023), societal (Khandelwal et al. 2023) and religious groups (Abid et al. 2021b; Abid et al. 2021), inheriting features of historical and contemporary discourse. Text-to-image models have recently been shown to amplify demographic stereotypes at scale (Bianchi et al. 2022; AI Image Stereotypes 2023). Multimodal systems (Jaegle et al. 2021) introduce another layer of complexity, with cross-modal stereotypes (Birhane et al. 2021) becoming detectable when considering modalities in relation to each other. Any biases in base models have the potential to emerge across their downstream applications, making it especially important to evaluate such models and their underlying training data prior to a wider release.
As an illustrative example of the types of xenophobic bias that may creep into large pretrained models, here we consider the case of media and arts. Indeed, narratives shaping the very formation of national and group identities, the us and them, are deeply embedded in media as societal cultural artifacts, either altering (Juan-José and Frutos Francisco 2017) and subverting or reinforcing and perpetuating pre-existing power dynamics within and across societies. Stereotyping and projection are central to the act of othering, as the other get associated with unwanted, negative traits. Undertaking a critical re-examination of how xenophobia is expressed and disseminated in media and arts is critical in understanding the risks of inadvertently incorporating xenophobic stereotypes in AI systems developed on broad multimedia data at scale. Recent advances in AI creativity and art co-creation (Ramesh et al. 2021; Saharia et al. 2022; Li et al. 2022) raise interesting questions around the role of AI-generated art and the cultural stereotypes potentially contained within (Jalal et al. 2021a; Cho et al. 2022).
In case of literature, xenophobia manifests not only in terms of harmful narratives leading to discrimination (Taras 2009) and violence (Minga 2015), but also via ethnolinguistic purity, centering only the works written by the favoured in-group as “true” national cultural heritage, whilst erasing contributions written in minority languages (Kamusella et al. 2021). Xenophobia may also manifest in the lack of availability and literary translation of foreign works (Dickens 2002), thereby sheltering the local xenophobic narratives from foreign critique. Portrayals of out-groups in film provide ample evidence for deeply rooted xenophobia, involving racial stereotyping (Yuen 2016), unfavourable portrayals of Arabs (Shaheen 2003), antisocial portrayal of LatinX people (Berg 2002), stereotyping queer people as villains (Long 2021; Brown 2021), etc. These works reinforce notions of superiority for the majority, while simultaneously promoting unbefitting images and traits with respect to out-groups (Inayat 2017; Benshoff and Griffin 2021). The pervasiveness of xenophobic stereotypes in cultural artifacts and their role in nationalistic narratives (Smith 2002; Joep 2006; Gordy 2010; Khandy 2021) make it imperative to audit large AI models and datasets on those grounds.
3.5.1 Risks
Recent work on enumerating risks in large language models (Weidinger et al. 2021, 2022) categorized the potential downstream harms across six types: 1) discrimination, exclusion and toxicity; 2) information hazards; 3) misinformation harms; 4) malicious uses; 5) human-computer interaction harms; and 6) automation, access and environmental harms. These categories intersect with material, representational and wider societal harms through which we’ve been discussing the potential impact of xenophobia in sociotechnical systems, adding another layer of analysis. LLMs may potentially surface direct xenophobic discrimination, they may pose information hazard by revealing sensitive or private information about vulnerable individuals, spread misinformation that disparages out-groups, be incorporated in technologies weaponized by malicious xenophobic users, subtly reinforce discriminatory stereotypes via repeated human-computer interaction, and indirectly lead to disparate socioeconomic and environmental impact. Prior studies have identified a concerning aptitude of such systems towards accurately emulating extremist views when prompted (McGuffie and Newhouse 2020), and numerous ethnic biases (Li et al. 2020). Perhaps unsurprisingly, these problems were shown to arise in multi-modal systems as well (Srinivasan and Bisk 2021; Cho et al. 2022; Birhane et al. 2021). A recent study identified numerous stereotypes in visual representations from text-to-image models, across 135 nationality-based identity groups (Jha et al. 2024). Disparate performance in attribute representation for different nationalities was highlighted as an issue, especially for the Global South. As this type of content may potentially influence and radicalise people into far-right ideologies at an unprecedented scale and pace, more focused investments in data curation are needed to address underlying risks (Bender et al. 2021).
3.5.2 Promise
AI systems need to be designed to transcend ethno-nationalistic exclusionary cultural narratives and challenge entrenched xenophobic views. This would involve developing better ways of incorporating domain knowledge on the cross-cultural underpinnings of social friction, and breaking away from a stochastic re-enactment of historical patterns. This stands in stark contrast to the historical AI approach, centered around supervised learning and reconstructing pre-existing data. There is a pressing need to imbue general AI systems with capacities for understanding and reasoning about the world, and therefore the ability to integrate and reinterpret information, deriving less-biased and better reasoned conclusions about social phenomena. It is yet to be determined whether such capabilities lie fully within the reach of current technology, or whether they require further innovation on the path towards more generally intelligent AI systems. While there is ongoing research work on improving reasoning in large language models (Huang and Chen-Chuan 2022; Wei et al. 2022; Zhao et al. 2022; Wang et al. 2023; Long 2023; Weng et al. 2023; Lu et al. 2023; Taylor 2023; Zhang et al. 2023; Besta et al. 2023), there is still a degree of skepticism on these nascent capabilities (Xu et al. 2023; Huang et al. 2023) and additional challenges that remain to be overcome (Ji et al. 2023; Shi et al. 2023). Importantly, a culture gap has been identified in the understanding capabilities of existing models (Liu et al. 2023).
Recent advances in understanding and mitigating social biases in language models (Zmigrod et al. 2019; Huang et al. 2019b; Li et al. 2020; Liang et al. 2021; Ousidhoum et al. 2021; Nozza et al. 2021), and computer vision systems (Joo et al. 2020; Wang et al. 2020a), coupled with a development of benchmarks (Nangia et al. 2020; Nadeem et al. 2020) for identification of malignant stereotypes in AI models, may pave the way towards quantifying the extent of xenophobic bias and harm in large pre-trained AI systems. Yet, for contemporary base models, the solution likely needs to be multi-faceted and case-specific.
Multi-lingual (Huang et al. 2019; Xu et al. 2021; Srinivasan et al. 2021) models may help incorporate a wider variety of viewpoints, especially if grounded in geospatially diverse imagery aimed at improving representation—though there are still open challenges when it comes to ensuring equal performance (Wang et al. 2021a; Monojit and Amit 2021; Ramesh et al. 2023). Self-supervised learning in computer vision has shown promise in terms of reducing the amount of bias (Goyal et al. 2022), as well as presenting new ways to incorporate fairness objectives into model training (Tsai et al. 2021). Data curation (Scao et al. 2022) plays a central role in mitigating the effects of historical bias. Mechanisms for incorporating comparative and natural language human feedback to refine and improve model outputs (Ouyang et al. 2022; Scheurer et al. 2022) present another useful avenue, as do efforts for soliciting human preferences on conversational rules, and highlighting sources of information to improve factuality of claims (Glaese et al. 2022). This myriad of technical mitigations needs to be incorporated in a more holistic participatory approach, to avoid pitfalls of technosolutionism and empower marginalised communities in AI system design.
4 Towards xenophilic AI systems
4.1 A moral imperative
Crucially, in terms of scope and impact, many of the AI tools and services discussed in this paper have global reach (Kate 2021). They easily traverse national boundaries and—in the case of social media platforms—involve billions of users worldwide. Cumulatively, these technologies therefore have a powerful shaping effect on the contours of social relationships at a global level. Considered in this light, it is worth pausing to more fully assess the combined effect of the practices outlined in this paper.
Taken together, xenophobic discrimination in the domain of AI can be best understood as a compound or structural phenomenon and set of experiences: people are not only exposed to automated xenophobic bias via interaction with one or two services, but across the entire range of digital services they interact with. Crucially, in an age of growing displacement and movement of people, a person who is new to a country, a non-native speaker, or a carrier of identity traits that attract xenophobic sentiment may expect to receive, among other things: (1) higher rates of harassment on social media, (2) problematic treatment by immigration services, (3) increased chance of medical error, (4) prejudicial access to employment opportunities, and (5) algorithmically embedded stereotypes, in addition to whatever other forms of disadvantage they may encounter. They are therefore in a position of intersectional vulnerability (Crenshaw 2017): there is added precarity that comes with the notion of being “foreign” in a digital age. This is a uniquely dangerous position to occupy and one that we believe warrants special protection when it comes to the design and deployment of algorithmic systems.
Given this wider context, technologists face an important choice when it comes to xenophobia and AI. They can build systems that heighten tensions between different groups and compound existing axes of disadvantage or they can promote a different, more inclusive, ideal—helping to mitigate the impact of xenophobia by learning from those affected and by taking concrete measures to forestall these effects. This aspiration, to build better systems that are capable of promoting civic inclusion and deescalating othering dynamics, we term “xenophilic technology”. When successful, xenophilic technology helps to build relationships between people of different backgrounds and to enable the rich and productive sharing of cultural differences.
Ultimately, the risks outlined in this paper indicate that the problem of xenophobic bias in AI systems cannot go unaddressed. Instead, we believe that technologists should partner with domain experts to jointly take the lead in this area, drawing upon sources such as the Universal Declaration on Human Rights (UDHR) which is quite explicit in its opposition to discrimination on the basis of national origin or related grounds (Prabhakaran et al. 2022). On this point, Article 2 of the UDHR states that “everyone is entitled to all the rights and freedoms set forth in this Declaration without distinction of any kind, such as race, colour, sex, language, religion, political or other opinion, national or social origin, property, birth or other status.” Additionally, the global reach of technology also brings with it significant promise. By addressing the potential for material harm, representational harm, and risk to people’s rights, responsibly designed technology may help bridge the divide between communities, minimize material and representational harms to foreigners and non-citizens, ensure that these people are secure in their standing as members of the relevant communities, and help to ensure that human rights are respected.
4.2 Measurement and mitigation
Evaluating and mitigating the potential xenophobic impact of AI systems is not common practice. In this section we highlight the technical considerations in relevant areas of ML fairness research that may prove useful for addressing this unmet need and help us move away from more hegemonic ML fairness approaches (Weinberg 2022).
4.2.1 Measuring xenophobia
Most approaches to measuring fairness rely on an understanding of marginalised identities, coupled with the availability of the corresponding information in the underlying data. This is why it’s especially concerning that there are deep issues with the overall quality of race and ethnicity data across observational databases (Polubriaginof et al. 2019), which often contain insufficiently fine-grained and static racial and ethnic categories (Strmic-Pawl et al. 2018; Hanna et al. 2020), involve discrepancies between recorded and self-reported identities (Moscou et al. 2003) and come with data collection (Varcoe et al. 2009; Ford and Kelly 2005; Routen et al. 2022; Andrus and Villeneuve 2022), privacy (Ringelheim 2011) and governance challenges. Despite recent integration efforts (Müller-Crepon et al. 2021), better data (Pushkarna et al. 2022) and model (Mitchell et al. 2019; Crisan et al. 2022) documentation are required to promote the understanding of ethnic representation and data provenance. Name-ethnicity classifiers could offer a potential solution, but they tend to themselves rely on machine learning, resulting in disparate performance across ethnicities (Hafner et al. 2023).
Fairness approaches rooted in comparing model performance and impact across groups need to be more cognizant of the normative challenges that arise when defining group identities (Leben 2020), and there also needs to be a deeper engagement with domain experts to establish consistent and meaningful ways of identifying in-groups and out-groups within the context of each AI application. The lack of frameworks and benchmarks for assessing xenophobic impact makes it especially hard to evaluate the impact of systems with a potentially global scope, which is a definitive concern. In the interim, practitioners should be encouraged to include ethnicity and nationality in routine ML fairness evaluation, in intersection with other relevant sensitive attributes (Wang et al. 2022), and drive technical innovation towards the development of methods that can transcend the confines of pre-defined categorizations (Jalal et al. 2021).
Counterfactual (Kusner et al. 2017; Pfohl et al. 2019; Joo et al. 2020) and contrastive (Chakraborti et al. 2020) methodologies may prove useful for more concretely identifying the pathways of discrimination, while leveraging expert knowledge. Most such methods would still require a meaningful categorization of group identities, and may not be applicable in certain contexts where certain social categories may not lend themselves to meaningful counterfactual manipulation, making it impossible to reliably assess the truthfulness or counterfactuals (Kasirzadeh and Smart 2021). Individual fairness (Dwork et al. 2012; Dwork and Ilvento 2018; Sharifi-Malvajerdi et al. 2019; Gupta and Kamble 2021) may prove applicable in absence of reliable categorizations, and there is ongoing research on eliciting (Jung et al. 2019; Yahav et al. 2020) or learning (Ilvento 2019; Mukherjee et al. 2020) similarity measures for its practical application.
Quantitative measures of impact should be complemented by qualitative assessments (Van der Veer et al. 2013; Krumpal 2012; Olonisakin and Adebayo 2021), and comprehensive participatory approaches towards deeper and more representative assessments of xenophobic impact. Wider community participation (Martin Jr et al. 2020; Bondi et al. 2021; Birhane et al. 2022) is key when creating benchmarks, soliciting feedback, and empowering the affected communities to inform policy and AI system design. As an ideological and social phenomenon, xenophobia may be targeted at an ever-evolving conception of what is foreign, according to local histories, cultural and political context (Mamdani 2018, 2012). Understanding who is deemed foreign, by whom, on what grounds, and in light of which set of unfolding circumstances, requires knowledge that is local, situated and contemporaneous.
Evaluation of xenophobic bias should also help re-center the question of discrimination onto the perpetrators. AI systems may be used with malicious intent, towards spreading hate and amplifying discrimination against vulnerable populations. Human bias may also skew the outcomes of assistive AI tech with human in the loop. To address the root cause of xenophobia, it is important to understand the role of AI systems in in-group radicalization, as well as how AI systems shift power. In terms of better understanding outcomes, we suggest for practitioners to consider, in line with what we propose in the paper, material harms, recognition harms and rights harms. This is merely a step towards a more holistic approach, as there is a need to also engage with the deeper questions of social justice (Birhane et al. 2022b; Schwöbel and Remmers 2022).
4.2.2 Mitigation strategies
The complexity and pervasiveness of xenophobia in society and consequently in the data used to train AI systems calls for the development of both technical and non-technical mitigation strategies for preventing the adverse impact of such systems on the most vulnerable and marginalised groups in society, citizens and non-citizens alike. These are currently lacking, in part due to a systemic failure to recognize the importance of xenophobic harms, and in part due to already discussed deficiencies in recording and utilising the relevant sensitive information, compounded by insufficient investments in participatory approaches.
Until such comprehensive frameworks for identifying and quantifying xenophobic harms are fully developed, practitioners may want to consider alternative approaches, tailored for systems with missing or incomplete data, that are capable of providing partial safety and robustness guarantees without a full categorical specification. This may involve considering plausible groupings based on the available inputs, and improving worst-case or average-case performance across such plausible group structures (Kearns et al. 2018; Kim et al. 2018; Hashimoto et al. 2018). The use of proxy targets (Gupta et al. 2018; Grari et al. 2021) is an alternative, though it comes with risks of stereotyping. Practitioners need to ensure robust evaluation (Mandal et al. 2020) while accounting for uncertainty and incompleteness (Awasthi et al. 2021) in the data. Purely technical approaches rarely map onto socially acceptable mitigation outcomes (Słowik and Bottou 2021), making it imperative to avoid technosolutionism and to embrace interdisciplinary efforts.
Transparency and model audits (Raji and Buolamwini 2019; Raji et al. 2020) need to complement such fairness assessments, especially given the difficulties involved with defining comprehensive quantitative measures that would consistently capture all xenophobic harms. Improvements in AI explainability would allow us to more thoroughly examine how AI systems reach their decisions, and check if these decisions are reached in ways that imply xenophobic bias. This is especially relevant given that large-scale deep learning models that have come to permeate the field and practical applications have had a reputation for being black boxes, significantly harder to understand and interpret compared to simpler statistical models and expert systems rooted in human-defined heuristics. Yet, recent advances in AI explainability research (Gilpin et al. 2018; Bhatt et al. 2020) make us optimistic that such analyses may yield valuable insights and help with identification of discrimination and xenophobic harms. For example, numerous explainability approaches are being developed for improved understanding of large language models (Zhao et al. 2023), both in terms of the mechanistic computations in the transformer architecture (Elhage et al. 2021; Bricken et al. 2023), as well as structured inference-time approaches where models themselves provide verbose step-by-step explanations (Wei et al. 2022; Zhao et al. 2022; Wang et al. 2023) and the ability to self-critique (Valmeekam et al. 2023) in the process of formulating their reply and making a recommendation. Detailed process supervision improves reliability and the reasoning capability of such models (Lightman et al. 2023), and compositional human explanations can assist with mitigating bias (Yao et al. 2021). Notable advances have also been made in improving explainability of computer vision systems (Chefer et al. 2021), multimodal AI systems (Joshi et al. 2021), and concept discovery in super-human narrow AI systems (McGrath et al. 2022; Schut et al. 2023). All these advances showcase the promise of AI explainability and its potential usefulness for conducting audits (Abdul et al. 2018; Hall et al. 2019; Bibal et al. 2021) and enabling rapid identification of model harms (Sokol and Flach 2019; Begley et al. 2020; Verma et al. 2020). Explainability methods may also be helpful in identifying vulnerable groups (Strümke and Slavkovik 2022) within each AI application context, shortcutting through sensitive attributes in AI decision making (DeGrave et al. 2021; Brown et al. 2023), and provide a fine-grained breakdown of model performance (Sharma et al. 2020). Yet, challenges remain, in ensuring that such model explanations are grounded and faithful (Turpin et al. 2023; Lyu et al. 2023). These efforts need to be coupled with the appropriate escalation procedures and enable recourse—which may require a greater focus on designing appropriate interventions (Karimi et al. 2021). These additional escalation pathways are important, as technologists may not be able to capture all of the model harms through techniques outlined above, given that there is always some degree of approximation involved with purely technical solutions, and they need to be complemented by appropriate qualitative assessments.
5 Conclusions
Despite being one of the key drivers of discrimination and conflict worldwide, xenophobic bias is yet to be formally recognised and comprehensively assessed in AI system development. This blind spot introduces a considerable risk of amplifying harms to marginalised communities and fuelling fear and intolerance towards foreigners, which may lead to violence and conflict both at the societal level and more widely.
Our overview of several prominent technological use cases—social media, immigration, healthcare, employment and the advances in the development of foundation models—reveals how historical and contemporary xenophobic attitudes may manifest in data and AI systems. Moreover, we illustrate the effect of these tendencies by drawing upon three categories of potential xenophobic harms: those that cause direct material disadvantage to people perceived as foreign, representational harms that undermine their social standing or status, and the wider societal harms which include barriers to the successful exercise of civic and human rights. Significantly, given the scope and depth of AI services, people who experience xenophobic bias in one domain of automation can also expect to experience unequal treatment across a range of other domains, further compounding their situation.
In this context, we suggest that the development of xenophilic technology is crucial for the ethical design and deployment of AI systems, understood as systems that treat people equitable at the societal level. AI should help promote civic inclusion, oppose the malignant “othering” of marginalised groups, and be centered around the potential to cultivate the kind of rich and productive cross-cultural discourse that is appropriate for a world like our own. Coordinated interdisciplinary action, participatory frameworks, and an unwavering commitment of technologists are required to fashion such a future, along with technical innovation in addressing the existing technological bases of xenophobia in ML systems. Xenophilic technology should be understood as merely a part of a much wider set of solutions, within and across societies, avoiding the pitfalls of technosolutionism.
Data availability
Not applicable.
References
Abdul A, Vermeulen J, Wang D, Lim BY, Kankanhalli M (2018) Trends and trajectories for explainable, accountable and intelligible systems: an hci research agenda. In: Proceedings of the 2018 CHI conference on human factors in computing systems, pp 1–18
Abid A, Farooqi M, Zou J (2021a) Large language models associate Muslims with violence. Nat Mach Intell 3(6):461–463
Abid A, Farooqi M, Zou J (2021b) Persistent anti-muslim bias in large language models. In: Proceedings of the 2021 AAAI/ACM conference on AI, ethics, and society, pp 298–306
Abma TA, Broerse JEW (2010) Patient participation as dialogue: setting research agendas. Health Expect 13(2):160–173
Adja KYC, Wu E, Golinelli D, Lenzi J, Fantini MP (2020) What about social determinants of health against xenophobia? Public Health 187(2020):187
Aharoni R, Johnson M, Firat O (2019) Massively multilingual neural machine translation. arXiv preprint arXiv:1903.00089
Ahmad N (2020) Refugees and algorithmic humanitarianism: applying artificial intelligence to RSD procedures and immigration decisions and making global human rights obligations relevant to AI governance. Int J Minor Group Rights 28(3):367–435. https://doi.org/10.1163/15718115-bja10007
AI Image Stereotypes (2023) https://restofworld.org/2023/ai-image-stereotypes/
Aitchison G, Meckled-Garcia S (2021) Against online public shaming: ethical problems with mass social media. Social Theory Pract 47(1):1–31
Aizenberg E, Dennis MJ, van den Hoven J (2023) Examining the assumptions of AI hiring assessments and their impact on job seekers’ autonomy over self-representation. AI Soc 2023:1–9
Akinola AO, Klimowich (2018) The political economy of xenophobia in Africa. Springer, Berlin
Alatawi HS, Alhothali AM, Moria KM (2021) Detecting white supremacist hate speech using domain specific word embedding with deep learning and BERT. IEEE Access 9(2021):106363–106374
Albright J (2017) Welcome to the era of fake news. Media Commun 5(2):87–89
Alfano M, Carter JA, Cheong M (2018) Technological seduction and self-radicalization. J Am Philos Assoc 4(3):298–322
Aluru SS, Mathew B, Saha P, Mukherjee A (2020) Deep learning models for multilingual hate speech detection. arXiv preprint arXiv:2004.06465
Anderson B (2020) Imagined communities: Reflections on the origin and spread of nationalism. In: The new social theory reader. Routledge, pp 282–288
Anderson B, Ruhs M, Rogaly B, Spencer S (2006) Fair enough? Central and East European migrants in low-wage employment in the UK, Joseph Rowntree Foundation
Andreassen R (2021) Social media surveillance, LGBTQ refugees and asylum: how migration authorities use social media profiles to determine refugees as “genuine” or “fraudulent”. First Monday
Andrus M, Villeneuve S (2022) Demographic-reliant algorithmic fairness: characterizing the risks of demographic data collection in the pursuit of fairness. arXiv preprint arXiv:2205.01038
Angel S, Frederiksen KN (2015) Challenges in achieving patient participation: a review of how patient participation is addressed in empirical studies. Int J Nurs Stud 52(9):1525–1538
Angwin J, Larson J (2022) Bias in criminal risk scores is mathematically inevitable, researchers say. In:Ethics of data and analytics. Auerbach Publications, pp 265–267
Angwin J, Larson J, Mattu S, Kirchner L (2022) Machine bias. In: Ethics of data and analytics. Auerbach Publications, pp 254–264
Arredondo P (2018) Latinx immigrants: transcending acculturation and xenophobia. Springer, Berlin
Avery M, Leibbrandt A, Vecci J (2023) Does artificial intelligence help or hurt gender diversity? In: Evidence from two field experiments on recruitment in tech (February 14, 2023)
Awal MR, Cao R, Lee RK-W, Mitrovic S (2021) AngryBERT: joint learning target and emotion for hate speech detection. arXiv:2103.11800 [cs.CL]
Awasthi P, Beutel A, Kleindessner M, Morgenstern J, Wang X (2021) Evaluating fairness of machine learning models under uncertain and incomplete information. In: Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pp 206–214
Azizi SS, Yektansani K (2020) Artificial intelligence and predicting illegal immigration to the USA. Int Migr 58(5):183–193
Badjatiya P, Gupta S, Gupta M, Varma V (2017) Deep learning for hate speech detection in tweets. In: Proceedings of the 26th international conference on World Wide Web companion, pp 759–760
Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473
Baldauf J, Ebner J, Guhl J (2019) Hate speech and radicalisation online: the OCCI research report. Institute for Strategic Dialogue
Baltruschat IM, Nickisch H, Grass M, Knopp T, Saalbach A (2019) Comparison of deep learning approaches for multi-label chest X-ray classification. Sci Rep 9(1):1–10
Banton M (1996) The cultural determinants of xenophobia. Anthropol Today 12(2):8–12. 0268540X, 14678322. http://www.jstor.org/stable/2783294
Barocas S, Selbst AD (2016) Big data’s disparate impact. Calif Law Rev 2016:671–732
Bechavod Y, Jung C, Wu SZ (2020) Metric-free individual fairness in online learning. Adv Neural Inf Process Syst 33(2020):11214–11225
Begley T, Schwedes T, Frye C, Feige I (2020) Explainability for fair machine learning. arXiv preprint arXiv:2010.07389
Behtoui A, Neergaard A (2010) Social capital and wage disadvantages among immigrant workers. Work Employ Soc 24(4):761–779
Bell MP, Kwesiga EN, Berry DP (2010) Immigrants: the new “invisible men and women” in diversity research. J Manag Psychol
Bender EM, Gebru T, McMillan-Major A, Shmitchell S (2021) On the dangers of stochastic parrots: can language models be too big?. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pp 610–623
Benesch S, Leader MJ (2015) Dangerous speech and dangerous ideology: an integrated model for monitoring and prevention. Genocide Stud Prev 9:70–95. https://doi.org/10.5038/1911-9933.9.3.1317
Benhabib S (2008) Another cosmopolitanism. Oxford University Press, Oxford
Benjamin R (2019) Race after technology: abolitionist tools for the new jim code. Wiley, Oxford
Benshoff HM, Griffin S (2021) America on film: Representing race, class, gender, and sexuality at the movies. Wiley, Oxford
Berg CR (2002) Latino images in film: stereotypes, subversion, and resistance. University of Texas Press, Austin
Bernasconi R (2014) Where is xenophobia in the fight against racism? Crit Philos Race 2:5–19. https://doi.org/10.5325/critphilrace.2.1.0005
Besta M, Blach N, Kubicek A, Gerstenberger R, Gianinazzi L, Gajda J, Lehmann T, Podstawski M, Niewiadomski H, Nyczyk P et al (2023) Graph of thoughts: Solving elaborate problems with large language models. arXiv preprint arXiv:2308.09687
Bhagat A (2018) Forced (Queer) migration and everyday violence: the geographies of life, death, and access in Cape Town. Geoforum 89(155–163):0016–7185. https://doi.org/10.1016/j.geoforum.2017.07.028
Bhatt U, Xiang A, Sharma S, Weller A, Taly A, Jia Y, Ghosh J, Puri R, Moura JMF, Eckersley Peter (2020) Explainable machine learning in deployment. In: Proceedings of the 2020 conference on fairness, accountability, and transparency, pp 648–657
Bianchi F, Kalluri P, Durmus E, Ladhak F, Cheng M, Nozza D, Hashimoto T, Jurafsky D, Zou J, Caliskan A (2022) Easily accessible text-to-image generation amplifies demographic stereotypes at large scale. https://doi.org/10.48550/ARXIV.2211.03759
Bibal A, Lognoul M, De Streel A, Frénay B (2021) Legal requirements on explainability in machine learning. Artif Intell Law 29(2):149–169
Birhane A (2022) The unseen Black faces of AI algorithms
Birhane A, Prabhu VU, Kahembwe E (2021) Multimodal datasets: misogyny, pornography, and malignant stereotypes. https://doi.org/10.48550/ARXIV.2110.01963
Birhane A, Isaac W, Prabhakaran V, Diaz M, Madeleine CE, Gabriel I, Mohamed S (2022a) Power to the people? Opportunities and challenges for participatory AI. In: Equity and access in algorithms, mechanisms, and optimization. ACM. https://doi.org/10.1145/3551624.3555290
Birhane A, Ruane E, Laurent T, Brown MS, Flowers J, Ventresque A, Dancy CL (2022b) The forgotten margins of AI ethics. arXiv:2205.04221
Bommasani R, Hudson DA, Adeli E, Altman R, Arora S, von Arx S, Bernstein MS, Bohg J, Bosselut A, Brunskill E et al (2021) On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258
Bondi E, Xu L, Acosta-Navas D, Killian JA (2021) Envisioning communities: a participatory approach towards AI for social good. In: Proceedings of the 2021 AAAI/ACM conference on AI, ethics, and society. ACM. https://doi.org/10.1145/3461702.3462612
Bower A, Kitchen SN, Niss L, Strauss MJ, Vargas A, Venkatasubramanian S (2017) Fair pipelines. arXiv preprint arXiv:1707.00391
Bowman G (2021) Xenophobia, fantasy and the nation: the logic of ethnic violence in former Yugoslavia. In: The anthropology of Europe. Routledge, pp 143–171
Bricken T, Templeton A, Batson J, Chen B, Jermyn A, Conerly T, Turner N, Anil C, Denison C, Askell A et al (2023) Towards monosemanticity: decomposing language models with dictionary learning. Transform Circuits Thread 2023:2
Brown A (2021) Hook, Ursula, and Elsa: Disney and Queer-coding from the 1950s to the 2010s. Johns Hopkins Univ 2(1):27887
Brown A, Tomasev N, Freyberg J, Liu Y, Karthikesalingam A, Schrouff J (2023) Detecting shortcut learning for fair medical AI using shortcut testing. Nat Commun 14(1):4314
Brubaker R, Laitin DD (1998) Ethnic and nationalist violence. Annu Rev Sociol 24(1):423–452
Bruns A (2019) Are filter bubbles real? Wiley, Oxford
Buolamwini J, Gebru T (2018) Gender shades: Intersectional accuracy disparities in commercial gender classification. In: Conference on fairness, accountability and transparency. PMLR, pp 77–91
Bursztyn L, Egorov G, Enikolopov R, Petrova M (2019) Social media and xenophobia: evidence from Russia. Technical Report, National Bureau of Economic Research
Cabrera ÁA, Epperson W, Hohman F, Kahng M, Morgenstern J, Chau DH (2019) FairVis: Visual analytics for discovering intersectional bias in machine learning. In: 2019 IEEE conference on visual analytics science and technology (VAST). IEEE, pp 46–56
Caney S (2005) Justice beyond borders: a global political theory. OUP Oxford
Cao R, Lee RK-W, Hoang T-A (2020) DeepHate: hate speech detection via multi-faceted text representations. In: 12th ACM conference on web science (Southampton, United Kingdom) (WebSci ’20). Association for Computing Machinery, New York, NY, USA, pp 11–20. https://doi.org/10.1145/3394231.3397890
Carlos-Roca LR, Torres IH, Tena CF (2018) Facial recognition application for border control. In: 2018 International joint conference on neural networks (IJCNN), pp 1–7. https://doi.org/10.1109/IJCNN.2018.8489113
Carlsson M, Rooth DO (2007) Evidence of ethnic discrimination in the Swedish labor market using experimental data. Labour Econ 14(4):716–729
Chakraborti T, Patra A, Noble JA (2020) Contrastive fairness in machine learning. IEEE Lett Comput Soc 3(2):38–41
Chan KYG (2022) AI employment decision-making: integrating the equal opportunity merit principle and explainable AI. AI Soc 2022:1–12
Chatterjee I (2021) Alt-right movement: dissecting racism, patriarchy and anti-immigrant xenophobia. Sage. https://books.google.co.uk/books?id=Uzd0zgEACAAJ
Chefer H, Gur S, Wolf L (2021) Transformer interpretability beyond attention visualization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 782–791
Chen MX, Firat O, Bapna A, Johnson M, Macherey W, Foster G, Jones L, Parmar N, Schuster M, Chen Z et al (2018) The best of both worlds: combining recent advances in neural machine translation. arXiv preprint arXiv:1804.09849
Chenzi V (2021) Fake news, social media and xenophobia in South Africa. Afr Ident 19(4):502–521
Choudhury M, Deshpande A (2021) How linguistically fair are multilingual pre-trained language models. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 12710–12718
Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, Barham P, Chung HW, Sutton C, Gehrmann S et al (2022) Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311
Cho J, Zala A, Bansal M (2022) DALL-Eval: probing the reasoning skills and social biases of text-to-image generative transformers. https://doi.org/10.48550/ARXIV.2202.04053. arXiv preprint arXiv:2202.04053
Chun A (2007) Using AI for e-government automatic assessment of immigration application forms. In: Proceedings of the national conference on artificial intelligence, vol 2, pp 1684–1691
Cohen L, Lipton ZC, Mansour Y (2019) Efficient candidate screening under multiple tests and implications for fairness. arXiv:1905.11361
Coley RY, Eric J, Simon EG, Maricela C, Shortreed MS (2021) Racial/ethnic disparities in the performance of prediction models for death by suicide after mental health visits. JAMA Psychiatry 78(7):726–734
Crawford K (2021) The atlas of AI: power, politics, and the planetary costs of artificial intelligence. Yale University Press, Yale
Crenshaw KW (2017) On intersectionality: essential writings. The New Press, New York
Crisan A, Drouhard M, Vig J, Rajani N (2022) Interactive model cards: a human-centered approach to model documentation. arXiv preprint arXiv:2205.02894
Crush J, Ramachandran S (2010) Xenophobia, international migration and development. J Hum Dev Capab 11(2):209–228. https://doi.org/10.1080/19452821003677327
Crush J, Tawodzera G (2014) Medical xenophobia and Zimbabwean migrant access to public health services in South Africa. J Ethnic Migr Stud 40(4):655–670
Daniels J (2018) The algorithmic rise of the “alt-right’’. Contexts 17(1):60–65
Dastin J (2018) Amazon scraps secret AI recruiting tool that showed bias against women. In: Ethics of data and analytics. Auerbach Publications, pp 296–299
Daugherty PR, Wilson HJ, Chowdhury R (2019) Using artificial intelligence to promote diversity. MIT Sloan Manag Rev 60(2):1
De Fauw J, Ledsam RJ, Romera-Paredes B, Nikolov S, Tomasev N, Blackwell S, Askham H, Glorot X, O’Donoghue B, Visentin D et al (2018) Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med 24(9):1342–1350
DeGrave AJ, Janizek Joseph D, Su-In L (2021) AI for radiographic COVID-19 detection selects shortcuts over signal. Nat Mach Intell 3(7):610–619
Demartini G, Mizzaro S, Spina D (2020) Human-in-the-loop artificial intelligence for fighting online misinformation: challenges and opportunities. Bull IEEE Comput Soc Tech Committ Data Eng 43(3):65–74
Dembrower K, Crippa A, Colón E, Eklund M, Strand F (2023) Artificial intelligence for breast cancer detection in screening mammography in Sweden: a prospective, population-based, paired-reader, non-inferiority study. Lancet Digit Health 5(10):e703–e711
Dickens E (2002) Literary translation in Britain and selective xenophobia. CLCWeb Compar Liter Culture 4(1):1
Dietz J, Joshi C, Esses VM, Hamilton LK, Gabarrot F (2015) The skill paradox: explaining and reducing employment discrimination against skilled immigrants. Int J Hum Resour Manag 26(10):1318–1334
Dlamini N, Anucha U, Wolfe B (2012) Negotiated positions: immigrant women’s views and experiences of employment in Canada. Affilia 27(4):420–434
Dolhansky B, Howes R, Pflaum B, Baram N, Ferrer CC (2019) The deepfake detection challenge (dfdc) preview dataset. arXiv preprint arXiv:1910.08854
Duguet E, Leandri N, Yannick L, Pascale P (2010) Are young French jobseekers of ethnic immigrant origin discriminated against? A controlled experiment in the Paris area. Ann Econ Stat Ann d’Écon Stat 2010:187–215
Dwork C, Ilvento C (2018) Individual fairness under composition. In: Proceedings of fairness, accountability, transparency in machine learning
Dwork C, Hardt M, Pitassi T, Reingold O, Zemel R (2012) Fairness through awareness. In: Proceedings of the 3rd innovations in theoretical computer science conference, pp 214–226
Earnshaw VA, Bogart LM, Klompas M, Katz IT (2019) Medical mistrust in the context of Ebola: implications for intended care-seeking and quarantine policy support in the United States. J Health Psychol 24(2):219–228
Elhage N, Nanda N, Olsson C, Henighan T, Joseph N, Mann B, Askell A, Bai Y, Chen A, Conerly T et al (2021) A mathematical framework for transformer circuits. Transform Circuits Thread 1
Fabbri F, Wang Y, Bonchi F, Castillo C, Mathioudakis M (2022) Rewiring what-to-watch-next recommendations to reduce radicalization pathways. arXiv preprint arXiv:2202.00640
Fagni T, Falchi F, Gambini M, Martella A, Tesconi M (2021) TweepFake: about detecting deepfake tweets. Plos One 16(5):e0251415
Falk AC, Schandl A, Frank C (2019) Barriers in achieving patient participation in the critical care unit. Intensive Crit Care Nurs 51:15–19
Fibbi R, Lerch M, Wanner P (2006) Unemployment and discrimination against youth of immigrant origin in Switzerland: when the name makes the difference. J Int Migr Integr Revue de l’integr Migr Int 7(3):351–366
Fink C (2018) Dangerous speech, anti-Muslim violence, and Facebook in Myanmar. J Int Aff 71(1.5):43–52
Ford ME, Kelly PA (2005) Conceptualizing and categorizing race and ethnicity in health services research. Health Serv Res 40:1658–1675
Fortuna P, Nunes S (2018) A survey on automatic detection of hate speech in text. ACM Comput Surv 51(4):30. https://doi.org/10.1145/3232676
Frías-Vázquez M, Arcila C (2019) Hate speech against Central American immigrants in Mexico: analysis of xenophobia and racism in politicians, media and citizens. In: Proceedings of the seventh international conference on technological ecosystems for enhancing multiculturality, pp 956–960
Futoma J, Hariharan S, Heller K (2017) Learning to detect sepsis with a multitask Gaussian process RNN classifier. In: International conference on machine learning. PMLR, pp 1174–1182
Galassi FM, Varotto E (2020) Sinophobia as the disease and palaeopathology as the cure: response to Adja, et al (2020) Public Health 187:189
Gambäck B, Sikdar UK (2017) Using convolutional neural networks to classify hate-speech. In: Proceedings of the first workshop on abusive language online, pp 85–90
Ganguli D, Hernandez D, Lovitt L, Askell A, Bai Y, Chen A, Conerly T, Dassarma N, Drain D, Elhage N et al (2022) Predictability and surprise in large generative models. In: 2022 ACM conference on fairness, accountability, and transparency, pp 1747–1764
Gellner E (2015) Nations and nationalism. In: Conflict after the Cold War. Routledge, pp 378–389
Giannakau E (2021) Migrants’ human rights facing surveillance technologies in immigration enforcement
Gilpin LH, Bau D, Yuan BZ, Bajwa A, Specter M, Kagal L (2018) Explaining explanations: an overview of interpretability of machine learning. In: 2018 IEEE 5th international conference on data science and advanced analytics (DSAA). IEEE, pp 80–89
Glaese A, McAleese N, Trȩbacz M, Aslanides J, Firoiu V, Ewalds T, Rauh M, Weidinger L, Chadwick M, Thacker P, Campbell-Gillingham L, Uesato J, Huang P-S, Comanescu R, Yang F, See A, Dathathri S, Greig R, Chen C, Fritz D, Elias JS, Green R, Mokrá S, Fernando N, Wu B, Foley R, Young S, Gabriel I, Isaac W, Mellor J, Hassabis D, Kavukcuoglu K, Hendricks LA, Irving G (2022) Improving alignment of dialogue agents via targeted human judgements. https://doi.org/10.48550/ARXIV.2209.14375
Göçeri E (2020) Impact of deep learning and smartphone technologies in dermatology: automated diagnosis. In: 2020 Tenth international conference on image processing theory, tools and applications (IPTA). IEEE, pp 1–6
Gomes A, Antonialli D, Oliva TD (2019) Drag queens and artificial intelligence: should computers decide what is ‘toxic’ on the internet? https://www.internetlab.org.br/en/freedom-of-expression/drag-queens-and-artificial-intelligence-should-computers-decide-what-is-toxic-on-the-internet/. Accessed 10 Sep 2020
Gordy ED (2010) Culture of power in Serbia: nationalism and the destruction of alternatives. Penn State Press, University Park
Goyal P, Duval Q, Seessel I, Caron M, Singh M, Misra I, Sagun L, Joulin A, Bojanowski P (2022) Vision models are more robust and fair when pretrained on uncurated images without supervision. arXiv preprint arXiv:2202.08360
Grand CL, Szulkin R (2002) Permanent disadvantage or gradual integration: explaining the immigrant-native earnings gap in Sweden. Labour 16(1):37–64
Grari V, Lamprier S, Detyniecki M (2021) Fairness without the sensitive attribute via Causal Variational Autoencoder. arXiv preprint arXiv:2109.04999
Güera D, Delp EJ (2018) Deepfake video detection using recurrent neural networks. In: 2018 15th IEEE international conference on advanced video and signal based surveillance (AVSS). IEEE, pp 1–6
Gu J, Hassan H, Devlin J, Li VOK (2018a) Universal neural machine translation for extremely low resource languages. arXiv preprint arXiv:1802.05368
Gu J, Wang Y, Chen Y, Cho K, Li VOK (2018b) Meta-learning for low-resource neural machine translation. arXiv preprint arXiv:1808.08437
Gui L, Leng J, Pergola G, Zhou Y, Xu R, He Y (2019) Neural topic model with reinforcement learning. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 3478–3483
Gupta S, Kamble V (2021) Individual fairness in hindsight. J Mach Learn Res 22(144):1–35
Gupta M, Cotter A, Fard MM, Wang S (2018) Proxy fairness. arXiv preprint arXiv:1806.11212
Hadžić F (2020) The European Union (EU) Political identity within the migrant crisis, and the Balkan-Bosnian route; xenophobia and religious identity. Res Soc Dev 9(10):e4809108685–e4809108685
Hafner L, Peifer TP, Hafner FS (2023) Equal accuracy for Andrew and Abubakar-detecting and mitigating bias in name-ethnicity classification algorithms. AI Soc:1–25
Haimson OL, Delmonaco D, Nie P, Wegner A (2021) Disproportionate removals and differing content moderation experiences for conservative, transgender, and black social media users: marginalization and moderation gray areas. Proc ACM Hum Comput Interact 5(CSCW2):1–35
Hakak LT, Holzinger I, Zikic J (2010) Barriers and paths to success: Latin American MBAs’ views of employment in Canada. J Manager Psychol
Hall P, Gill N, Schmidt N (2019) Proposed guidelines for the responsible use of explainable machine learning. arXiv preprint arXiv:1906.03533
Handayani IGAKR, Karjoko L et al (2021) Dysfunction of foreign worker employment regulation to prevent xenophobia. In: International conference on environmental and energy policy (ICEEP 2021). Atlantis Press, pp 14–17
Hanna A, Denton E, Smart A, Smith-Loud J (2020) Towards a critical race methodology in algorithmic fairness. In: Proceedings of the 2020 conference on fairness, accountability, and transparency, pp 501–512
Hannah B, Suruchi T-B, Sarah H, Beth MA (2019) Undoing the unspeakable: researching racism in Swedish healthcare using a participatory process to build dialogue. Health Res Policy Syst 17(1):1–6
Hashimoto T, Srivastava M, Namkoong H, Liang P (2018) Fairness without demographics in repeated loss minimization. In: International conference on machine learning. PMLR, pp 1929–1938
Haunschild J, Kaufhold M-A, Reuter C (2022) Cultural violence and fragmentation on social media: interventions and countermeasures by humans and social bots. In: Cyber Security Politics. Routledge, pp 48–63
Helberger N, Karppinen K, D’acunto L (2018) Exposure diversity as a design principle for recommender systems. Inf Commun Soc 21(2):191–207
Hirsch B, Elke JJ (2015) Is there monopsonistic discrimination against immigrants? ILR Rev 68(3):501–528
Hooper MW, Nápoles AM, Eliseo P-SJ (2020) COVID-19 and racial/ethnic disparities. Jama 323(24):2466–2467
Hsu J (2020) Can AI hiring systems be made antiracist? Makers and users of AI-assisted recruiting software reexamine the tools’ development and how they’re used-[News]. IEEE Spectr 57(9):9–11
Huang J, Chen-Chuan CK (2022) Towards reasoning in large language models: a survey. arXiv preprint arXiv:2212.10403
Huang P-Y, Chang X, Hauptmann A (2019a) Multi-head attention with diversity for learning grounded multilingual multimodal representations. arXiv preprint arXiv:1910.00058
Huang P-S, Zhang H, Jiang R, Stanforth R, Welbl J, Rae J, Maini V, Yogatama D, Kohli P (2019b) Reducing sentiment bias in language models via counterfactual evaluation. arXiv preprint arXiv:1911.03064
Huang J, Chen X, Mishra S, Zheng HS, Yu AW, Song X, Zhou D (2023) Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798
Hu L, Chen Y (2017) Fairness at equilibrium in the labor market. arXiv preprint arXiv:1707.01590
Hu L, Chen Y (2018) A short-term intervention for long-term fairness in the labor market. In: Proceedings of the 2018 World Wide Web conference, pp 1389–1398
Hwang Y, Ryu JY, Jeong S-H (2021) Effects of disinformation using deepfake: the protective effect of media literacy education. Cyberpsychol Behav Soc Netw 24(3):188–193
Igartua J-J, Frutos FJ (2017) Enhancing attitudes toward stigmatized groups with movies: mediating and moderating processes of narrative persuasion. Int J Commun 11:20
Ilvento C (2019) Metric learning for individual fairness. arXiv preprint arXiv:1906.00250
Jaegle A, Borgeaud S, Alayrac J-B, Doersch C, Ionescu C, Ding D, Koppula S, Zoran D, Brock A, Shelhamer E et al (2021) Perceiver io: a general architecture for structured inputs & outputs. arXiv preprint arXiv:2107.14795
Jalal A, Karmalkar S, Hoffmann J, Dimakis A, Price E (2021b) Fairness for image generation with uncertain sensitive attributes. In: Meila M, Zhang T (eds) Proceedings of the 38th international conference on machine learning (proceedings of machine learning research, Vol. 139). PMLR, pp 4721–4732. https://proceedings.mlr.press/v139/jalal21b.html
Janmyr M (2016) Precarity in exile: the legal status of Syrian refugees in Lebanon. Refugee Surv Q 35(4):58–78
Jardina A (2019) White identity politics. Cambridge University Press, Cambridge
Jha A, Prabhakaran V, Denton R, Laszlo S, Dave S, Qadri R, Reddy CK, Dev S (2024) Beyond the surface: a global-scale analysis of visual stereotypes in text-to-image generation. arXiv:2401.06310 [cs.CV]
Ji Z, Lee N, Frieske Rita Yu, Dan Tiezheng S, Yan X, Ishii E, Ye JB, Madotto A, Fung P (2023) Survey of hallucination in natural language generation. Comput Surv 55(12):1–38
Jiang R, Chiappa S, Lattimore T, György A, Kohli P (2019) Degenerate feedback loops in recommender systems. In: Proceedings of the 2019 AAAI/ACM conference on AI, ethics, and society, pp 383–390
Jin N, Jiaxian W, Ma X, Yan K, Mo Y (2020) Multi-task learning model based on multi-scale CNN and LSTM for sentiment classification. IEEE Access 8(2020):77060–77072
Jones W, Teytelboym A (2017) The international refugee match: a system that respects refugees’ preferences and the priorities of States. Refug Surv Q 36(2):84–109. 1020-4067 https://doi.org/10.1093/rsq/hdx004. https://academic.oup.com/rsq/article-pdf/36/2/84/17620356/hdx004.pdf
Joo J, Kärkkäinen K (2020) Gender slopes: Counterfactual fairness for computer vision models by attribute manipulation. In: Proceedings of the 2nd international workshop on fairness, accountability, transparency and ethics in multimedia, pp 1–5
Jörke D (2019) Routledge handbook of global populism. Carlos de la Torre ed. Routledge, London and New York. Constellations 26:655–658. https://doi.org/10.1111/1467-8675.12455
José G-R, Raquel T, Miguel I-C et al (2021) Multimodal disinformation about otherness on the internet. The spread of racist, xenophobic and Islamophobic fake news in 2020. Anàlisi, pp 49–64
Joshi G, Walambe R, Kotecha K (2021) A review on explainability in multimodal deep neural nets. IEEE Access 9:59800–59821
Jung C, Kearns MJ, Neel S, Roth A, Stapleton L, Wu ZS (2019) Eliciting and enforcing subjective individual fairness
Jureidini R (2005) Migrant workers and xenophobia in the Middle East. In: Racism and public policy. Springer, pp 48–71
Kaliyar RK, Goswami A, Narang P (2021) FakeBERT: fake news detection in social media with a BERT-based deep learning approach. Multimedia Tools Appl 80(8):11765–11788
Kamusella T et al (2021) Xenophobia and anti-Semitism in the concept of polish literature. Śląskie Stud Polonistyczne 1(17):1–18
Karakanta A, Dehdari J, van Genabith J (2018) Neural machine translation for low-resource languages without parallel corpora. Mach Transl 32(1):167–189
Karimi A-H, Schölkopf B, Valera I (2021) Algorithmic recourse: from counterfactual explanations to interventions. In: Proceedings of the 2021 acm conference on fairness, accountability, and transparency (virtual event, Canada) (FAccT ’21). Association for Computing Machinery, New York, NY, USA, pp 353–362. https://doi.org/10.1145/3442188.3445899
Kasirzadeh A, Smart A (2021) The use and misuse of counterfactuals in ethical machine learning. In: Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pp 228–236
Kearns M, Neel S, Roth A, Wu ZS (2018) Preventing fairness gerrymandering: auditing and learning for subgroup fairness. In: International conference on machine learning. PMLR, pp 2564–2572
Kee P (1994) Native-immigrant employment differentials in The Netherlands: the role of assimilation and discrimination. Int Rev Appl Econ 8(2):174–196
Khan MUS, Abbas A, Rehman A, Nawaz R (2020) HateClassify: a service framework for hate speech identification on social media. IEEE Internet Comput 25(1):40–49
Khandelwal K, Tonneau M, Bean AM, Kirk HR, Hale SA (2023) Casteist but not racist? Quantifying disparities in large language model bias between India and the West. arXiv:2309.08573 [cs.CL]
Khandy I (2021) Pop culture: a vehicle of state nationalism in India. In: Nationalism and the postcolonial. Brill, pp 111–130
KhosraviNik M (2017) Right wing populism in the west: social media discourse and echo chambers. Insight Turkey 19(3):53–68
Kim PT (2018) Big Data and artificial intelligence: new challenges for workplace equality. U Louisville L Rev 57:313
Kim PT, Bodie MT (2021) Artificial intelligence and the challenges of workplace discrimination and privacy. J Labor Employ Law 35(2):289–315
Kim HD, Ronald S (2014) Xenophobia and racism. Crit Philos Race 2:1
Kim M, Reingold O, Rothblum G (2018) Fairness through computationally-bounded awareness. Adv Neural Inf Process Syst 31
Kopyciok S, Silver H (2021) Left-wing xenophobia in Europe. Front Sociol 6
Kosny A, Santos I, Reid A (2017) Employment in a “land of opportunity’’ Immigrants’ experiences of racism and discrimination in the Australian workplace. J Int Migr Integr 18(2):483–497
Krings F, Johnston C, Binggeli S, Maggiori C (2014) Selective incivility: immigrant groups experience subtle workplace discrimination at different rates. Cult Divers Ethnic Minor Psychol 20(4):491
Krumpal I (2012) Estimating the prevalence of xenophobia and anti-Semitism in Germany: a comparison of randomized response and direct questioning. Soc Sci Res 41(6):1387–1403
Kunaver M, Požrl T (2017) Diversity in recommender systems—a survey. Knowl Based Syst 123:154–162
Kusner MJ, Loftus J, Russell C, Silva R (2017) Counterfactual fairness. Adv Neural Inf Process Syst 30
Lavenex S (2001) Migration and the EU’s new eastern border: between realism and liberalism. J Eur Public Policy 8(1):24–42
Le TK, Cha L, Han H-R, Tseng W (2020) Anti-Asian xenophobia and Asian American COVID-19 disparities, pp 1371–1373
Leben D (2020) Normative principles for evaluating fairness in machine learning. In: Proceedings of the AAAI/ACM conference on AI, ethics, and society, pp 86–92
Lee RK-W, Li Z (2020) Online xenophobic behavior amid the COVID-19 pandemic: a commentary. Digit Gov Res Pract 2(1):5. https://doi.org/10.1145/3428091
Leerssen J (2006) Nationalism and the cultivation of culture. Nations Natl 12(4):559–578
Leonhardt J, Anand A, Khosla M (2018) User fairness in recommender systems. In: Companion proceedings of the the web conference 2018, pp 101–102
Liang PP, Wu C, Morency L-P, Salakhutdinov R (2021) Towards understanding and mitigating social biases in language models. In: International conference on machine learning. PMLR, pp 6565–6576
Liem C, Langer M, Demetriou A, Hiemstra AMF, Sukma WA, Born MPh, König CJ (2018) Psychology meets machine learning: interdisciplinary perspectives on algorithmic job candidate screening. In: Explainable and interpretable models in computer vision and machine learning. Springer, pp 197–253
Lightman H, Kosaraju V, Burda Y, Edwards H, Baker B, Lee T, Leike J, Schulman J, Sutskever I, Cobbe K (2023) Let’s verify step by step. arXiv:2305.20050 [cs.LG]
Li T, Khot T, Khashabi D, Sabharwal A, Srikumar V (2020) UNQOVERing stereotyping biases via underspecified questions. arXiv preprint arXiv:2010.02428
Li J, Tang T, Zhao WX, Wen J-R (2021) Pretrained language models for text generation: a survey. https://doi.org/10.48550/ARXIV.2105.10311
Li Z, Zhou H, Bai S, Li P, Zhou C, Yang H (2022) M6-fashion: high-fidelity multi-modal image generation and editing. https://doi.org/10.48550/ARXIV.2205.11705
Liu CC, Koto F, Baldwin T, Gurevych I (2023) Are multilingual LLMs culturally-diverse reasoners? an investigation into multicultural proverbs and sayings. arXiv:2309.08591 [cs.CL]
Loganathan T, Rui D, Ng C-W, Nicola SP (2019) Breaking down the barriers: understanding migrant workers’ access to healthcare in Malaysia. PloS one 14(7):e0218669
Long AS (2021) Representation in American Film, Queer and Now
Long J (2023) Large language model guided tree-of-thought. arXiv preprint arXiv:2305.08291
Longhi V (2012) The immigrant war: a global movement against discrimination and exploitation. Policy Press, Bristol
Longo M (2017) The politics of borders: sovereignty, security, and the citizen after 9/11. Cambridge University Press, Cambridge
Luceri L, Giordano S, Ferrara E (2020) Detecting troll behavior via inverse reinforcement learning: a case study of russian trolls in the 2016 us election. In: Proceedings of the International AAAI conference on web and social media 14, pp 417–427
Lucherini E, Sun M, Winecoff A, Narayanan A (2021) T-RECS: A simulation tool to study the societal impact of recommender systems. arXiv preprint arXiv:2107.08959
Lu P, Peng B, Cheng H, Galley M, Chang K-W, Wu YN, Zhu S-C, Gao J (2023) Chameleon: Plug-and-play compositional reasoning with large language models. arXiv preprint arXiv:2304.09842
Lyu Q, Havaldar S, Stein A, Zhang L, Rao D, Wong E, Apidianaki M, Callison-Burch C (2023) Faithful chain-of-thought reasoning. arXiv preprint arXiv:2301.13379
Mahmoud AA, Shawabkeh TAL, Salameh WA, Al Amro I (2019) Performance predicting in hiring process and performance appraisals using machine learning. In: 2019 10th International conference on information and communication systems (ICICS). IEEE, pp 110–115
Makoni B (2020) Metalinguistic discourses on translanguaging and multimodality: acts of passing by black African immigrants in Johannesburg. Lang Cult Soc 2(1):66–91
Mamdani M (2002) When victims become killers: colonialism, nativism, and the genocide in Rwanda. Princeton University Press. 9780691102801 00065213 https://books.google.co.uk/books?id=QUEamxb89JcC
Mamdani M (2012) Define and rule. In: Define and rule. Harvard University Press, Harvard
Mamdani M (2018) Citizen and subject: contemporary Africa and the legacy of late colonialism. Princeton University Press, Princeton
Mandal D, Deng S, Jana S, Wing J, Hsu Daniel J (2020) Ensuring fairness beyond the training data. Adv Neural Inf Process Syst 33(2020):18445–18456
Mann M (2005) The dark side of democracy: explaining ethnic cleansing. Cambridge University Press, Cambridge
Mansouri M (2023) A call for epistemic analysis of cultural theories for AI methods. AI Soc 38(2):969–971
Mansoury M, Abdollahpouri H, Pechenizkiy M, Mobasher B, Burke R (2020) Fairmatch: a graph-based approach for improving aggregate diversity in recommender systems. In: Proceedings of the 28th ACM conference on user modeling, adaptation and personalization, pp 154–162
Martin Jr D, Prabhakaran V, Kuhlberg J, Smart A, Isaac WS (2020) Participatory problem formulation for fairer machine learning through community based system dynamics. arXiv preprint arXiv:2005.07572
Maximilian Z, Emese D (2022) Thirty years after the German reunification-exploring stereotypes about east Germans on twitter. Int J Commun 16(2022):21
Mayadas NS, Elliott D (1992) Integration and xenophobia: an inherent conflict in international migration. J Multicult Soc Work 2(1):47–62
McGrath T, Kapishnikov A, Tomašev N, Pearce A, Wattenberg M, Hassabis D, Kim B, Paquet U, Kramnik V (2022) Acquisition of chess knowledge in alpha zero. Proc Natl Acad Sci 119(47):e2206625119
McGuffie K, Newhouse A (2020) The radicalization risks of GPT-3 and advanced neural language models. CoRR . arXiv:2009.06807
McKinney SM, Sieniek M, Godbole V, Godwin J, Antropova N, Ashrafian H, Back T, Chesus M, Corrado GS, Darzi A et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577(7788):89–94
Mead EL (2020) Creating a machine learning model for the prediction of refugee flows. Ph.D. Dissertation. University of Arkansas at Little Rock
Mehrabi N, Morstatter F, Saxena N, Lerman K, Galstyan A (2021) A survey on bias and fairness in machine learning. ACM Comput Surv (CSUR) 54(6):1–35
Meurs D (2017) The role of discrimination in immigrant unemployment. Popul Soc 7(2017):1–4
Midtbøen AH (2014) The invisible second generation? Statistical discrimination and immigrant stereotypes in employment processes in Norway. J Ethnic Migr Stud 40(10):1657–1675
Minga KJ (2015) Xenophobia in literature and film as a Re-claim of space and Re-make of apartheid. Glob Media J Afr Ed 9(2):268–297
Mitchell M, Wu S, Zaldivar A, Barnes P, Vasserman L, Hutchinson B, Spitzer E, Raji ID, Gebru T (2019) Model cards for model reporting. In: Proceedings of the conference on fairness, accountability, and transparency, pp 220–229
Moscou S, Anderson MR, Kaplan BJ, Valencia L (2003) Validity of racial/ethnic classifications in medical records data: an exploratory study. Am J Public Health 93(7):1084–1086
Mubangizi JC (2021) Xenophobia in the labour market: a South African legal and human rights perspective. Int J Discrimin Law 21(2):139–156
Mujtaba DF, Mahapatra NR (2019) Ethical considerations in ai-based recruitment. In: 2019 IEEE international symposium on technology and society (ISTAS). IEEE, pp 1–7
Mukherjee D, Yurochkin M, Banerjee M, Sun Y (2020) Two simple ways to learn individual fairness metrics from data. In: International conference on machine learning. PMLR, pp 7097–7107
Müller K, Schwarz C (2021) Fanning the flames of hate: social media and hate crime. J Eur Econ Assoc 19(4):2131–2167
Müller-Crepon C, Pengl Y, Bormann N-C (2021) Linking ethnic data from Africa (LEDA). J Peace Res 2021:00223433211016528
Munyaneza Y, Euphemia MM (2019) Medical xenophobia: the voices of women refugees in Durban, Kwazulu-Natal, South Africa. BioRxiv 2019:603753
Nadeem M, Bethke A, Reddy S (2020) Stereoset: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456
Nalbandian L (2022) An eye for an ‘I:’a critical assessment of artificial intelligence tools in migration and asylum management. Compar Migr Stud 10(1):1–23
Nandy P, Diciccio C, Venugopalan D, Logan H, Basu K, El Karoui N (2022) Achieving fairness via post-processing in web-scale recommender systems. In: 2022 ACM conference on fairness, accountability, and transparency, pp 715–725
Nangia N, Vania C, Bhalerao R, Bowman SR (2020) CrowS-pairs: a challenge dataset for measuring social biases in masked language models. arXiv preprint arXiv:2010.00133
Natasha AM, Dineke ZP, McKee M (2017) Public health must confront racism and xenophobia. Lancet 390(10098):931
Neekhara P, Dolhansky B, Bitton J, Ferrer CC (2021) Adversarial threats to deepfake detection: a practical perspective. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 923–932
Nguyen TT, Criss S, Dwivedi P, Huang D, Keralis J, Hsu E, Phan L, Nguyen LH, Yardi I, Glymour MM, Allen AM, Chae DH, Gee GC, Nguyen QC (2020) Exploring U.S. shifts in anti-Asian sentiment with the emergence of COVID-19. In: International journal of environmental research and public health 17, 19, pp 1660-4601. https://doi.org/10.3390/ijerph17197032
Nguyen-Truong CKY, Rakha S, Eti Deborah U, Angelesco L (2021) Deconstructing racialized experiences in healthcare: what a missed opportunity for healing looks like and healthcare resources for children and their families. Asian/Pac Island Nurs J 5(4):227
Nickell A, Stewart Susan L, Burke Nancy J, Guerra C, Cohen E, Lawlor C, Colen S, Cheng J, Joseph G (2019) Engaging limited English proficient and ethnically diverse low-income women in health research: a randomized trial of a patient navigator intervention. Patient Educ Counsel 102(7):1313–1323
Nielsen HS, Rosholm M, Smith N, Husted L (2004) Qualifications, discrimination, or assimilation? An extended framework for analysing immigrant wage gaps. Emp Econ 29(4):855–883
Norris KC, Eneanya ND, Ebony BL (2021) Removal of race from estimates of kidney function: first, do no harm. Jama 325(2):135–137
Nozza D, Bianchi F, Hovy D (2021) HONEST: Measuring hurtful sentence completion in language models. In: The 2021 conference of the North American chapter of the association for computational linguistics: human language technologies. Association for Computational Linguistics
Obenauer WG (2023) More on why Lakisha and Jamal didn’t get interviews: extending previous findings through a reproducibility study. J Manag Sci Rep 2023:27550311231167370
O’Callaghan D, Greene D, Conway M, Carthy J, Cunningham P (2015) Down the (white) rabbit hole: the extreme right and online recommender systems. Soc Sci Comput Rev 33(4):459–478
Oliva TD, Marcelo AD, Gomes A (2021) Fighting hate speech, silencing drag queens? Artificial intelligence in content moderation and risks to LGBTQ voices online. Sex Cult 25(2):700–733
Olonisakin TT, Adebayo SO (2021) Xenophobia: scale development and validation. J Contemp Afr Stud 39(3):484–496
O’Neil C (2016) Weapons of math destruction: how big data increases inequality and threatens democracy. Crown Publishing Group, USA, p 0553418815
Ousidhoum ND, Zhao X, Fang T, Song Y, Yeung DY (2021) Probing toxic content in large pre-trained language models. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing
Ouyang L, Wu J, Jiang X, Almeida D, Wainwright CL, Mishkin P, Zhang C, Agarwal S, Slama K, Ray A, Schulman J, Hilton J, Kelton F, Miller L, Simens M, Askell A, Welinder P, Christiano P, Leike J, Lowe R (2022) Training language models to follow instructions with human feedback. https://doi.org/10.48550/ARXIV.2203.02155
Pan X, Wang M, Wu L, Li L (2021) Contrastive learning for many-to-many multilingual neural machine translation. arXiv preprint arXiv:2105.09501
Papakyriakopoulos O (2020) Political machines: machine learning for understanding the politics of social machines. Ph.D. Dissertation. Technische Universität München
Parfett A, Townley S, Allerfeldt K (2021) AI-based healthcare: a new dawn or apartheid revisited? AI Soc 36(2021):983–999
Patti V, Damiano R, Bosco C (2017) Ethical implications of analyzing opinions, emotions and interactions in social media. In: 2017 Seventh international conference on affective computing and intelligent interaction workshops and demos (ACIIW). IEEE, pp 153–158
Pavlíková M, Šenkỳřová B, Drmola J (2021) Propaganda and disinformation go online. In: Challenging online propaganda and disinformation in the 21st century. Springer, pp 43–74
Peng J, Zhou Y, Cao L, Sun X, Su J, Ji R (2019) Towards cross-modality topic modelling via deep topical correlation analysis. In: ICASSP 2019—2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 4115–4119. https://doi.org/10.1109/ICASSP.2019.8682753
Pfohl SR, Duan T, Ding DY, Shah NH (2019) Counterfactual reasoning for fair clinical risk prediction. In: Machine learning for healthcare conference. PMLR, pp 325–358
Plaza-Del-Arco F-M, Dolores M-GM, Alfonso U-LL, Teresa M-VM (2020) Detecting misogyny and xenophobia in Spanish tweets using language technologies. ACM Trans Internet Technol (TOIT) 20(2):1–19
Pogge T (2002) Cosmopolitanism: a defence. Crit Rev Int Soc Polit Philos 5(3):86–91
Polubriaginof FCG, Ryan P, Salmasian H, Andrea WS, Perotte A, Safford Monika M, Hripcsak G, Smith S, Tatonetti NP, Vawdrey DK (2019) Challenges with quality of race and ethnicity data in observational databases. J Am Med Informat Assoc 26(8–9):730–736
Popat K, Mukherjee S, Yates A, Weikum G (2018) Declare: debunking fake news and false claims using evidence-aware deep learning. arXiv preprint arXiv:1809.06416
Postill J (2018) Populism and social media: a global perspective. Media Cult Soc 40(5):754–765
Prabhakaran V, Mitchell M, Gebru T, Gabriel I (2022) A human rights-based approach to responsible AI. arXiv preprint arXiv:2210.02667
Prince AER, Schwarcz D (2019) Proxy discrimination in the age of artificial intelligence and big data. Iowa L Rev 105(2019):1257
Pushkarna M, Zaldivar A, Kjartansson O (2022) Data cards: purposeful and transparent dataset documentation for responsible AI. arXiv preprint arXiv:2204.01075
Qian X, Pei J, Zheng H, Xie X, Yan L, Zhang H, Han C, Gao X, Zhang H, Zheng W et al (2021) Prospective assessment of breast cancer risk from multimodal multiview ultrasound images via clinically applicable deep learning. Nat Biomed Eng 5(6):522–532
Rae JW, Borgeaud S, Cai T, Millican K, Hoffmann J, Song F, Aslanides J, Henderson S, Ring R, Young S et al (2021) Scaling language models: methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446
Raghavan M, Barocas S, Kleinberg J, Levy K (2020) Mitigating bias in algorithmic hiring: evaluating claims and practices. In: Proceedings of the 2020 conference on fairness, accountability, and transparency, pp 469–481
Rajalakshmi R, Subashini R, Ranjit MA, Mohan V (2018) Automated diabetic retinopathy detection in smartphone-based fundus photography using artificial intelligence. Eye 32(6):1138–1144
Raji ID, Buolamwini J (2019) Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial ai products. In: Proceedings of the 2019 AAAI/ACM conference on AI, ethics, and society, pp 429–435
Raji ID, Smart A, White RN, Mitchell M, Gebru T, Hutchinson B, Smith-Loud J, Theron D, Barnes P (2020) Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In: Proceedings of the 2020 conference on fairness, accountability, and transparency, pp 33–44
Ramesh A, Pavlov M, Goh G, Gray S, Voss C, Radford A, Chen M, Sutskever I (2021) Zero-shot text-to-image generation. https://doi.org/10.48550/ARXIV.2102.12092
Ramesh K, Sitaram S, Choudhury M (2023) Fairness in language models beyond English: gaps and challenges. arXiv:2302.12578 [cs.CL]
Ravi D, Wong C, Lo B, Yang G-Z (2016) A deep learning approach to on-node sensor data analytics for mobile or wearable devices. IEEE J Biomed Health Informat 21(1):56–64
Rehaag S (2023) Using AI to examine decision-making in federal court stays of removal, luck of the draw III
Reny TT, Barreto MA (2020) Xenophobia in the time of pandemic: othering, anti-Asian attitudes, and COVID-19. Polit Groups Ident 2020:1–24
Ribeiro MH, Ottoni R, West R, Almeida VAF, Meira Jr W (2020) Auditing radicalization pathways on YouTube. In: Proceedings of the 2020 conference on fairness, accountability, and transparency, pp 131–141
Richter R (2015) Medical outcasts: gendered and institutionalized xenophobia in undocumented forced migrants’ emergency health care. Lexington Books, Lanham
Ringelheim J (2011) Ethnic categories and European human rights law. Ethn Racial Stud 34(10):1682–1696
Rizos G, Hemker K, Schuller B (2019) Augment to prevent: short-text data augmentation in deep learning for hate-speech classification. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 991–1000
Robinson C, Dilkina B (2018) A machine learning approach to modeling human migration (COMPASS ’18). Association for Computing Machinery, New York, NY, USA, Article 30, p 8. https://doi.org/10.1145/3209811.3209868
Röttger P, Vidgen B, Nguyen D, Waseem Z, Margetts HZ, Pierrehumbert JB (2020) HateCheck: functional tests for hate speech detection models. CoRRarXiv:2012.15606
Routen A, Akbari A, Banerjee A, Srinivasa VK, Mathur R, McKee M, Nafilyan V, Khunti K (2022) Strategies to record and use ethnicity information in routine health data. Nat Med 2022:1–4
Rydgren J (2004) The Logic of Xenophobia. Rational Soc 16(2):123–148. https://doi.org/10.1177/1043463104043712
Saharia C, Chan W, Saxena S, Li L, Whang J, Denton E, Ghasemipour SKS, Ayan BK, Mahdavi SS, Lopes RG, Salimans T, Ho J, Fleet DJ, Norouzi M (2022) Photorealistic text-to-image diffusion models with deep language understanding. https://doi.org/10.48550/ARXIV.2205.11487
Sanchez-Mazas M, Licata L (2015) Xenophobia: social psychological aspects. Int Encycl Soc Behav Sci 802
Sánchez-Monedero J, Dencik L, Edwards L (2020) What does it mean to ’solve’ the problem of discrimination in hiring? Social, technical and legal perspectives from the UK on automated hiring systems. In: Proceedings of the 2020 conference on fairness, accountability, and transparency, pp 458–468
Sarbu I, Matei C, Benea V, Georgescu SR (2014) Brief history of syphilis. J Med Life 7(1):4
Scao TL, Fan A, Akiki C, Pavlick E, Ilić S, Hesslow D, Castagné R, Luccioni AS, Yvon F, Gallé M et al (2022) BLOOM: a 176B-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100
Schäfer C, Schadauer A (2018) Online fake news, hateful posts against refugees, and a surge in xenophobia and hate crimes in Austria. In: Refugee news, refugee politics. Routledge, pp 109–116
Scheurer J, Campos JA, Chan JS, Chen A, Cho K, Perez E (2022) Training language models with language feedback. https://doi.org/10.48550/ARXIV.2204.14146
Schinkel S, Schouten BC, Kerpiclik F, Den PBV, Van Weert JCM (2018) Perceptions of barriers to patient participation: are they due to language, culture, or discrimination? Health Commun
Schumann C, Foster J, Mattei N, Dickerson J (2020) We need fairness and explainability in algorithmic hiring. In: International conference on autonomous agents and multi-agent systems (AAMAS)
Schut L, Tomasev N, McGrath T, Hassabis D, Paquet U, Kim B (2023) Bridging the Human-AI knowledge gap: concept discovery and transfer in AlphaZero. arXiv preprint arXiv:2310.16410
Schwöbel P, Remmers P (2022) The long arc of fairness: formalisations and ethical discourse. arXiv preprint arXiv:2203.06038
Seneviratne MG, Shah Nigam H, Chu L (2020) Bridging the implementation gap of machine learning in healthcare. BMJ Innov 6:2
Sennrich R, Zhang B (2019) Revisiting low-resource neural machine translation: a case study. arXiv preprint arXiv:1905.11901
Seo S, Kim C, Kim H, Mo K, Kang P (2020) Comparative study of deep learning-based sentiment classification. IEEE Access 8:6861–6875
Shaheen JG (2003) Reel bad Arabs: how Hollywood vilifies a people. Ann Am Acad Polit Soc Sci 588(1):171–193
Shahvisi A (2019) Austerity or xenophobia? The causes and costs of the “hostile environment’’ in the NHS. Health Care Anal 27(3):202–219
Sharifi-Malvajerdi S, Kearns M, Roth A (2019) Average individual fairness: algorithms, generalization and experiments. Adv Neural Inf Process Syst 32
Sharma S, Henderson J, Ghosh J (2020) Certifai: A common framework to provide explanations and analyse the fairness and robustness of black-box models. In: Proceedings of the AAAI/ACM conference on AI, ethics, and society, pp 166–172
Shi F, Chen X, Misra K, Scales N, Dohan D, Chi EH, Schärli N, Zhou D (2023) Large language models can be easily distracted by irrelevant context. In: International conference on machine learning. PMLR, pp 31210–31227
Siddhant A, Bapna A, Cao Y, Firat O, Chen M, Kudugunta S, Arivazhagan N, Wu Y (2020) Leveraging monolingual data with self-supervision for multilingual neural machine translation. arXiv preprint arXiv:2005.04816
Słowik A, Bottou L (2021) Algorithmic bias and data bias: understanding the relation between distributionally robust optimization and data curation. arXiv preprint arXiv:2106.09467
Smith AD (2002) The politics of culture: ethnicity and nationalism. In: Companion encyclopedia of anthropology. Routledge, pp 740–768
Smith RC, Winschiers-Theophilus H, Loi D, Paula RAde, Kambunga AP, Samuel MM, ZT (2021) Decolonizing design practices: towards pluriversality. In: Extended abstracts of the 2021 CHI conference on human factors in computing systems, pp 1–5
Sokol K, Flach PA (2019) Counterfactual explanations of machine learning predictions: opportunities and challenges for AI safety. SafeAI@ AAAI
Sonntag J (2019) Evidence from Facebook, social norms and xenophobia
Srinivasan T, Bisk Y (2021) Worst of both worlds: biases compound in pre-trained vision-and-language models. https://doi.org/10.48550/ARXIV.2104.08666
Srinivasan K, Raman K, Chen J, Bendersky M, Najork M (2021) Wit: Wikipedia-based image text dataset for multimodal multilingual machine learning. In: Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pp 2443–2449
Straus S (2015) Making and unmaking nations. Cornell University Press, Cornell
Stray J (2021) Designing recommender systems to depolarize. arXiv preprint arXiv:2107.04953
Strickland E (2018) AI-human partnerships tackle “fake news’’: machine learning can get you only so far-then human judgment is required-[News]. IEEE Spectr 55(9):12–13
Strmic-Pawl HV, Jackson BA, Garner S (2018) Race counts: racial and ethnic data on the US Census and the implications for tracking inequality. Sociol Race Ethn 4(1):1–13
Strümke I, Slavkovik M (2022) Explainability for identification of vulnerable groups in machine learning models. arXiv preprint arXiv:2203.00317
Suleman S, Garber Kent D, Rutkow L (2018) Xenophobia as a determinant of health: an integrative review. J Publ Health Policy 39(4):407–423
Sutejo TL, Lestari DP (2018) Indonesia hate speech detection using deep learning. In: 2018 international conference on Asian language processing (IALP). IEEE, pp 39–43
Swain D, Chakraborty K, Dombe A, Ashture A, Valakunde N (2018) Prediction of H1B visa using machine learning algorithms. In: 2018 International conference on advanced computation and telecommunication (ICACAT), pp 1–7. https://doi.org/10.1109/ICACAT.2018.8933628
Taran P, Beijl RZ de, McClure I (2004) Challenging discrimination in employment: a summary of research and a typology of measures. Int Migr Pap 68
Taras R (2009) Transnational xenophobia in Europe? Literary representations of contemporary fears. Eur Legacy 14(4):391–407
Teelucksingh C, Galabuzi G-E (2007) Working precariously: The impact of race and immigrant status on employment opportunities and outcomes in Canada. Race Racial Essent Read 202
Teixeira C, Lo L, Truelove M (2007) Immigrant entrepreneurship, institutional discrimination, and implications for public policy: a case study in Toronto. Environ Plann C Gov Policy 25(2):176–193
Tekiroglu SS, Bonaldi H, Fanton M, Guerini M (2022) Using pre-trained language models for producing counter narratives against hate speech: a comparative study. https://doi.org/10.48550/ARXIV.2204.01440
Tewolde AI (2021) “Passing’’ as Black South African: identity concealment tactics of Eritrean refugees in the face of xenophobia. Soc Ident 27(2):184–196
Tomašev N, Glorot X, Rae JW, Zielinski M, Askham H, Saraiva A, Mottram A, Meyer C, Ravuri S, Protsyuk I et al (2019) A clinically applicable approach to continuous prediction of future acute kidney injury. Nature 572(7767):116–119
Tomašev N, Harris N, Baur S, Mottram A, Glorot X, Rae JW, Zielinski M, Askham H, Saraiva A, Magliulo V et al (2021) Use of deep learning to develop continuous-risk models for adverse event prediction from electronic health records. Nat Protoc 16(6):2765–2787
Tomasev N, McKee KR, Jackie K, Shakir M (2021) Fairness for unobserved characteristics: insights from technological impacts on queer communities. Association for Computing Machinery, New York, pp 254–265
Tsai Y-HH, Ma MQ, Zhao H, Zhang K, Morency L-P, Salakhutdinov R (2021) Conditional contrastive learning: removing undesirable information in self-supervised representations. arXiv preprint arXiv:2106.02866 (2021)
Turpin M, Michael J, Perez E, Bowman SR (2023) Language models don’t always say what they think: unfaithful explanations in chain-of-thought prompting. arXiv:2305.04388 [cs.CL]
Ullah I, Shahzor K (2017) Cultural (mis) appropriation, ideological essentialism and language: analysis of stereotyping in Hollywood movie. Int J English Linguist 6(7):171–177
Um K-H, Kim S-M (2018) Application of fairness theory to medical tourists’ dissatisfaction and complaint behaviors: the moderating role of patient participation in medical tourism. J Soc Serv Res 44(2):191–208
Upadhyay AK, Khandelwal K (2018) Applying artificial intelligence: implications for recruitment. Strateg HR Rev
Vaccaro K, Sandvig C, Karahalios K (2020) At the end of the day facebook does what it wants’’ How users experience contesting algorithmic content moderation. Proc ACM Hum Comput Interact 4(CSCW2):1–22
Valmeekam K, Marquez M, Kambhampati S (2023) Can large language models really improve by self-critiquing their own plans? arXiv:2310.08118 [cs.AI]
Van der Veer K, Ommundsen R, Yakushko O, Higler L, Woelders S, Hagen KA (2013) Psychometrically and qualitatively validating a cross-national cumulative measure of fear-based xenophobia. Qual Quant 47(3):1429–1444
van den Broek E, Sergeeva A, Huysman M (2020) Hiring algorithms: An ethnography of fairness in practice. In: 40th international conference on information systems, ICIS 2019. Association for Information Systems, pp 1–9
Varcoe C, Browne AJ, Wong S, Smye VL (2009) Harms and benefits: collecting ethnicity data in a clinical context. Soc Sci Med 68(9):1659–1666
Venkit PN, Gautam S, Panchanadikar R, Huang T-HK, Wilson S (2023) Nationality bias in text generation. arXiv: 2302.02463 [cs.CL]
Verma S, Dickerson J, Hines K (2020) Counterfactual explanations for machine learning: a review. arXiv preprint arXiv:2010.10596
von Ost A, West N (2022) Ethnizität oder Klassismus? Von den Schwierigkeiten und Möglichkeiten, die Benachteiligung Ostdeutscher antidiskriminierungsrechtlich zu fassen. Feministische Visionen vor und nach, (1989) Geschlecht. Medien und Aktivismen in der DDR, BRD und im östlichen Europa 2022:315
Wahlström M, Törnberg A, Ekbrand H (2021) Dynamics of violent and dehumanizing rhetoric in far-right social media. New Media Soc 23(11):3290–3311
Wang R, Zhou D, He Y (2019) ATM: adversarial-neural topic model. Inf Process Manag 56(6):102098. https://doi.org/10.1016/j.ipm.2019.102098
Wang A, Narayanan A, Russakovsky O (2020a) REVISE: a tool for measuring and mitigating bias in visual datasets. In: European conference on computer vision. Springer, pp 733–751
Wang Y, Yang W, Ma F, Xu J, Zhong B, Deng Q, Gao J (2020b) Weak supervision for fake news detection via reinforcement learning. In: Proceedings of the AAAI conference on artificial intelligence 34, pp 516–523
Wang J, Liu Y, Wang XE (2021a) Assessing multilingual fairness in pre-trained multimodal representations. arXiv preprint arXiv:2106.06683
Wang J, Wu Z, Chen J, Jiang Y-G (2021b) M2tr: multi-modal multi-scale transformers for deepfake detection. arXiv preprint arXiv:2104.09770
Wang A, Ramaswamy VV, Russakovsky O (2022) Towards intersectionality in machine learning: including more identities, handling underrepresentation, and performing evaluation. arXiv:2205.04610
Wang X, Wei J, Schuurmans D, Le Q, Chi E, Narang S, Chowdhery A, Zhou D (2023) Self-consistency improves chain of thought reasoning in language models. arXiv:2203.11171 [cs.CL]
Webb T, Holyoak KJ, Lu H (2023) Emergent analogical reasoning in large language models. Nat Hum Behav 7(9):1526–1541
Wei J, Bosma M, Zhao VY, Guu K, Yu AW, Lester B, Du N, Dai AM, Le QV (2021) Finetuned language models are zero-shot learners. https://doi.org/10.48550/ARXIV.2109.01652
Wei J, Wang X, Schuurmans D, Bosma M, Xia F, Chi E, Le QV, Zhou D et al (2022) Chain-of-thought prompting elicits reasoning in large language models. Adv Neural Inf Process Syst 35:24824–24837
Weidinger L, Mellor J, Rauh M, Griffin C, Uesato J, Huang P-S, Cheng M, Glaese M, Balle B, Kasirzadeh A et al (2021) Ethical and social risks of harm from Language Models. arXiv preprint arXiv:2112.04359
Weidinger L, Uesato J, Rauh M, Griffin C, Huang P-S, Mellor J, Glaese A, Cheng M, Balle B, Kasirzadeh A et al (2022) Taxonomy of risks posed by language models. In: 2022 ACM conference on fairness, accountability, and transparency, pp 214–229
Weinberg L (2022) Rethinking fairness: an interdisciplinary survey of critiques of hegemonic ML fairness approaches. J Artif Intell Res 74:75–109. https://doi.org/10.1613/jair.1.13196
Weng Y, Zhu M, Xia F, Li B, He S, Liu S, Sun B, Liu K, Zhao J (2023) Large language models are better reasoners with self-verification. In: Findings of the Association for Computational Linguistics: EMNLP 2023, pp 2550–2575
Westerlund M (2019) The emergence of deepfake technology: a review. Technol Innov Manag Rev 9:11
Whittaker J, Looney S, Reed A, Votta F (2021) Recommender systems and the amplification of extremist content. Internet Policy Rev 10(2):1–29
Wimmer A (1997) Explaining xenophobia and racism: a critical review of current research approaches. Ethn Racial Stud 20(1):17–41. https://doi.org/10.1080/01419870.1997.9993946
World Health Organization et al (1946) Preamble to the constitution of WHO as adopted by the international health conference. New York 19:100
Xia M, Field A, Tsvetkov Y (2020) Demoting racial bias in hate speech detection. CoRRarXiv:2005.12246
Xu Y, Lv T, Cui L, Wang G, Lu Y, Florencio D, Zhang C, Wei F (2021) LayoutXLM: multimodal pre-training for multilingual visually-rich document understanding. arXiv preprint arXiv:2104.08836
Xu F, Lin Q, Han J, Zhao T, Liu J, Cambria E (2023) Are large language models really good logical reasoners? A comprehensive evaluation and beyond. arXiv:2306.09841 [cs.CL]
Yakushko O (2009) Xenophobia: understanding the roots and consequences of negative attitudes toward immigrants. Counsel Psychol 37(1):36–66. https://doi.org/10.1177/0011000008316034
Yakushko O (2018) Modern-day xenophobia: critical historical and theoretical perspectives on the roots of anti-immigrant prejudice. Springer, Berlin
Yamaguchi T (2013) Xenophobia in action: ultranationalism, hate speech, and the internet in Japan. Radic History Rev 117:98–118
Yao H, Chen Y, Ye Q, Jin X, Ren X (2021a) Refining language models with compositional explanations. Adv Neural Inf Process Syst 34(2021):8954–8967
Yao S, Halpern Y, Thain N, Wang X, Lee K, Prost F, Chi EH, Chen J, Beutel A (2021b) Measuring recommender system effects with simulated users. arXiv preprint arXiv:2101.04526
Yi SS, Kwon SC, Suss R, Đoàn LN, John I, Islam NS, Trinh-Shevrin C (2022) The mutually reinforcing cycle of poor data quality and racialized stereotypes that shapes Asian American health: study examines poor data quality and racialized stereotypes that shape Asian American health. Health Aff 41(2):296–303
Yuen NW (2016) Reel inequality: Hollywood actors and racism. Rutgers University Press
Zeng A, Attarian M, Ichter B, Choromanski K, Wong A, Welker S, Tombari F, Purohit A, Ryoo M, Sindhwani V, Lee J, Vanhoucke V, Florence P (2022) Socratic models: composing zero-shot multimodal reasoning with language. https://doi.org/10.48550/ARXIV.2204.00598
Zepeda-Millán C (2016) Weapons of the (not so) weak: immigrant mass mobilization in the US South. Crit Sociol 42(2):269–287
Zhang H, Huang J, Li Z, Naik M, Xing E (2023) Improved logical reasoning of language models via differentiable symbolic programming. arXiv:2305.03742 [cs.AI]
Zhao H, Zhou W, Chen D, Wei T, Zhang W, Yu N (2021) Multi-attentional deepfake detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2185–2194
Zhao H, Chen H, Yang F, Liu N, Deng H, Cai H, Wang S, Yin D, Du M (2023) Explainability for large language models: a survey. ACM Trans Intell Syst Technol
Zhou D, Schärli N, Hou L, Wei J, Scales N, Wang X, Schuurmans D, Cui C, Bousquet O, Le Q, et al (2022) Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625
Zihindula G, Meyer-Weitz A, Akintola O (2017) Lived experiences of Democratic Republic of Congo refugees facing medical xenophobia in Durban, South Africa. J Asian Afr Stud 52(4):458–470
Zmigrod R, Mielke SJ, Wallach H, Cotterell R (2019) Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology. arXiv preprint arXiv:1906.04571
Zuiderveen Borgesius F, Trilling D, Moller J, Bodo B, De Vreese CH, Helberger N (2016) Should we worry about filter bubbles? Internet Policy Rev J Internet Regul 5:1
Acknowledgements
We would like to thank Stevie Bergman, William Isaac and Shakir Mohamed for the support and insightful feedback that they provided for this paper. Any opinions presented in this paper represent the personal views of the authors and do not necessarily reflect the official policies or positions of their organisations.
Curmudgeon Corner
Curmudgeon Corner is a short opinionated column on trends in technology, arts, science and society, commenting on issues of concern to the research community and wider society. Whilst the drive for super-human intelligence promotes potential benefits to wider society, it also raises deep concerns of existential risk, thereby highlighting the need for an ongoing conversation between technology and society. At the core of Curmudgeon concern is the question: What is it to be human in the age of the AI machine? -Editor.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no competing interests to declare.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Tomasev, N., Maynard, J.L. & Gabriel, I. Manifestations of xenophobia in AI systems. AI & Soc (2024). https://doi.org/10.1007/s00146-024-01893-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00146-024-01893-4