publications

2025

1. Nullius in Explanans: an Ethical Risk Assessment for Explainable AI

   Nannini, L., Huyskes, D., Panai, E., Pistilli, G., and Tartaro, A.

   Ethics and Information Technology Jan 2025

   Abs PDF

   Explanations are conceived to ensure the trustworthiness of AI systems. Yet, relying solemnly on algorithmic solutions, as provided by explainable artificial intelligence (XAI), might fall short to account for sociotechnical risks jeopardizing their factuality and informativeness. To mitigate these risks, we delve into the complex landscape of ethical risks surrounding XAI systems and their generated explanations. By employing a literature review combined with rigorous thematic analysis, we uncover a diverse array of technical risks tied to the robustness, fairness, and evaluation of XAI systems. Furthermore, we address a broader range of contextual risks jeopardizing their security, accountability, reception alongside other cognitive, social, and ethical concerns of explanations. We advance a multi-layered risk assessment framework, where each layer advances strategies for practical intervention, management, and documentation of XAI systems within organizations. Recognizing the theoretical nature of the framework advanced, we discuss it in a conceptual case study. For the XAI community, our multifaceted investigation represents a path to practically address XAI risks while enriching our understanding of the ethical ramifications of incorporating XAI in decision-making processes.

2024

1. Introducing ELLIPS: An Ethics-Centered Approach to Research on LLM-Based Inference of Psychiatric Conditions

   Rocca, R., Pistilli, G., Maheshwari, K., and Fusaroli, R.

   Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society Oct 2024

   Abs PDF

   As mental health care systems worldwide struggle to meet demand, there is increasing focus on using language models (LM) to infer neuropsychiatric conditions or psychopathological traits from language production. Yet, so far, this research has only delivered solutions with limited clinical applicability, due to insufficient consideration of ethical questions crucial to ensuring the synergy between possible applications and model design. To accelerate progress towards clinically applicable models, our paper charts the ethical landscape of research on language-based inference of psychopathology and provides a practical tool for researchers to navigate it. We identify seven core ethical principles that should guide model development and deployment in this domain, translate them into ELLIPS, an ethical toolkit operationalizing these principles into questions that can guide researchers’ choices with respect to data selection, architectures, evaluation, and model deployment, and provide a case study exemplifying its use. With this, we aim to facilitate the emergence of model technology with concrete potential for real-world applicability.

2. CIVICS: Building a Dataset for Examining Culturally-Informed Values in Large Language Models

   Pistilli, G., Leidinger, A., Jernite, Y., Kasirzadeh, A., A., S. Luccioni, and Mitchell, M.

   Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society Oct 2024

   Abs PDF

   This paper introduces the "CIVICS: Culturally-Informed & Values-Inclusive Corpus for Societal impacts" dataset, designed to evaluate the social and cultural variation of Large Language Models (LLMs) across multiple languages and value-sensitive topics. We create a hand-crafted, multilingual dataset of value-laden prompts which address specific socially sensitive topics, including LGBTQI rights, social welfare, immigration, disability rights, and surrogacy. CIVICS is designed to generate responses showing LLMs’ encoded and implicit values. Through our dynamic annotation processes, tailored prompt design, and experiments, we investigate how open-weight LLMs respond to value-sensitive issues, exploring their behavior across diverse linguistic and cultural contexts. Using two experimental set-ups based on log-probabilities and long-form responses, we show social and cultural variability across different LLMs. Specifically, experiments involving long-form responses demonstrate that refusals are triggered disparately across models, but consistently and more frequently in English or translated statements. Moreover, specific topics and sources lead to more pronounced differences across model answers, particularly on immigration, LGBTQI rights, and social welfare. As shown by our experiments, the CIVICS dataset aims to serve as a tool for future research, promoting reproducibility and transparency across broader linguistic settings, and furthering the development of AI technologies that respect and reflect global cultural diversities and value pluralism. The CIVICS dataset and tools will be made available upon publication under open licenses.

3. Implications of Regulations on the Use of AI and Generative AI for Human-Centered Responsible Artificial Intelligence

   Constantinides, M., Tahaei, M., Quercia, D., Stumpf, S., Madaio, M., Kennedy, S., Wilcox, L., Vitak, J., Cramer, H., Bogucka, E. P., Baeza-Yates, R., Luger, E., Holbrook, J., Muller, M., Blumenfeld, I. G., and Pistilli, G.

   Extended Abstracts of the CHI Conference on Human Factors in Computing Systems - Association for Computing Machinery May 2024

   Abs PDF

   With the upcoming AI regulations (e.g., EU AI Act) and rapid advancements in generative AI, new challenges emerge in the area of Human-Centered Responsible Artificial Intelligence (HCR-AI). As AI becomes more ubiquitous, questions around decision-making authority, human oversight, accountability, sustainability, and the ethical and legal responsibilities of AI and their creators become paramount. Addressing these questions requires a collaborative approach. By involving stakeholders from various disciplines in the 2nd edition of the HCR-AI Special Interest Group (SIG) at CHI 2024, we aim to discuss the implications of regulations in HCI research, develop new theories, evaluation frameworks, and methods to navigate the complex nature of AI ethics, steering AI development in a direction that is beneficial and sustainable for all of humanity.

4. For an ethics of conversational Artificial Intelligence

   Pistilli, G.

   PhD Thesis at Sorbonne Université Mar 2024

   Abs PDF

   This research aims to probe the ethical intricacies of conversational Artificial Intelligence (AI), specifically focusing on Large Language Models and conversational agents. This manuscript constructs a framework that melds empirical analysis with philosophical discourse. We aim to urgently advocate for a well-founded ethical structure for conversational AI, highlighting the necessity to involve all stakeholders, from developers to end-users. Firstly, we champion the integration of engineering and other scientific disciplines with philosophy, facilitating a more nuanced understanding of the ethical dimensions underpinning AI. This collaborative approach allows for a richer, more informed ethical discourse. Secondly, we advocate for the dynamic use of applied ethical frameworks as foundational guides for setting the initial objectives of an AI system. These frameworks serve as evolving tools that adapt to the ethical complexities encountered during development and deployment. Lastly, grounded in hands-on, interdisciplinary research, we make an argument for the prioritization of narrow, task-specific AI over Artificial General Intelligence, a stance that is based on the enhanced feasibility of ethical oversight and technical controllability.With this research, we aim to contribute to the literature on AI ethics, enriching the academic discourse in both philosophy and computer science

5. The Moral Landscape of General-Purpose Large Language Models

   Pistilli, G.

   Human-Centered AI: A Multidisciplinary Perspective for Policy-Makers, Auditors, and Users Mar 2024

   Abs

   The confusion around the term “Artificial General Intelligence” (AGI), often trapped and disputed between the marketing and research fields, deserves to be defined and analyzed from an ethical perspective. In 1980, American philosopher John Searle published an article in which he argued against what was then called “strong AI.” Following the legacy of Alan Turing, the question Searle posed was: “Is a machine capable of thinking?” (Searle, 1980). To briefly summarize the experiment, the philosopher illustrated a thought experiment known today as “the Chinese room” to attempt to answer his question. The thought experiment consists of imagining a room in which Artificial Intelligence (AI) has at its disposal a set of documents (knowledge base) with Chinese sentences in it. A native Chinese speaker enters the room and begins to converse with this AI; the latter can answer, considering it can easily find which sentence corresponds to the questions asked. The American philosopher’s argument is simple: although AI can provide answers in Chinese, it has no background knowledge of the language. In other words, the syntax is not a sufficient condition for the determination of semantics.

2023

1. Debating AI in archaeology: applications, implications, and ethical considerations

   Tenzer, M., Pistilli, G., Brandsen, A., and Shenfield, A.

   SocArXiv preprint May 2023

   Abs PDF

   “Artificial Intelligence” (AI) is not a recent development. However, with increasing computational capabilities, AI has developed into Natural Language Processing and Machine Learning, technologies particularly good at detecting correlations and patterns, and categorising, predicting, or extracting information. Within archaeology, AI can process “big data” accumulated over decades of research and deposited in archives. By combining these capabilities, AI offers new insights and exciting opportunities to create knowledge from archaeological archives for contemporary and future research. However, ethical implications and human costs are not yet fully understood. Therefore, we question whether AI in archaeology is a blessing or a curse?

2. Stronger Together: On the Articulation of Ethical Charters, Legal Tools, and Technical Documentation in ML

   Pistilli, G., Muñoz Ferrandis, C., Jernite, Y., and Mitchell, M.

   In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency May 2023

   Abs PDF

   The growing need for accountability of the people behind AI systems can be addressed by leveraging processes in three fields of study: ethics, law, and computer science. While these fields are often considered in isolation, they rely on complementary notions in their interpretation and implementation. In this work, we detail this interdependence and motivate the necessary role of collaborative governance tools in shaping a positive evolution of AI. We first contrast notions of compliance in the ethical, legal, and technical fields; we outline both their differences and where they complement each other, with a particular focus on the roles of ethical charters, licenses, and technical documentation in these interactions. We then focus on the role of values in articulating the synergies between the fields and outline specific mechanisms of interaction between them in practice. We identify how these mechanisms have played out in several open governance fora: an open collaborative workshop, a responsible licensing initiative, and a proposed regulatory framework. By leveraging complementary notions of compliance in these three domains, we can create a more comprehensive framework for governing AI systems that jointly takes into account their technical capabilities, their impact on society, and how technical specifications can inform relevant regulations. Our analysis thus underlines the necessity of joint consideration of the ethical, legal, and technical in AI ethics frameworks to be used on a larger scale to govern AI systems and how the thinking in each of these areas can inform the others.

2022

1. BigScience: A Case Study in the Social Construction of a Multilingual Large Language Model

   Akiki, C., Pistilli, G., Mieskes, M., Gallé, M., Wolf, T., Ilic, S., and Jernite, Y.

   Proceedings in NeurIPS 2022 Workshop WBRC Nov 2022

   Abs PDF

   The BigScience Workshop was a value-driven initiative that spanned one and half years of interdisciplinary research and culminated in the creation of ROOTS, a 1.6TB multilingual dataset that was used to train BLOOM, one of the largest multilingual language models to date. In addition to the technical outcomes and artifacts, the workshop fostered multidisciplinary collaborations around large models, datasets, and their analysis. This in turn led to a wide range of research publications spanning topics from ethics to law, data governance, modeling choices and distributed training. This paper focuses on the collaborative research aspects of BigScience and takes a step back to look at the challenges of large-scale participatory research, with respect to participant diversity and the tasks required to successfully carry out such a project. Our main goal is to share the lessons we learned from this experience, what we could have done better and what we did well. We show how the impact of such a social approach to scientific research goes well beyond the technical artifacts that were the basis of its inception.

2. BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

   Scao, T. Le, Fan, A., Akiki, C., Pavlick, E.J., Ilic, S., Hesslow, D., Castagne, R., Luccioni, A.S., Yvon, F., Galle, M., Tow, J., Rush, A. M., Biderman, S.R., Webson, A., Ammanamanchi, P. S., Wang, T., Sagot, B., Muennighoff, N., Moral, A. Villanova, Ruwase, O., Bawden, R., Bekman, S., McMillan-Major, A., Beltagy, Iz, Nguyen, H., Saulnier, L., Tan, S., Suarez, P. Ortiz, Sanh, V., Laurenccon, H., Jernite, Y., Launay, J., Mitchell, M., Raffel, C., Gokaslan, A., Simhi, A., Etxabe, A. S., Aji, A. Fikri, Alfassy, A., Rogers, A., Nitzav, A. Kreisberg, Xu, C., Mou, C., Emezue, C. C., Klamm, C., Leong, C., Strien, D. A., Adelani, D. Ifeoluwa, Radev, D., Ponferrada, E. G., Levkovizh, E., Kim, E., Natan, E. B., Toni, F. De, Dupont, G., Kruszewski, G., and G. Pistilli, et al.

   arXiv preprint Nov 2022

   Abs PDF

   Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

3. The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset

   Laurencon, H., Saulnier, L., Wang, T., Akiki, C., Moral, A. Villanova, Scao, T. Le, Werra, L., Mou, C., Ponferrada, E. Gonzalez, Nguyen, H., Frohberg, J., Sasko, M., Lhoest, Q., McMillan-Major, A., Dupont, G., Biderman, S., Rogers, A., allal, L. Ben, Toni, F. De, and G. Pistilli, et al.

   Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track Nov 2022

   Abs PDF

   As language models grow ever larger, the need for large-scale high-quality text datasets has never been more pressing, especially in multilingual settings. The BigScience workshop, a 1-year international and multidisciplinary initiative, was formed with the goal of researching and training large language models as a values-driven undertaking, putting issues of ethics, harm, and governance in the foreground. This paper documents the data creation and curation efforts undertaken by BigScience to assemble the Responsible Open-science Open-collaboration Text Sources (ROOTS) corpus, a 1.6TB dataset spanning 59 languages that was used to train the 176-billion-parameter BigScience Large Open-science Open-access Multilingual (BLOOM) language model. We further release a large initial subset of the corpus and analyses thereof, and hope to empower large-scale monolingual and multilingual modeling projects with both the data and the processing tools, as well as stimulate research around this large multilingual corpus.

4. The Ghost in the Machine has an American accent: value conflict in GPT-3

   R., L. Johnson, Pistilli, G., Menédez-González, N., L., D. Dias Duran, Panai, E., Kalpokiene, J., and D., J. Bertulfo

   arXiv preprint Mar 2022

   Abs PDF

   The alignment problem in the context of large language models must consider the plurality of human values in our world. Whilst there are many resonant and overlapping values amongst the world’s cultures, there are also many conflicting, yet equally valid, values. It is important to observe which cultural values a model exhibits, particularly when there is a value conflict between input prompts and generated outputs. We discuss how the co-creation of language and cultural value impacts large language models (LLMs). We explore the constitution of the training data for GPT-3 and compare that to the world’s language and internet access demographics, as well as to reported statistical profiles of dominant values in some Nation-states. We stress tested GPT-3 with a range of value-rich texts representing several languages and nations; including some with values orthogonal to dominant US public opinion as reported by the World Values Survey. We observed when values embedded in the input text were mutated in the generated outputs and noted when these conflicting values were more aligned with reported dominant US values. Our discussion of these results uses a moral value pluralism (MVP) lens to better understand these value mutations. Finally, we provide recommendations for how our work may contribute to other current work in the field.

5. What lies behind AGI: ethical concerns related to LLMs

   Pistilli, G.

   Éthique et Numérique Mar 2022

   Abs PDF

   This paper opens the philosophical debate around the notion of Artificial General Intelligence (AGI) and its application in Large Language Models (LLMs). Through the lens of moral philosophy, the paper raises questions about these AI systems’ capabilities and goals, the treatment of humans behind them, and the risk of perpetuating a monoculture through language.

6. La logique algorithmique confrontée à l’organisation de l’administration publique française

   Pistilli, G.

   Giornale di Filosofia Mar 2022

   Abs PDF

   Cet article montre comment la logique algorithmique d’un agent conversationnel peut aider l’organisation des connaissances au sein d’une organisation de l’administration publique française, notamment une collectivité territoriale. Par le bias d’une recherche sur le terrain, je cherche à montrer comment il existe deux différentes adoptions de la technologie de la part de l’administration publique : une complexifiante et une simplifiante.

7. Journal issue, vol. 2: Technology and Constructive Critical Thought

   Pistilli, G., and Puech, M.

   Giornale di Filosofia Mar 2022

   Abs PDF

   Does critical thinking have anything to say to technology? Something constructive? We asked ourselves this question, based on our mixed experience of research in philosophy and work in business. We tried to generate articles and start a conversation, through the association Filosofia in Movimento, on the theme "technology and constructive critical thinking". This issue collects the articles that came out of this process. We suggested a list of themes: why are technophobic intellectuals dominant? How to build the societal acceptability of new technologies? Is the Internet a tool for critical thinking? Can artificial intelligences function as mediators of a critical conversation? What is the middle way between technophile utopia and technophobe dystopia? What does science fiction teach us as a "sandbox" for critical thinking about technology? Should we politicize or depoliticize technology? Some of these questions have been answered directly, others indirectly, and some have not been answered at all, which could be significant as well.