publications

2025

1. arXiv

   Designing Rules to Pick a Rule: Aggregation by Consistency

   Ratip Emin Berker , Ben Armstrong, Vincent Conitzer , and 1 more author

   Aug 2025

   Preprint.

   Abs arXiv PDF

   Given a set of items and a set of evaluators who all individually rank them, how do we aggregate these evaluations into a single societal ranking? Work in social choice and statistics has produced many aggregation methods for this problem, each with its desirable properties, but also with its limitations. Further, existing impossibility results rule out designing a single method that achieves every property of interest. Faced with this trade-off between incompatible desiderata, how do we decide which aggregation rule to use, i.e., what is a good rule picking rule? In this paper, we formally address this question by introducing a novel framework for rule picking rules (RPRs). We then design a data-driven RPR that identifies the best aggregation method for each specific setting, without assuming any generative model. The principle behind our RPR is to pick the rule which maximizes the consistency of the output ranking if the data collection process were repeated. We introduce several consistency-related axioms for RPRs and show that our method satisfies them, including those failed by a wide class of natural RPRs. While we prove that the algorithmic problem of maximizing consistency is computationally hard, we provide a sampling-based implementation of our RPR that is efficient in practice. We run this implementation on known statistical models and find that, when possible, our method selects the maximum likelihood estimator of the data. Finally, we show that our RPR can be used in many real-world settings to gain insights about how the rule currently being used can be modified or replaced to substantially improve the consistency of the process. Taken together, our work bridges an important gap between the axiomatic and statistical approaches to rank aggregation, laying a robust theoretical and computational foundation for principled rule picking.

2. arXiv

   What Voting Rules Actually Do: A Data-Driven Analysis of Multi-Winner Voting

   Joshua Caiata , Ben Armstrong, and Kate Larson

   Aug 2025

   Presented at Game, Agents, Incentives Workshop held as part of AAMAS 2025

   Abs arXiv PDF

   Committee-selection problems arise in many contexts and applications, and there has been increasing interest within the social choice research community on identifying which properties are satisfied by different multi-winner voting rules. In this work, we propose a data-driven framework to evaluate how frequently voting rules violate axioms across diverse preference distributions in practice, shifting away from the binary perspective of axiom satisfaction given by worst-case analysis. Using this framework, we analyze the relationship between multi-winner voting rules and their axiomatic performance under several preference distributions. We then show that neural networks, acting as voting rules, can outperform traditional rules in minimizing axiom violations. Our results suggest that data-driven approaches to social choice can inform the design of new voting systems and support the continuation of data-driven research in social choice.

3. AIES

   Investigating Political and Demographic Associations in Large Language Models Through Moral Foundations Theory

   Nicole Smith-Vaniz , Harper Lyon , Lorraine Steigner , and 2 more authors

   Sep 2025

   To appear in the Proceedings of the Eighth Conference on Artificial Intelligence, Ethics, and Society 2025

   Abs

   Large Language Models (LLMs) have become increasingly incorporated into everyday life for many internet users, taking on significant roles as advice givers in the domains of medicine, personal relationships, and even legal matters. The importance of these roles raise questions about how and what responses LLMs make in difficult political and moral domains, especially questions about possible biases. To quantify the nature of potential biases in LLMs, various works have applied Moral Foundations Theory (MFT), a framework that categorizes human moral reasoning into five dimensions: Harm, Fairness, Ingroup Loyalty, Authority, and Purity. Previous research has used the MFT to measure differences in human participants along political, national, and cultural lines. While there has been some analysis of the responses of LLM with respect to political stance in role-playing scenarios, no work so far has directly assessed the moral leanings in the LLM responses, nor have they connected LLM outputs with robust human data. In this paper we analyze the distinctions between LLM MFT responses and existing human research directly, investigating whether commonly available LLM responses demonstrate ideological leanings — either through their inherent responses, straightforward representations of political ideologies, or when responding from the perspectives of constructed human personas. We assess whether LLMs inherently generate responses that align more closely with one political ideology over another, and additionally examine how accurately LLMs can represent ideological perspectives through both explicit prompting and demographic-based role-playing. By systematically analyzing LLM behavior across these conditions and experiments, our study provides insight into the extent of political and demographic dependency in AI-generated responses.

2024

1. AAMAS

   Liquid Democracy for Low-Cost Ensemble Pruning

   Ben Armstrong, and Kate Larson

   May 2024

   Extended Abstract. Full version available on ArXiV.

   Abs arXiv PDF

   We argue that there is a strong connection between ensemble learning and a delegative voting paradigm - liquid democracy – that can be leveraged to reduce ensemble training costs. We present an incremental training procedure that identifies and removes redundant classifiers from an ensemble via delegation mechanisms inspired by liquid democracy. Through both analysis and extensive experiments we show that this process greatly reduces the computational cost of training compared to training a full ensemble. By carefully selecting the underlying delegation mechanism, weight centralization in the classifier population is avoided, leading to higher accuracy than some boosting methods. Furthermore, this work serves as an exemplar of how frameworks from computational social choice literature can be applied to problems in nontraditional domains.

2. GAIW

   Optimizing Viscous Democracy

   Ben Armstrong, Shiri Alouf-Heffetz , and Nimrod Talmon

   Games, Agents, Incentives Workshop at the 23rd International Conference on Autonomous Agents and Multi-Agent Systems, May 2024

   Abs PDF

   Viscous democracy is a generalization of liquid democracy, a social choice framework in which voters may transitively delegate their votes. In viscous democracy, a "viscosity" factor decreases the weight of a delegation the further it travels, reducing the chance of excessive weight flowing between ideologically misaligned voters. We demonstrate that viscous democracy often significantly improves the quality of group decision-making over liquid democracy. We first show that finding optimal delegations within a viscous setting is NP-hard. However, simulations allow us to explore the practical effects of viscosity. Across social network structures, competence distributions, and delegation mechanisms we find high viscosity reduces the chance of “super-voters” attaining large amounts of weight and increases the number of voters that are able to affect the outcome of elections. This, in turn, improves group accuracy as a whole. As a result, we argue that viscosity should be considered a core component of liquid democracy.

3. IJCAI

   Optimizing Viscous Democracy

   Ben Armstrong, Shiri Alouf-Heffetz , and Nimrod Talmon

   33rd International Joint Conference on Artificial Intelligence, Aug 2024

   Abs PDF

   Viscous democracy is a generalization of liquid democracy, a social choice framework in which voters may transitively delegate their votes. In viscous democracy, a "viscosity" factor decreases the weight of a delegation the further it travels, reducing the chance of excessive weight flowing between ideologically misaligned voters. We demonstrate that viscous democracy often significantly improves the quality of group decision-making over liquid democracy. We first show that finding optimal delegations within a viscous setting is NP-hard. However, simulations allow us to explore the practical effects of viscosity. Across social network structures, competence distributions, and delegation mechanisms we find high viscosity reduces the chance of “super-voters” attaining large amounts of weight and increases the number of voters that are able to affect the outcome of elections. This, in turn, improves group accuracy as a whole. As a result, we argue that viscosity should be considered a core component of liquid democracy.

4. Canadian AI

   Liquid Ensemble Selection for Continual Learning

   Carter Blair , Ben Armstrong, and Kate Larson

   37th Canadian Conference on Artificial Intelligence, May 2024

   Note: SCaLA version contains more experimental results due to length limits.

   Abs PDF

   Continual learning aims to enable machine learning models to acquire new knowl- edge from a shifting data distribution without forgetting what has already been learned. Such shifting distributions can be broken into disjoint subsets of related examples called contexts; training each ensemble member on a different context makes it possible for the ensemble as a whole to achieve much higher accuracy with less forgetting than a naive model. We address the problem of selecting which models within an ensemble should learn on any given data and which should predict. By drawing on work from delegative voting we develop an algorithm for using delegation to dynamically select which mod- els in an ensemble are active. We explore various delegation methods and performance metrics, ultimately finding that delegation can provide a significant performance boost over naive learning in the face of distribution shifts.

5. SCaLA

   Liquid Ensemble Selection for Continual Learning

   Carter Blair , Ben Armstrong, and Kate Larson

   Social Choice and Learning Algorithms Workshop at the 23rd International Conference on Autonomous Agents and Multi-Agent Systems, May 2024

   Abs PDF

   Continual learning aims to enable machine learning models to acquire new knowl- edge from a shifting data distribution without forgetting what has already been learned. Such shifting distributions can be broken into disjoint subsets of related examples called contexts; training each ensemble member on a different context makes it possible for the ensemble as a whole to achieve much higher accuracy with less forgetting than a naive model. We address the problem of selecting which models within an ensemble should learn on any given data and which should predict. By drawing on work from delegative voting we develop an algorithm for using delegation to dynamically select which mod- els in an ensemble are active. We explore various delegation methods and performance metrics, ultimately finding that delegation can provide a significant performance boost over naive learning in the face of distribution shifts.

2022

1. IJCAI

   How Should We Vote? A Comparison of Voting Systems within Social Networks

   Shiri Alouf-Heffetz , Ben Armstrong, Kate Larson , and 1 more author

   In Proceedings of the 31st International Joint Conference on Artificial Intelligence , Jul 2022

   Abs PDF

   Voting is a central methodology for eliciting and combining agents’ preferences and information across many applications. Just as there are numerous voting rules exhibiting different properties, we also see many different voting systems. In this paper we investigate how different voting systems perform as a function of the characteristics of the underlying voting population and social network. In particular, we compare direct democracy, liquid democracy, and sortition in a ground truth voting context. Through simulations – using both real and artificially generated social networks – we illustrate how voter competency distributions and levels of direct participation affect group accuracy differently in each voting mechanism. Our results can be used to guide the selection of a suitable voting system based on the characteristics of a particular voting setting.

2021

1. AAMAS-DC

   Exploring the Relationship Between Social Choice and Machine Learning

   Ben Armstrong

   20th International Conference on Autonomous Agents and Multi-Agent Systems (Doctoral Consortium), May 2021

   Abs PDF

   My thesis will study the intersection of social choice and machine learning, with a focus on recent or under-explored social choice paradigms, such as liquid democracy, and how social choice and ML can benefit each other. My initial results show the idea of using ML and social choice to understand the other holds promise. An early project of mine uses deep learning to enhance social choice by creating a neural network that acts as a voting rule, able to be trained to select a winner satisfying customizable sets of axioms. More recently, I have explored the idea of using liquid democracy as a framework for ensembles for classification problems. I am particularly interested in improving the real-world applicability of existing social choice methods and understanding how they can more beneficially impact the world. Going forward, my primary tools in these goals are simulation and provable axiomatic or performance guarantees.

2. GAIW

   On the Limited Applicability of Liquid Democracy

   Ben Armstrong, and Kate Larson

   Games, Agents, Incentives Workshop at the 20th International Conference on Autonomous Agents and Multi-Agent Systems, May 2021

   Abs PDF

   Liquid democracy is a proxy-voting method in which voters may transitively delegate their votes, allowing others to vote in their place. It is often analysed and used in settings where there is some single, objectively correct outcome that voters collectively aim to select. As a result, much of the previous work on liquid democracy has proposed agent utility functions which depend directly on the accuracy of the delegatee voters. We argue, instead, that a more realistic model would have voters’ incentives more closely aligned with the final outcome, rather than the proxy vote cast. We explore this alternative and show that there exists a pure-strategy Nash equilibirum that can be reached by best-response delegations. Furthermore, allowing for vote-delegation always weakly improves the accuracy of the decision-making process compared to having all agents vote directly. We apply this model to a classifier ensemble problem. While our theoretical findings are positive, our empirical results show that the assumption of independence between voters required for theoretical analysis is critical. Once removed, liquid democracy fails to materialize any practical improvement over direct voting.

2019

1. Could AI Drive Transformative Social Progress? What Would This Require?

   Edward (Ted) A. Parson , Robert Lempert , Ben Armstrong, and 3 more authors

   SSRN Electronic Journal, May 2019

   PDF
2. Siri Humphrey: Design Principles for an AI Policy Analyst

   Ben Armstrong, Megan Beretta , Evan Crothers , and 5 more authors

   May 2019

   Abs PDF

   This workgroup considered whether the policy analysis function in government could be replaced by an artificial intelligence policy analyst (AIPA) that responds directly to requests for information and decision support from political and administrative leaders. We describe the current model for policy analysis, identify the design criteria for an AIPA, and consider its limitations should it be adopted. A core limitation is the essential human interaction between a decision maker and an analyst/advisor, which extends the meaning and purpose of policy analysis beyond a simple synthesis or technical analysis view (each of which is nonetheless a complex task in its own right). Rather than propose a wholesale replacement of policy analysts with AIPA, we reframe the question focussing on the use of AI by human policy analysts for augmenting their current work, what we term intelligenceamplified policy analysis (IAPA). We conclude by considering how policy analysts, schools of public affairs, and institutions of government will need to adapt to the changing nature of policy analysis in an era of increasingly capable AI.

3. AI4SG

   Machine Learning to Strengthen Democracy

   Ben Armstrong, and Kate Larson

   NeurIPS Joint Workshop on AI for Social Good, May 2019

   Abs PDF

   Democratic institutions making use of plurality voting have, in recent years, been shown to be susceptible to interference and to poorly reflect the interests of voters. While many agree that better systems exist, there is no clear consensus on how the perfect election system functions. This paper describes a machine learning approach to create a voting rule aimed at satisfying highly customisable sets of ideal electoral fairness criteria. We first provide a description of our framework, then show in a simple example that it is able to achieve a strong result. Our system aims to provide room for opposing groups to agree on the principles they believe an election should reflect, rather than arguing over pre-existing preferences for particular reforms.

2018

1. Coordination in a Peer Production Platform: A study of Reddit’s /r/Place experiment

   Ben Armstrong

   Masters Thesis, May 2018

   Abs PDF

   Understanding the factors causing groups to engage in coordinating behaviour has been an active research area for decades. In this thesis, we study this problem using a novel dataset of crowd behaviour from an online experiment hosted by Reddit. This experiment allowed users to attempt to build an image alone, or to work collaboratively in the hope of building something greater. We use data provided by Reddit, in addition to crowdsourced coordination information, in order to compare this experiment with a platform containing many similarities to our experiment: Wikipedia. Comparison with Wikipedia shows that many behavioural trends appear to generalize across domains. We go on to construct an agent-based model of the experiment, allowing investigation into the effects of spontaneous and planned coordination. We find that while coordinated work leads to significant productivity improvements in concentrated areas, there is little effect on the experiment as a whole as a result of coordination.

2017

1. Approval in the Echo Chamber

   Ben Armstrong, and Kate Larson

   EXPLORE Workshop at the 16th International Conference on Autonomous Agents and Multi-Agent Systems, May 2017

   Abs PDF

   Recently there has been interest in iterative voting, where voters are able to update their votes based on voting information from previous rounds. In this paper we conduct a series of empirical studies in order to understand the strategic issues which arise when agents, voting to approve a set of k candidates, can base their voting or approval decisions on information from their neighbours in a social network. We illustrate that the k-approval voting rule often results in cyclic voting behaviour, that social network structure matters in terms of strategization, and that homophily in the network decreases strategization for the k-approval voting rule.