publications

2024

1. AAMAS

   Liquid Democracy for Low-Cost Ensemble Pruning

   Ben Armstrong, and Kate Larson

   May 2024

   Extended Abstract. Full version available on ArXiV.

   Abs arXiv PDF

   We argue that there is a strong connection between ensemble learning and a delegative voting paradigm - liquid democracy – that can be leveraged to reduce ensemble training costs. We present an incremental training procedure that identifies and removes redundant classifiers from an ensemble via delegation mechanisms inspired by liquid democracy. Through both analysis and extensive experiments we show that this process greatly reduces the computational cost of training compared to training a full ensemble. By carefully selecting the underlying delegation mechanism, weight centralization in the classifier population is avoided, leading to higher accuracy than some boosting methods. Furthermore, this work serves as an exemplar of how frameworks from computational social choice literature can be applied to problems in nontraditional domains.

2. GAIW

   Optimizing Viscous Democracy

   Ben Armstrong, Shiri Alouf-Heffetz , and Nimrod Talmon

   Games, Agents, Incentives Workshop at the 23rd International Conference on Autonomous Agents and Multi-Agent Systems, May 2024

   Abs PDF

   Viscous democracy is a generalization of liquid democracy, a social choice framework in which voters may transitively delegate their votes. In viscous democracy, a "viscosity" factor decreases the weight of a delegation the further it travels, reducing the chance of excessive weight flowing between ideologically misaligned voters. We demonstrate that viscous democracy often significantly improves the quality of group decision-making over liquid democracy. We first show that finding optimal delegations within a viscous setting is NP-hard. However, simulations allow us to explore the practical effects of viscosity. Across social network structures, competence distributions, and delegation mechanisms we find high viscosity reduces the chance of “super-voters” attaining large amounts of weight and increases the number of voters that are able to affect the outcome of elections. This, in turn, improves group accuracy as a whole. As a result, we argue that viscosity should be considered a core component of liquid democracy.

3. IJCAI

   Optimizing Viscous Democracy

   Ben Armstrong, Shiri Alouf-Heffetz , and Nimrod Talmon

   33rd International Joint Conference on Artificial Intelligence, Aug 2024

   Abs PDF

   Viscous democracy is a generalization of liquid democracy, a social choice framework in which voters may transitively delegate their votes. In viscous democracy, a "viscosity" factor decreases the weight of a delegation the further it travels, reducing the chance of excessive weight flowing between ideologically misaligned voters. We demonstrate that viscous democracy often significantly improves the quality of group decision-making over liquid democracy. We first show that finding optimal delegations within a viscous setting is NP-hard. However, simulations allow us to explore the practical effects of viscosity. Across social network structures, competence distributions, and delegation mechanisms we find high viscosity reduces the chance of “super-voters” attaining large amounts of weight and increases the number of voters that are able to affect the outcome of elections. This, in turn, improves group accuracy as a whole. As a result, we argue that viscosity should be considered a core component of liquid democracy.

4. Canadian AI

   Liquid Ensemble Selection for Continual Learning

   Carter Blair , Ben Armstrong, and Kate Larson

   37th Canadian Conference on Artificial Intelligence, May 2024

   Note: SCaLA version contains more experimental results due to length limits.

   Abs PDF

   Continual learning aims to enable machine learning models to acquire new knowl- edge from a shifting data distribution without forgetting what has already been learned. Such shifting distributions can be broken into disjoint subsets of related examples called contexts; training each ensemble member on a different context makes it possible for the ensemble as a whole to achieve much higher accuracy with less forgetting than a naive model. We address the problem of selecting which models within an ensemble should learn on any given data and which should predict. By drawing on work from delegative voting we develop an algorithm for using delegation to dynamically select which mod- els in an ensemble are active. We explore various delegation methods and performance metrics, ultimately finding that delegation can provide a significant performance boost over naive learning in the face of distribution shifts.

5. SCaLA

   Liquid Ensemble Selection for Continual Learning

   Carter Blair , Ben Armstrong, and Kate Larson

   Social Choice and Learning Algorithms Workshop at the 23rd International Conference on Autonomous Agents and Multi-Agent Systems, May 2024

   Abs PDF

   Continual learning aims to enable machine learning models to acquire new knowl- edge from a shifting data distribution without forgetting what has already been learned. Such shifting distributions can be broken into disjoint subsets of related examples called contexts; training each ensemble member on a different context makes it possible for the ensemble as a whole to achieve much higher accuracy with less forgetting than a naive model. We address the problem of selecting which models within an ensemble should learn on any given data and which should predict. By drawing on work from delegative voting we develop an algorithm for using delegation to dynamically select which mod- els in an ensemble are active. We explore various delegation methods and performance metrics, ultimately finding that delegation can provide a significant performance boost over naive learning in the face of distribution shifts.

2022

1. IJCAI

   How Should We Vote? A Comparison of Voting Systems within Social Networks

   Shiri Alouf-Heffetz , Ben Armstrong, Kate Larson , and 1 more author

   In Proceedings of the 31st International Joint Conference on Artificial Intelligence , Jul 2022

   Abs PDF

   Voting is a central methodology for eliciting and combining agents’ preferences and information across many applications. Just as there are numerous voting rules exhibiting different properties, we also see many different voting systems. In this paper we investigate how different voting systems perform as a function of the characteristics of the underlying voting population and social network. In particular, we compare direct democracy, liquid democracy, and sortition in a ground truth voting context. Through simulations – using both real and artificially generated social networks – we illustrate how voter competency distributions and levels of direct participation affect group accuracy differently in each voting mechanism. Our results can be used to guide the selection of a suitable voting system based on the characteristics of a particular voting setting.

2021

1. AAMAS-DC

   Exploring the Relationship Between Social Choice and Machine Learning

   Ben Armstrong

   20th International Conference on Autonomous Agents and Multi-Agent Systems (Doctoral Consortium), May 2021

   Abs PDF

   My thesis will study the intersection of social choice and machine learning, with a focus on recent or under-explored social choice paradigms, such as liquid democracy, and how social choice and ML can benefit each other. My initial results show the idea of using ML and social choice to understand the other holds promise. An early project of mine uses deep learning to enhance social choice by creating a neural network that acts as a voting rule, able to be trained to select a winner satisfying customizable sets of axioms. More recently, I have explored the idea of using liquid democracy as a framework for ensembles for classification problems. I am particularly interested in improving the real-world applicability of existing social choice methods and understanding how they can more beneficially impact the world. Going forward, my primary tools in these goals are simulation and provable axiomatic or performance guarantees.

2. GAIW

   On the Limited Applicability of Liquid Democracy

   Ben Armstrong, and Kate Larson

   Games, Agents, Incentives Workshop at the 20th International Conference on Autonomous Agents and Multi-Agent Systems, May 2021

   Abs PDF

   Liquid democracy is a proxy-voting method in which voters may transitively delegate their votes, allowing others to vote in their place. It is often analysed and used in settings where there is some single, objectively correct outcome that voters collectively aim to select. As a result, much of the previous work on liquid democracy has proposed agent utility functions which depend directly on the accuracy of the delegatee voters. We argue, instead, that a more realistic model would have voters’ incentives more closely aligned with the final outcome, rather than the proxy vote cast. We explore this alternative and show that there exists a pure-strategy Nash equilibirum that can be reached by best-response delegations. Furthermore, allowing for vote-delegation always weakly improves the accuracy of the decision-making process compared to having all agents vote directly. We apply this model to a classifier ensemble problem. While our theoretical findings are positive, our empirical results show that the assumption of independence between voters required for theoretical analysis is critical. Once removed, liquid democracy fails to materialize any practical improvement over direct voting.

2019

1. Could AI Drive Transformative Social Progress? What Would This Require?

   Edward (Ted) A. Parson , Robert Lempert , Ben Armstrong, and 3 more authors

   SSRN Electronic Journal, May 2019

   PDF
2. Siri Humphrey: Design Principles for an AI Policy Analyst

   Ben Armstrong, Megan Beretta , Evan Crothers , and 5 more authors

   May 2019

   Abs PDF

   This workgroup considered whether the policy analysis function in government could be replaced by an artificial intelligence policy analyst (AIPA) that responds directly to requests for information and decision support from political and administrative leaders. We describe the current model for policy analysis, identify the design criteria for an AIPA, and consider its limitations should it be adopted. A core limitation is the essential human interaction between a decision maker and an analyst/advisor, which extends the meaning and purpose of policy analysis beyond a simple synthesis or technical analysis view (each of which is nonetheless a complex task in its own right). Rather than propose a wholesale replacement of policy analysts with AIPA, we reframe the question focussing on the use of AI by human policy analysts for augmenting their current work, what we term intelligenceamplified policy analysis (IAPA). We conclude by considering how policy analysts, schools of public affairs, and institutions of government will need to adapt to the changing nature of policy analysis in an era of increasingly capable AI.

3. AI4SG

   Machine Learning to Strengthen Democracy

   Ben Armstrong, and Kate Larson

   NeurIPS Joint Workshop on AI for Social Good, May 2019

   Abs PDF

   Democratic institutions making use of plurality voting have, in recent years, been shown to be susceptible to interference and to poorly reflect the interests of voters. While many agree that better systems exist, there is no clear consensus on how the perfect election system functions. This paper describes a machine learning approach to create a voting rule aimed at satisfying highly customisable sets of ideal electoral fairness criteria. We first provide a description of our framework, then show in a simple example that it is able to achieve a strong result. Our system aims to provide room for opposing groups to agree on the principles they believe an election should reflect, rather than arguing over pre-existing preferences for particular reforms.

2018

1. Coordination in a Peer Production Platform: A study of Reddit’s /r/Place experiment

   Ben Armstrong

   Masters Thesis, May 2018

   Abs PDF

   Understanding the factors causing groups to engage in coordinating behaviour has been an active research area for decades. In this thesis, we study this problem using a novel dataset of crowd behaviour from an online experiment hosted by Reddit. This experiment allowed users to attempt to build an image alone, or to work collaboratively in the hope of building something greater. We use data provided by Reddit, in addition to crowdsourced coordination information, in order to compare this experiment with a platform containing many similarities to our experiment: Wikipedia. Comparison with Wikipedia shows that many behavioural trends appear to generalize across domains. We go on to construct an agent-based model of the experiment, allowing investigation into the effects of spontaneous and planned coordination. We find that while coordinated work leads to significant productivity improvements in concentrated areas, there is little effect on the experiment as a whole as a result of coordination.

2017

1. Approval in the Echo Chamber

   Ben Armstrong, and Kate Larson

   EXPLORE Workshop at the 16th International Conference on Autonomous Agents and Multi-Agent Systems, May 2017

   Abs PDF

   Recently there has been interest in iterative voting, where voters are able to update their votes based on voting information from previous rounds. In this paper we conduct a series of empirical studies in order to understand the strategic issues which arise when agents, voting to approve a set of k candidates, can base their voting or approval decisions on information from their neighbours in a social network. We illustrate that the k-approval voting rule often results in cyclic voting behaviour, that social network structure matters in terms of strategization, and that homophily in the network decreases strategization for the k-approval voting rule.