AI Research Projects

AI Research Projects is sponsored by Cooperative AI Foundation (CAIF). CAIF supports research that aims to improve the cooperative intelligence of advanced AI for the benefit of all. This includes work that helps build the infrastructure of the field, such as novel benchmark environments and metrics of cooperative success.

Cooperative AI – Foundation Introduction Mission Activities Announcements Apply People Introduction Mission Activities Announcements Apply People Audrey Tang Thore Graepel Allan Dafoe Gillian Hadfield Jesse Clifton Importance, Neglectedness, and Tractability Scientific progress is often hard to predict.

Great research is frequently driven by intrinsic curiosity, and it can be difficult to say in advance what the most important research directions for a field will turn out to be. On the other hand, we think that there are features which make some research directions more promising than others.

Following an increasing number of philanthropic organisations (see here , for example), CAIF is guided by the importance, neglectedness, and tractability of potential activities, while maintaining an awareness of the value of curiosity and serendipity for great science. ‍ Importance. Is this research area likely to make an important contribution to the cooperative intelligence of advanced AI systems for the benefit of all?

Neglectedness. Is this work likely to be done anyway? ‍ Tractability.

Does the research area lend itself especially well to making progress? While tractability is a common dimension along which to assess potential research, our emphasis on the qualities of importance and neglectedness differ somewhat from their standard interpretations in academia.

For instance, a result may be viewed as particularly important within the researcher’s own field, but to warrant the highest prioritisation such work should also be likely to help us build more cooperative AI systems, now and in the future.

Likewise, in many fields there may be multiple research groups knowingly racing to solve the same problem, but from the perspective of counterfactual impact this may not be the kind of research that’s essential to prioritise, because it’s likely to happen with or without our support.

Supporting research that will improve the cooperative intelligence of advanced AI for the benefit of all In the course of building AI systems to be more cooperatively intelligent, those systems might also gain capabilities that could be used to harm others rather than contribute to improvements in social welfare.

The alignment problem is one example of this: as AI systems become more generally intelligent, divergences in their goals and humans’ become more dangerous for humans.

In the context of Cooperative AI, the ability to understand other agents can lead to improvements in cooperation, but also in deception and manipulation, and the same abilities that allow one to commit to honouring mutually beneficial agreements could also be used to commit to coercive threats. With these risks in mind, CAIF is interested in supporting differential progress on cooperative intelligence.

That is, we want to support research that leads to significant progress on cooperative capabilities – capabilities that lead to increases in social welfare in a wide range of environments – relative to progress on capabilities that are dual-use (e.g., useful for deception, manipulation, disempowering other agents) and therefore may not robustly improve social welfare.

This idea is discussed in further detail in a recent seminar from the New Directions in Cooperative AI series. Improvements that CAIF prioritises are counterfactual and long-term, i.e., those improvements over the next 10-20 years (or longer) that would have been unlikely without our support (see also the discussion of 'Neglectedness' further below).

Advanced AI systems include not only the present day state-of-the-art, but the kinds of powerful AI systems we can expect to see in the next 10-20 years, and the networks of humans and organisations in which they are embedded. The benefit of all is our fundamental concern, and highlights the fact that not all advances in Cooperative AI may be beneficial for everyone; we must take into account different perspectives and values.

Cooperative intelligence refers to the skills required for promoting cooperation between humans, machines, or organisations, though further research is required to fully conceptualise and define these skills. Supporting research includes standard academic grantmaking, but also fostering research in other ways, such as organising workshops and other events, supporting students, awarding prizes, and providing educational tools.

Intro text about our activities CAIF intends to use its philanthropic endowment to: – Make grants to support Cooperative AI research, especially that which is important, tractable, and neglected . This includes work which helps to build up the infrastructure of the field, such as novel benchmark environments and metrics of cooperative success.

– Offer scholarships to promising young researchers intent on entering the field of Cooperative AI. Details on calls for proposals and applications forthcoming. In 2020, the first Cooperative AI workshop was organised at NeurIPS.

CAIF intends to continue to organize workshops at major machine learning conferences, including IJCAI, AAAI, AAMAS, and NeurIPS. CAIF will host a series of online seminars featuring scholars working on the frontier of Cooperative AI. Further details of our first seminar series, New Directions in Cooperative AI , and our call for seminar proposals can be found here .

CAIF will explore additional ways of contributing to the growth of Cooperative AI, including administering prizes and hosting tournaments which encourage progress in our understanding of the cooperative intelligence of AI systems. Announcing the 2026 Cooperative AI PhD Fellows We're delighted to welcome 14 exceptional early careerists who'll be joining our next Cooperative AI PhD Fellowship cohort.

Thore Graepel Joins Board of Trustees The Cooperative AI Foundation is delighted to welcome Google DeepMind Distinguished Research Scientist and prominent multi-agent researcher Thore Graepel to our board of trustees. Recent Grants Awarded by the Cooperative AI Foundation The Cooperative AI Foundation has provided a number of grants to support research on cooperative AI for the benefit of all.

Partnerships to Support Early-Career Researchers The Cooperative AI Foundation has partnered with two external research initiatives (the PIBBSS Fellowship and the MATS Program) to support early-career researchers. Voluptatum molestiae aliquid neque dicta. Eaque rerum perspiciatis non quibusdam minima.

Enim impedit aut neque ipsa dolorem ut praesentium consequatur alias. Cumque rem aut quia hic praesentium deserunt pariatur nemo dolorem. Non repudiandae est tenetur sint.

Quae ipsum quidem tempore blanditiis ducimus. Nihil vel maxime iste possimus veniam est consectetur dolor. Tempora eligendi et voluptatem et aspernatur aut iste.

Nobis iusto consequatur sunt voluptatem est tempore sunt perspiciatis totam. Atque molestiae ullam quis autem eaque cumque culpa. Et qui nisi dolorem aut nam nemo.

Sit laudantium ea non doloremque Doloremque Voluptatibus Vel Culpa Quia aut ratione et iure delectus quae magnam eum laudantium. Est voluptas aut incidunt quam. Ipsum molestias non sint ut autem.

Iste quas consequuntur sed qui qui incidunt esse vero cumque. Eum iure dolore et quaerat dolorem aut ab rem repudiandae. Quis culpa in accusantium et iste at beatae non.

Doloremque libero dolor consequatur corporis. Praesentium eos quia id nemo neque nobis maio Omnis animi commodi ut consequuntur natus totam optio. Voluptatum molestiae aliquid neque dicta.

Eaque rerum perspiciatis non quibusdam minima. Enim impedit aut neque ipsa dolorem ut praesentium consequatur alias. Cumque rem aut quia hic praesentium deserunt pariatur nemo dolorem.

Non repudiandae est tenetur sint. Quae ipsum quidem tempore blanditiis ducimus. Nihil vel maxime iste possimus veniam est consectetur dolor.

Tempora eligendi et voluptatem et aspernatur aut iste. Nobis iusto consequatur sunt voluptatem est tempore sunt perspiciatis totam. Atque molestiae ullam quis autem eaque cumque culpa.

Et qui nisi dolorem aut nam nemo. Sit laudantium ea non doloremque Doloremque Voluptatibus Vel Culpa Quia aut ratione et iure delectus quae magnam eum laudantium. Est voluptas aut incidunt quam.

Ipsum molestias non sint ut autem. Iste quas consequuntur sed qui qui incidunt esse vero cumque. Eum iure dolore et quaerat dolorem aut ab rem repudiandae.

Quis culpa in accusantium et iste at beatae non. Doloremque libero dolor consequatur corporis. Praesentium eos quia id nemo neque nobis maio Omnis animi commodi ut consequuntur natus totam optio.

How to apply for opportunities As we scale up our programmes and deepen our impact, we're looking for a highly organised and proactive Administration Associate to keep our operations running smoothly. This is a varied, hands-on role sitting at the heart of the organisation, from supporting our trustee meetings and team events to helping deliver our PhD Fellows programme and keeping our communications humming.

You will work closely with colleagues across the foundation, playing a vital behind-the-scenes role in a small, ambitious team that genuinely believes in what it's building. If you thrive on bringing order to complexity, take pride in getting the details right, and want your work to matter, we'd love to hear from you.

Application Deadline: 29 June 2026 08:00 GMT Cooperative AI PhD Fellowships The Cooperative AI PhD Fellowship is designed to provide future and current PhD students in the field of cooperative AI financial support to achieve their full potential. Deadline: 16 November 2025 23:59 AoE Cooperative AI Research Grants The Cooperative AI Foundation is seeking proposals for research projects in cooperative AI.

Deadline: 18 January 2025 23:59 AoE Policy and Partnerships Manager People and Operations Manager Associate Director (Research and Grants) Senior Staff Research Scientist, DeepMind President, Centre for the Governance of AI Professor of Government and Policy and Research Professor in Computer Science, Johns Hopkins University Distinguished Research Scientist, Google DeepMind Chair of Machine Learning, University College London Grantmaking Officer, Macroscopic Ventures Research Scientist, Google DeepMind Associate Professor, University of Oxford Staff Research Engineer, Google DeepMind Professor, University of Waterloo Assistant Professor, University of Washington Senior Research Scientist, Google DeepMind Professor, Carnegie Mellon University Professor, University of Oxford DPhil candidate at University of Oxford | Google DeepMind Member of Technical Staff, Anthropic Thank you!

Your submission has been received! Oops! Something went wrong while submitting the form.

Foundation Events Blog Resources Blog Contact The Cooperative AI Foundation is registered with the Charity Commission for England and Wales under charity number 1201294, and incorporated as a company limited by guarantee established in England for charitable purposes only under company number 13485176. Its registered address is Courtenay House, Pynes Hill, Exeter, EX2 5AZ.