AI Safety Research Program

AI Safety Research Program is sponsored by Open Philanthropy. A call for technical AI safety research, particularly focusing on preventing AI systems from pursuing misaligned goals and harming people.

AI safety research attracts funding from Open Philanthropy | College of Engineering AI safety research attracts funding from Open Philanthropy Posted: September 2, 2025 Computer Science and Engineering Associate Professor Huan Sun has been awarded two competitive research grants from Open Philanthropy focused on the rapidly evolving field of artificial intelligence (AI) safety.

The grants were provided through Open Philanthropy’s Call for Technical AI Safety Research to accelerate impactful work addressing the risk that AI systems could become misaligned — possibly pursuing goals no one gave them and harm people in the process. Sun’s research will tackle these challenges through two distinct but complementary projects, which are the first from Ohio State to receive funding from Open Philanthropy.

One project focuses on a critical failure mode in computer-use agents (CUAs) — autonomous AI systems capable of performing complex tasks like file management and web browsing. Despite receiving benign input, these agents can sometimes pursue goals that misalign with user or developer intentions.

The research aims to identify and characterize these misalignment risks, which are increasingly relevant as CUAs become more capable and widely deployed. The other delves into the foundational limitations of transformer models, neural networks that underpin many modern AI systems. Specifically, it will investigate phenomena such as the “reversal curse” and challenges in multi-hop reasoning.

By developing novel techniques to better understand these issues, Sun hopes to uncover solutions. Earlier this year, Sun received research funding through Schmidt Sciences’ AI Safety Science program for her work to defend multimodal computer-use agents from adversarial attacks.

Since 2014, Open Philanthropy has put hundreds of millions of dollars toward scientific research, funding groundbreaking work on computational protein design, novel methods for malaria eradication, and cutting-edge strategies for pandemic prevention. With transformative AI on the horizon, they see another opportunity for funding to accelerate highly impactful technical research.

Categories: Research Faculty Lighter, stronger, tougher Kramer named 2026 Engineering Unleashed Fellow Paparrizos to advance trustworthy AI systems with NSF CAREER funding Ohio State joins Midwest Quantum Collaboratory Study: Illicit fentanyl use more than double medical use Nanoscale DNA sensors could reveal hidden signals inside spinal discs