Technical AI Safety (Navigating Transformative AI)

Technical AI Safety (Navigating Transformative AI) is sponsored by Coefficient Giving. This program funds technical AI safety research aimed at making advanced AI systems more trustworthy, robust, controllable, and aligned.

Perrault receives funding from Coefficient Giving for AI safety research | College of Engineering Perrault receives funding from Coefficient Giving for AI safety research Computer Science and Engineering Assistant Professor Andrew Perrault was awarded a $187,510 grant from Coefficient Giving (formerly Open Philanthropy) to support his AI safety research.

The grant was provided through Coefficient Giving’s Call for Technical AI Safety Research to accelerate impactful work addressing the risk that AI systems could become misaligned—possibly pursuing goals no one gave them and harming people in the process.

Perrault is the second faculty member in the College of Engineering to receive funding from Coefficient Giving, following two grants received by Associate Professor Huan Sun earlier this year. Perrault’s research project focuses on the area of alignment faking—when a large language model (LLM) pretends to be aligned with its developers’ specifications during training to achieve another goal once deployed.

Specifically, his project will study whether an LLM can collude with the model assigned to monitor it, effectively bypassing the safeguards designed to keep them honest. “Most work thus far addresses LLM behavior in isolation. Some cursorily address collusion.

As yet, LLM agents are comparatively rare in the world and thus almost always interact either with humans or simpler traditional software tools,” Perrault said. “As they proliferate, they will interact with each other rather than us. AI safety must transition from solitary LLM settings to multi-agent ones.

” The project’s primary goal is to understand how LLMs might cooperate in dishonest ways so researchers can develop stronger safety measures, especially in critical settings such as law or healthcare. Several present-day techniques rely on LLM evaluation of model outputs for safety and validity. While this pattern is commonly assumed benign, caution is warranted when subversion can go unnoticed, Perrault said.

He is supported in this work by computer science and engineering PhD student Trebor Shankle. Since 2014, Coefficient Giving has put hundreds of millions of dollars toward scientific research, funding groundbreaking work on computational protein design, novel methods for malaria eradication, and cutting-edge strategies for pandemic prevention.

With transformative AI on the horizon, they see another opportunity for funding to accelerate highly impactful technical research. by Candi Clevenger, College of Engineering Communications, clevenger. 87@osu.

edu Categories: Research Faculty Awards From Hormuz to the cockpit: Warfare, criminal activity undermine GPS Applied aeronautics program takes flight in its first year Sanjay Krishna appointed to lead Ohio State’s Innovation and Strategic Ventures team Metzler chosen to lead academic programs and student services Smart students excel in Smart Campus Challenge New course helps turn ideas into software startups