MATS empowers researchers to advance AI safety

The ML Alignment & Theory Scholars (MATS) Program is an independent research and educational seminar program that connects talented scholars with top mentors in the fields of AI alignment, interpretability, and governance. For 10 weeks, MATS scholars will conduct research while also attending talks, workshops, and networking events with other members of the Berkeley alignment research community.

The Summer 2024 Program will run Jun 17-Aug 23, 2024 and the Winter 2024-25 Program will run Jan 6-Mar 14, 2025.

—

General applications for the Summer 2024 Program have closed.

Applications for the Winter 2024-25 Program remain open and will close on Aug 1, 2024.

Our Mission

The MATS program aims to find and train talented individuals for what we see as the world’s most urgent and talent-constrained problem: reducing risks from unaligned artificial intelligence (AI). We believe that ambitious researchers from a variety of backgrounds have the potential to meaningfully contribute to the field of alignment research. We aim to provide the training, logistics, and community necessary to aid this transition. We also connect our scholars with financial support to ensure their financial security. Please see our theory of change for more details.

Program Details

  • MATS is an independent research program and educational seminar that connects talented scholars with top mentors in the fields of AI alignment, interpretability, and governance. Read more about the program timeline and content in our program overview.

    The Summer 2024 Cohort will run Jun 17-Aug 23 in Berkeley, California and feature seminar talks from leading AI safety researchers, workshops on research strategy, and networking events with the Bay Area AI safety community. Applications close on Mar 24. [EDIT: extended to April 7.]

    The Winter 2024-25 Cohort will run Jan 6-Mar 14.

    The MATS program is an initiative supported by the Berkeley Existential Risk Initiative. We have historically received funding from Open Philanthropy, the Survival and Flourishing Fund, DALHAP Investments, and several donors via Manifund. We are accepting donations to support further research scholars.

  • Since its inception in late 2021, the MATS program has supported 213 scholars and 47 mentors. After completion of the program, MATS alumni have:

  • Our ideal applicant has:

    • An understanding of the AI alignment research landscape equivalent to completing the AI Safety Fundamentals Alignment Course;

    • Previous experience with technical research (e.g. ML, CS, maths, physics, neuroscience, etc.) or governance research, ideally at a postgraduate level;

    • Strong motivation to pursue a career in AI alignment research, particularly to reduce global catastrophic risk, prevent human disempowerment, and enable sentient flourishing.

    Even if you do not entirely meet these criteria, we encourage you to apply! Several past scholars applied without strong expectations and were accepted.

    We are currently unable to accept applicants who will be under the age of 18 on June 17, 2024.

  • MATS now accepts rolling applications for future cohorts.

    If you submit an application for Summer 2024 by March 10, your application will be in the first batch considered for admission. We will consider all applications submitted before March 24. [EDIT: Extended to April 7.]

    In late March, applicants who make it past the first stage will receive emails that allow them to indicate which mentors’ streams they wish to apply to. Applicants will receive the corresponding mentor selection problems, which are questions and tasks selected by mentors to evaluate your aptitude as a researcher. Note: some mentors request that you spend as long as ~10 hours on their selection problems.

    We expect to send out all admissions decisions by mid May.

Program Tracks for Summer 2024

  • Cooperative AI

    Anthony DiGiovanni (Center on Long-Term Risk)

    The world may soon contain many advanced AI systems frequently interacting with humans and with each other. Can we create a solid game-theoretic foundation for reasoning about these interactions to prevent catastrophic conflict and incentivize cooperation?

  • Evaluations

    Evan Hubinger (Anthropic), Marius Hobbhahn (Apollo Research), Owain Evans (University of Oxford)

    Many stories of AI accident and misuse involve potentially dangerous capabilities, such as sophisticated deception and situational awareness, that have not yet been demonstrated in AI. Can we evaluate such capabilities in existing AI systems to form a foundation for policy and further technical work?

  • Governance

    Nico Miailhe (PRISM Eval and The Future Society), Timothy Fist (IFP and CNAS)

    As AI systems continue to advance and develop even stronger capabilities, can we develop policies, standards, and frameworks to guide the ethical development, deployment, and regulation of AI technologies, focusing on ensuring safety and societal benefit?

  • Interpretability

    Neel Nanda (Google DeepMind), Alex Turner (Google DeepMind), Lee Sharkey (Apollo Research), Jessica Rumbelow (Leap Labs), Stephen Casper (MIT AAG)

    Rigorously understanding how ML models function may allow us to identify and train against misalignment. Can we reverse engineer neural nets from their weights, similar to how one might reverse engineer a binary compiled program?

  • Oversight/control

    Ethan Perez (Anthropic), Buck Shlegeris (Redwood Research), Fabien Roger (Redwood Research), David Lindner (Google DeepMind), Akbir Khan (UCL DARK), Mantas Mazeika (UIUC)

    As model develop potential dangerous behaviors, can we develop and evaluate methods to monitor and regulate AI systems, ensuring they adhere to desired behaviors while minimally undermining their efficiency or performance?