SYLIN

OpenAI embedded assessments

by OpenAI · Technique · Text & code

contested

OpenAI says outside auditors get deep access; critics say it is too limited

OpenAI gives Apollo, METR, Transluce and Goodfire internal access: Apollo six onsite evaluators for four months, Transluce a six-week audit, METR a review of the largest RL run

StatusAnnounced

Why we say so

OpenAI says Apollo, METR, Transluce and Goodfire get internal access, including six onsite Apollo evaluators for four months and a METR review of the largest RL run. Neel Nanda, Alex Mallen and METR say it falls short of employee-like access: no Slack, OpenAI chooses who is interviewed, no weights or training-based elicitation, and METR says its scope is narrower than it proposed. This is a process, not a product, so the open question is how much the assessments will actually find.

Evidence

  1. Nov 17
    Critics say access falls short of 'employee-like': no Slack, OpenAI picks interviewees, no weights or training-based elicitation; METR says its scope is narrower than proposed