OpenAI embedded assessments
by OpenAI · Technique · Text & code
OpenAI says outside auditors get deep access; critics say it is too limited
OpenAI gives Apollo, METR, Transluce and Goodfire internal access: Apollo six onsite evaluators for four months, Transluce a six-week audit, METR a review of the largest RL run
Why we say so
OpenAI says Apollo, METR, Transluce and Goodfire get internal access, including six onsite Apollo evaluators for four months and a METR review of the largest RL run. Neel Nanda, Alex Mallen and METR say it falls short of employee-like access: no Slack, OpenAI chooses who is interviewed, no weights or training-based elicitation, and METR says its scope is narrower than it proposed. This is a process, not a product, so the open question is how much the assessments will actually find.
Evidence
- Nov 17Critics say access falls short of 'employee-like': no Slack, OpenAI picks interviewees, no weights or training-based elicitation; METR says its scope is narrower than proposed