OpenAI o1 preview

OpenAI recently announced the development of a new series of AI models designed to spend more time thinking before responding. These models can reason through complex tasks and solve harder problems than previous iterations in science, coding, and math.

OpenAI o1 preview, has been released in ChatGPT and the API

The first model in this series, named OpenAI o1-preview, has been released in ChatGPT and the API. This is a preview version, and OpenAI expects regular updates and improvements. Alongside this release, evaluations for the next update, which is currently in development, have also been included.

How it works

These models have been trained to take time to think through problems before responding, similar to human thought processes. Through this training, they learn to refine their thinking, explore different strategies, and recognize their mistakes.

In tests, the upcoming model update has demonstrated performance comparable to PhD students on challenging benchmark tasks in physics, chemistry, and biology. It has also shown exceptional capabilities in math and coding. For instance, in a qualifying exam for the International Mathematics Olympiad (IMO), the GPT-4o model correctly solved only 13% of problems, whereas the reasoning model scored 83%. Its coding abilities were assessed in contests, achieving the 89th percentile in Codeforces competitions. More details can be found in OpenAI's technical research post.

As an early model, OpenAI o1-preview does not yet include many features that enhance the ChatGPT experience, such as web browsing for information and the ability to upload files and images. However, for many common tasks, GPT-4o will be more capable in the near term.

This significant advancement in complex reasoning tasks represents a new level of AI capability, prompting OpenAI to reset the counter and name this series OpenAI o1.

OpenAI has introduced a new safety training approach

As part of the development of these new models, OpenAI has introduced a new safety training approach that leverages their reasoning capabilities to ensure adherence to safety and alignment guidelines. By reasoning about safety rules in context, these models can apply them more effectively.

One method of measuring safety involves testing how well the model adheres to its safety rules when users attempt to bypass them, a process known as "jailbreaking." In one of the most challenging jailbreaking tests, the GPT-4o model scored 22 on a scale of 0-100, while the o1-preview model achieved a score of 84. Further details are available in the system card and accompanying research post.

To align with the enhanced capabilities of these models, OpenAI has strengthened its safety initiatives, internal governance, and collaboration with federal governments. This includes rigorous testing and evaluations utilizing their Preparedness Framework, best-in-class red teaming, and board-level review processes, including oversight from the Safety & Security Committee.

To further their commitment to AI safety, OpenAI has recently formalized agreements with the U.S. and U.K. AI Safety Institutes. The organization has begun operationalizing these agreements, which include granting the institutes early access to a research version of the model. This initiative marks an important first step in establishing a framework for the research, evaluation, and testing of future models both prior to and following their public release.

These enhanced reasoning capabilities may be particularly beneficial for addressing complex problems in fields such as science, coding, and mathematics. For instance, the o1 model can assist healthcare researchers in annotating cell sequencing data, enable physicists to generate complex mathematical formulas required for quantum optics, and support developers across various disciplines in building and executing multi-step workflows.