San Francisco (California) [US], September 17 (ANI): OpenAI on Wednesday (local time) published six reports of misaligned behavior observed during the training or evaluation of its AI models. AI alignment is the practice of making artificial intelligence systems act in line with human values, safety goals, and intentions.
OpenAI says these are reports of individual instances, and shouldn’t be considered reflective of how often misalignment occurs across its models. In a statement Open AI said this new framework is intended to expedite publishing misalignment reports.
"We are sharing a new framework for tracking, investigating, and disclosing instances of model misalignment at OpenAI, along with six reports on unexpected or concerning model behaviour we’ve observed in the last six months. In the past, so as to better inform researchers, AI developers, policymakers, and the general public, we’ve sought to make our findings about misalignment public. But without a systematic approach to reporting these findings, our disclosures have been ad hoc and less frequent than ideal: we’ve often waited until we could collate several instances into one report, or added them to system cards for newly released models. This new framework is intended to expedite publishing misalignment reports following observation, even when we haven’t fully explained or mitigated the behaviour we’re reporting," the company said.
OpenAI further said that as AI systems grow more advanced and more widely deployed, there was a need to build a broader and better-informed consensus on the progress of alignment research.
"We do not believe that the AI industry has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. Decisions about how AI development should proceed in the months and years to come need to draw on evidence that people outside the companies building frontier models can examine for themselves. Examples of misalignment may help identify problems other AI developers might encounter as their systems reach similar capabilities, reveal weaknesses in safeguards, or challenge assumptions about model behaviour," the company said.
The company called for an industry-wide framework with explicit standards for how AI developers should disclose examples of misalignment in their models.
"We hope that the framework we’re outlining today is a first step toward creating such standards, setting out which misalignment instances developers should disclose and what their reports should contain. We regard this framework as a work in progress, which we’ll refine through experience and public feedback," it said.
Open AI further said it was committed to disclosing instances of misalignment that meet this framework’s criteria, including more complex cases requiring longer investigation or coordination with third parties.
Earlier, speaking an event hosted by Salesforce CEO and Chairman Marc Benioff, Open AI CEO Sam Altman said that AI safety should not have any qualifier and said it was great that the industry wants to come together to ensure this.
"You have companies saying things like, we will only slow down, or we'll only be responsible if other companies are responsible...And then the public naturally says, we'd really like to know that you're going to do this safely and be responsible no matter what...There should be no qualifier on that," he said.
"The world should trust that we are going to do the right thing because it's the right thing, and because we feel the magnitude of this...I think it's great for our industry to say we want to come together and we want to be able to coordinate and make sure we have enough time to do this safely...But when there's any implication that because of the commercial pressures and the race, some company, or between countries, some countries might not do the right thing, I think that's when people get very scared," he added.
The debate over slowing AI down is being described as an inflection point in the industry. What set-off as a question of principle first raised when Anthropic researcher Jacob Coxon made a public resignation claiming that building of 'superintelligent' systems possessed a genuine risk to humanity, is now a full-fledged debate on how the AI industry will evolve in the future. (ANI)
OpenAI flags six instances of misaligned behaviour of AI models, calls for industry-wide framework
