CHICAGO — As concerns mount over artificial intelligence agents “going rogue” and accessing unauthorized websites, AI behemoth OpenAI recently alerted the city of Chicago that its technology probed an online city database, a city spokesperson confirmed.
OpenAI’s technology accessed information via an online, “public-facing” database, mayoral press secretary Allison Novelo said. It’s not clear why this query prompted the AI company to alert city officials.
ChatGPT — OpenAI’s public-facing chatbot — regularly utilizes city data when answering questions about Chicago. For example, a question posed by a reporter this week about crime data in Chicago cited a police database as a source.
“At this time, the city of Chicago is unaware of any sensitive information being obtained by OpenAI, nor is the city aware of OpenAI engaging in unauthorized use of city systems,” Novelo said in a statement. “The city publishes several data sets and dashboards for transparency and public awareness. The data in question was public data from a public facing dashboard.”
OpenAI did not return a request for more details on Tuesday.
The disclosure comes as OpenAI on Friday paused the training of its latest AI model after it disclosed its non-human agents probed government websites in unexpected ways beyond what it was instructed to do, the Associated Press reported over the weekend.
That included sites operated by the U.S. Department of Education and the Securities and Exchange Commission.
It is the second time in three months that OpenAI has halted development of its models. The first came in July after disclosure of a cyberattack targeting AI startup Hugging Face, a now notorious incident that raised fears the industry was losing control of its technology.
In a blog post Friday, OpenAI said that a notification from the company to another company or body of government does not necessarily mean a security incident has taken place.
“Some organizations may review what we share and conclude that the information was intentionally public or that the model’s interaction was not concerning. Others may identify a design issue or security weakness they want to address,” the post said.
“Some of the websites involved are operated by governments, universities, public agencies, and other institutions. That is partly because models performing research tasks are often directed toward authoritative sources of public information.”
However, OpenAI apologized Monday after its agents on four separate occasions in recent months improperly accessed Australian government websites, including at least one incident in which nonpublic information was obtained.
‘A Serious Problem’
Taken together, the reports of AI agents accessing U.S. government sites in unexpected ways pose a “serious problem” — even if the incidents have been mostly benign so far, said Professor Henry Hoffmann, chair of the computer science department at the University of Chicago.
“The problem here is that, these are very powerful systems that are able to act faster than we can currently observe and verify and validate them, and the proper controls are not being put into place to make sure that they don’t do this,” Hoffmann said. “We haven’t heard, publicly anyway, that any of these things have accessed real sensitive information, but it does demonstrate how quickly real harm could happen.”
Hoffmann added that he hopes scientific consensus will work towards greater oversight on artificial intelligence to prevent unauthorized incidents and access. He’d like to see AI companies expand transparency into how their models are operating and put greater controls in place to limit or halt rogue behavior.
Mayor Brandon Johnson on Tuesday acknowledged the OpenAI alert to the city at an unrelated press conference, but said he’s confident in the Chicago’s technology department to protect its data.
“It’s a strong, talented team that is constantly working to make sure that our sensitive information is protected,” Johnson said. “This is going to be an ongoing effort not just for the city of Chicago but for all of the globe to be far more diligent around protecting these forward-facing data spaces that are public. So, again, I’m not aware of any sensitive information or unauthorized information being released.”
Safety fears continued to plague OpenAI into this week. The company announced Monday that it was delaying the release of a new artificial intelligence model after security concerns were voiced by its researchers.
The Associated Press contributed to this story.
______________________________________________________________________
Support Local News!
Subscribe to Block Club Chicago, an independent, 501(c)(3), journalist-run newsroom. Every dime we make funds reporting from Chicago’s neighborhoods. Already subscribe? Click here to gift a subscription, or you can support Block Club with a tax-deductible donation.
Listen to the Block Club Chicago podcast: