Headline: New Tactics Employed by Iranian and Chinese Threat Actors to Misuse US AI Models for Covert Manipulation: Study
Body: Robert O'Brien, former National Security Adviser for President Trump, recently addressed the issue of China's military expansion. However, a more pressing matter is the innovative ways threat actors, potentially from Iran and China, are hijacking American artificial intelligence (AI) models for malicious purposes such as covert influence operations, as revealed by a new OpenAI study.
The study, published in February, highlights two instances where the threat actors seem to have originated from China. The report suggests these actors attempted, and in some cases succeeded, in utilizing AI models developed by OpenAI and Meta.
OpenAI recently suspended a ChatGPT account that created comments criticizing Chinese dissident, Cai Xia. These comments were shared on social media by accounts that falsely claimed to be based in India and the U.S., although they failed to gain significant traction online.
The same threat actor also employed ChatGPT to produce extensive Spanish news articles that cast the U.S. in a negative light. These articles were subsequently published by major news media in Latin America. The credit for these stories was often given to an individual or, in some instances, a Chinese firm.
During a recent media briefing, Ben Nimmo, Principal Investigator on OpenAI’s Intelligence and Investigations team, noted that a translation was cited as sponsored content at least once, implying it was paid for. This incident marks the first known successful attempt by a Chinese actor to plant long-form articles in mainstream media, targeting Latin American audiences with anti-U.S. sentiments.
Nimmo further explained that the threat actors gave OpenAI insight into their actions on other parts of the internet through their use of the AI models. He described it as a worrying look at how non-democratic actors attempted to manipulate democratic or U.S.-based AI for non-democratic ends.
OpenAI also suspended a ChatGPT account that generated tweets and articles later posted on third-party platforms publicly associated with known Iranian influence operations. These operations are currently considered as separate endeavors.
However, the potential intersection of these operations, while minimal and isolated, poses the question of collaboration amongst Iranian influence operations, suggesting that one operator might work on behalf of seemingly distinct networks.
OpenAI also suspended numerous ChatGPT accounts that were using their models to translate and generate comments for a "pig butchering" romance scam network on platforms like X, Facebook, and Instagram. Following this discovery, Meta suggested that the activity seemed to stem from a recently established scam hub in Cambodia.
OpenAI made history last year by becoming the first AI research lab to publish reports on efforts to prevent abuse by adversaries and other harmful actors. The lab collaborates with the U.S., allied governments, industry partners, and stakeholders.
Since publishing its first report, OpenAI has significantly expanded its investigative capabilities and knowledge of new forms of abuse, successfully disrupting a broad spectrum of malicious uses.
OpenAI believes that AI companies can gain considerable insights into threat actors by sharing information with upstream providers like hosting and software providers, as well as downstream distribution platforms such as social media companies and open-source researchers.
OpenAI adds that their investigations also gain significantly from the insights shared by their peers. The company is committed to continuously identifying, preventing, disrupting, and exposing efforts to exploit their models for harmful purposes, acknowledging that threat actors will persist in testing their defenses.