Over the past eight months, ChatGPT has impressed millions of people with its ability to generate realistic-looking text, writing everything from stories to code. But the chatbot, developed by OpenAI, is still relatively limited in what it can do.
The large language model (LLM) takes "prompts" from users that it uses to generate ostensibly related text. These responses are created partly from data scraped from the internet in September 2021, and it doesn't pull in new data from the web. Enter plug-ins, which add functionality but are available only to people who pay for access to GPT-4, the updated version of OpenAI's model.
Since OpenAI launched plug-ins for ChatGPT in March, developers have raced to create and publish plug-ins that allow the chatbot to do a lot more. Existing plug-ins let you search for flights and plan trips, and make it so ChatGPT can access and analyze text on websites, in documents, and on videos. Other plug-ins are more niche, promising you the ability to chat with the Tesla owner’s manual or search through British political speeches. There are currently more than 100 pages of plug-ins listed on ChatGPT’s plug-in store.
But amid the explosion of these extensions, security researchers say there are some problems with the way that plug-ins operate, which can put people’s data at risk or potentially be abused by malicious hackers.
Johann Rehberger, a red team director at Electronic Arts and security researcher, has been documenting issues with ChatGPT’s plug-ins in his spare time. The researcher has documented how ChatGPT plug-ins could be used to steal someone’s chat history, obtain personal information, and allow code to be remotely executed on someone’s machine. He has mostly been focusing on plug-ins that use OAuth, a web standard that allows you to share data across online accounts. Rehberger says he has been in touch privately with around a half-dozen plug-in developers to raise issues, and has contacted OpenAI a handful of times.
“ChatGPT cannot trust the plug-in,” Rehberger says. “It fundamentally cannot trust what comes back from the plug-in because it could be anything.” A malicious website or document could, through the use of a plug-in, attempt to run a prompt injection attack against the large language model (LLM). Or it could insert malicious payloads, Rehberger says.
Data could also potentially be stolen through cross plug-in request forgery, the researcher says. A website could include a prompt injection that makes ChatGPT open another plug-in and perform extra actions, which he has shown through a proof of concept. Researchers call this “chaining,” where one plug-in calls another one to operate. “There are no real security boundaries” within ChatGPT plug-ins, Rehberger says. “It is not very well defined, what the security and trust, what the actual responsibilities [are] of each stakeholder.”
Since they launched in March, ChatGPT’s plug-ins have been in beta—essentially an early experimental version. When using plug-ins on ChatGPT, the system warns that people should trust a plug-in before they use it, and that for the plug-in to work ChatGPT may need to send your conversation and other data to the plug-in.
Niko Felix, a spokesperson for OpenAI, says the company is working to improve ChatGPT against “exploits” that can lead to its system being abused. It currently reviews plug-ins before they are included in its store. In a blog post in June, the company said it has seen research showing how “untrusted data from a tool’s output can instruct the model to perform unintended actions.” And that it encourages developers to make people click confirmation buttons before actions with “real-world impact,” such as sending an email, are done by ChatGPT.
“While ChatGPT plug-ins are developed externally to OpenAI, we aim to provide a library of third-party plug-ins that our users can trust,” Felix says, adding it is “exploring” ways to make plug-ins safer for people using them. “For example, making it easier to provide a user confirmation flow if they intend for their plug-in to take a significant action.” OpenAI has removed at least one plug-in—which created entries on a developer’s GitHub page without asking the users’ permission—for breaching its policy of requiring confirmation before taking action.
Unlike on Apple and Google’s app stores, ChatGPT’s plug-in library currently doesn’t appear to list the developers behind the plug-in or provide any information about how they may use any data collected the plug-in collects. Developers creating plug-ins, according to OpenAI’s guidance, must follow its content guidelines and provide a manifest file, which includes contact information for the plug-in’s creators, among other details. When searching for and turning on a plug-in in ChatGPT, only its name, a short description, and logo are shown. (An unaffiliated third-party website shows more information).
When OpenAI launched plug-ins in March, researchers warned of potential security risks and the implications of connecting GPT-4 to the web. However, the issues with plug-ins aren’t confined to OpenAI and ChatGPT. Similar risks apply to any LLMs or generative AI systems connected to the web. It’s possible that plug-ins will play a big role in the way people use LLMs in the future. Microsoft, which has heavily invested in OpenAI, has said it will use the same standards for plug-in creation as ChatGPT. “I think there's going to eventually be an incredibly rich ecosystem of plug-ins,” Microsoft’s chief technology officer Kevin Scott said in May.
Chang Kawaguchi, vice president of AI security at Microsoft, says the firm is taking an "iterative" approach to launching support for plug-ins in its AI Copilot assistant tool. "We'll extend our existing processes for publishing, validating, certifying, deploying, and managing product integrations to plug-ins, to ensure that customers of Microsoft Copilots have full control of their plug-ins, the data they can access, and the people authorized to deploy them," Kawaguchi says, adding the company will document security guidelines and work with external researchers on problems they find.
Many of the issues around plug-ins—and LLMs more widely—are around trust. This includes whether people can trust their private and corporate data with the systems and whether controls and measures are put in place to make sure what is handed over can’t be improperly used or accessed.
“You're potentially giving it the keys to the kingdom—access to your databases and other systems,” says Steve Wilson, chief product officer at Contrast Security and the lead of a project detailing security risks with LLMs. Around 450 security and AI experts have come together to create a list of the 10 top security threats around LLMs as part of the Open Worldwide Application Security Project (OWASP), according to Wilson, the project’s coordinator.
The effort exists, he says, as developers are rushing to create LLM-powered applications and services. But at the moment, there is little guidance on what they need to do to secure what they’re making. The top threat is listed as prompt injection attacks (where malicious data tries to take control of an AI system) but also included are data poisoning and supply chain vulnerabilities. The list also singles out plug-ins as a security risk.
The OWASP researchers list six possible ways LLM plug-ins can be attacked. These include malicious URLs being used through plug-ins and SQL attacks, and by allowing plug-ins to take actions without validation. The group points to a myriad of steps that developers should take to avoid risks, including making sure proper authentication exists and preventing “sensitive plug-ins from being called after any other plug-in.”
Wilson says that in general, he would tell anyone using public LLMs to be “really careful” about what information they put into them. “You're not necessarily sure how that's gonna get used, preserved, and possibly regurgitated elsewhere,” Wilson says. “Those plug-ins, for sure, add another level of exposure. The art around securing these things is barely understood, and thus the ability to really secure them doesn't exist yet.”
