The UK government organisation established to advise the public and private sectors on cyber security has urged proportionality in the embrace of ChatGPT and Large Language Models (LLMs).
In a blog post on its website, the NCSC says: “LLMs are undoubtedly impressive for their ability to generate a huge range of convincing content in multiple human and computer languages. However, they’re not magic, they’re not artificial general intelligence, and contain some serious flaws”.
As with all emerging technologies, there’s always concern over the security aspects. The NCSC says that there are “undoubtedly risks involved” in the unrestrained use of public LLMs, and that individuals and organizations should take great care with what type of data they choose to submit in prompts, but that there is no cause for alarm.
Large Language Model (LLM)
A LLM is an artificial intelligence tool that can read a large amount of text, summarize and translate it. It can also predict upcoming words in a sentence by generating sentences similar to how humans talk and write.
Large amount of data
A LLM can be very useful and save a lot of time by scraping the internet and handling a lot of different text-based data, including social media posts, but the NCSC highlights the faults in its filtering abilities. “This covers such a large volume of data that it’s not possible to filter all offensive or inaccurate content at ingest, and so ‘controversial’ content is likely to be included in its model.”
The flaws are that they can:
- get things wrong and ‘hallucinate’ incorrect facts;
- be biased, and often gullible in response to leading questions;
- be coaxed into creating toxic content and prone to ‘injection attacks’; and
- need huge compute resources and vast data to train from scratch.
Personal data
Another common worry is that a LLM might ‘learn’ from one person’s prompts, and then offer that information to others who search for similar things. The NCSC says that there is some ‘cause for concern’ here, but not in the way that one might think.
As of now, a LLM does not automatically add on new information from queries to its model for others to query. But queries will be visible to the organization that provides the LLM, which means the provider and/or its partners can read them. And that the queries could be incorporated in future developments. Therefore, the NCSC advises that terms of use and privacy policies need to be fully understood before asking sensitive questions.
“LLMs are undoubtedly impressive for their ability to generate a huge range of convincing content in multiple human and computer languages. However, they’re not magic, they’re not artificial general intelligence, and contain some serious flaws.”
NCSC
Other risks are that queries stored online face being hacked or leaked, or ‘accidentally’ made public – this includes user-identifiable information – and the LLM operator being acquired by an organization that holds a different privacy approach to personal data.
To minimize such risks, the NCSC recommends:
- to not include sensitive information in queries to public LLMs; and
- to not submit queries to public LLMs that would lead to issues if they were made public.
Writing malware
But do LLMs make life easier for cyber criminals? The quick answer is yes. There have already been demonstrations of how LLMs and ChatGPT can help write malware.
“ChatGPT could easily be used to create polymorphic malware. This malware’s advanced capabilities can easily evade security products and make mitigation cumbersome with very little effort or investment by the adversary”, states a report from security firm CyberArk.
The NCSC is also concerned that an LLM might help someone create malicious tools they would not otherwise be able to. And to assist with cyberattacks and phishing emails.
However, a positive is that current LLMs are more suited to performing simple tasks than complex ones. And the NSCS says that “it’s currently easier for an expert to create the malware from scratch, rather than having to spend time correcting what the LLM has produced”. Still, experts who can create harmful malware will mostly be able to coax an LLM into writing capable malware too.
Even if the risks are piling up, the NCSC say it is best to cautious when considering the type of data to submit when exploring the positive use of LLMs.
