The recent advent of artificial intelligence (AI)-generated chat programs or large language models (LLMs), such as ChatGPT and other LLMs, provide a revolutionary new capability that inspires both awe and trepidation.
OpenAI and similar LLM developers argue that the new technology is a much-needed tool. For example, they see it as a time-saving method to help writers create content or website developers write code.
Alternatively, critics have pointed out that LLMs present serious problems that need to be addressed. These include, among other issues, problems with accuracy and LLM-generated content that is biased and, sometimes maliciously, distorted.
The problems ChatGPT and other LLMs have experienced are not exclusive to the private sector though. Indeed, the Intelligence Community (IC) is also wrestling with the same issues—and trying to take advantage of the promise—LLMs present.
To mitigate these concerns, the Intelligence Advanced Research Projects Activity (IARPA) has developed the Bias Effects and Notable Generative AI Limitations (BENGAL) targeted seedling program. The BENGAL program aims to understand LLM threats, quantify them, and find novel methods and technologies to address threats and vulnerabilities or to work effectively with imperfect LLMs.
BENGAL’s genesis stems from U.S. government interest in safe LLM use for a wide variety of applications, including rapidly summarizing and contextualizing information relevant to the IC. However, these applications must avoid unwarranted biases and toxic outputs, preserve source attribution, and be error free. The U.S. government is also interested in identifying and mitigating malicious, adversarial LLM use.
“The inherent characteristics of LLMs—such as ease of use, human-like dialogue, and complexity—also make them vulnerable to hostile applications,” said BENGAL Program Manager, Dr. Tim McKinnon. “LLMs may conceal threats to users, including quick generation of mis/disinformation or elicitation of sensitive information.”
BENGAL performers will represent a range of academic and private-sector organizations with deep experience in AI and LLM technology.
An independent testing and evaluation team will verify performer results and validate software performance.
BENGAL is structured into two phases, with a phase A and an option for a phase B if the initial phase is successful. Phase B, if exercised, will build upon the proof-of-concept research in phase A to deliver a technology demonstration. Each phase will last 12 months.
Once developed, BENGAL technology will work something akin to an anti-virus scan on a computer, probing LLMs for nefarious code and applications. That said, how exactly the IC or other agencies may use this technology will be up to them.
“It may be disconcerting to some, but LLMs increasingly affect the way we work and interact, and the IC recognizes this and that we need to safely take advantage of this new technology to advance our mission priorities,” Dr. McKinnon said. “The flip side is if we don’t find ways to ensure safe adoption of this technology, we will miss out on opportunities to make the IC substantially more effective and efficient in its mission to inform decision-makers.”
