Open AI’s ChatGPT and its ilk have dominated headlines this year, captivating billionaires, fans, regulators, and doomsayers. much of the recent coverage also reveals why the Pentagon pursues quite different AI approaches: military leaders must have tools that they can trust.
One big reason ChatGPT and other very large language models are so good at mimicking human writing is the same reason they are given to deceit: the neural network at the heart of the AI is fueled by scraping information from millions of websites. While Open AI hasn’t disclosed what websites it used to train its tools, a recent Washington Post investigation looked at 15 million websites that researchers have used to train similar models. Unsurprisingly, this large mass of data contains much that isn’t true–and solarge language and generative AI models frequently lie.
Even if you were to train large language models on a carefully selected pool of websites, you might still run into “artificial hallucination“: “the phenomenon of a machine, such as a chatbot, generating seemingly realistic sensory experiences that do not correspond to any real-world input.”
So DOD is being very careful about using such tools.
“We are not going to use ChatGPT in its present instantiation. However, large language models have a lot of utility,” Maynard Holliday, DOD’s deputy chief technology officer for critical technologies, said Thursday at the Defense One Tech Summit. “We will use these large language models, these generative AI models, based on our data. And so they will be tailored with Defense Department data, trained on our data, and then also on our compute–either our compute in the cloud and or on [premises] so that it’s encrypted, and we’re able to, essentially…analyze, its feedback.”
This week, Holliday said, the Defense Department will convene a gathering “to get after, you know, just what the use cases are; just what the state of the art is in the industry, and academia.”
DOD also needs to get better at structuring and sharing data, even two years after a seminal directive on the matter, saidMike Horowitz, who leads the Emerging Capabilities Policy Office in Office of the defense undersecretary for policy.
“You need good data, like data that’s applicable to the questions that you want to use AI to answer,” Horowitz said. This data must be tagged and cleaned. The process can be time-consuming. This process was. I think,…challenging. And it’s been challenging because we build all of these sort of pipes of data that were designed to be independent from each other.”
Commanders aren’t going to trust a tool unless they can understand how it was trained and on what data, Holliday said.
“Back in 2015, when I was on the Defense Science Board doing a study on on autonomy, when we briefed to our combatant commanders they said, ‘You know, this is great, potentially game changing, but…we’re not going to use it unless we can trust it,'” he said.
Building Trust
Anyone can play around with ChatGPT to figure out how much to trust it for a given use, but DODis taking a more formal route.
“Initial trust can be gained from design and development decisions through soldier touchpoints and basic psychological safety and continuously calibrated trust through evidence of effectiveness and feedback provided by the system during integration and operation. And so challenges exist in measuring the warfighters’ trust, which require additional research and understanding to define what influences that,” he said.
In reality, this looks a great deal like the exercises that CENTCOM has been undertaking. They bring together service providers and AI operators in a series of games, evaluations, and assessments centered around emerging technologies.
Exercises like Scarlet Dragon Oasis and Falcon Oasis are structured differently from the traditional military training game, said Schuyler Moore, U.S. Central Command’s Chief Technology Officer. These new tech-focused CENTCOM exercises occur in quick succession and are geared around innovating the technology based on soldier feedback as much as building operator skills, Moore said at the Tech Summit. Getting operators and builders collaborating as part of the exercises is also a key component.
“These exercises are intended to follow best practices in the private and software sector. You do them in sprints, you iterate and repeat these over and over to improve with time,” said she. “So for the for the exercise that we’re doing right now, there is a muscle memory that we’re building, iterating back and forth with a software developer and not saying whatever software capability I’ve been handed is static…The expectation now is that you can and will poke holes in it, share your feedback, iterate with the team, continue to give your feedback every single time and that-to be frank-has been a cultural mindset shift because exercises previous have never given people the opportunity to experiment with that type of activity.”
Andrew Moore, a CENTCOM advisor on AI, robotics, cloud computing, and data analytics, came to the command from Google, where he worked on a variety of AI related projects, including Project Maven, which is seen as a model for how the military might develop human-AI teams in the future.
CENTCOM played a key role in launching Maven, as many analysts had the job of sifting through hours and hours of drone data to understand how different people on the ground were behaving and which ones might pose a threat.
The command is working to take that sort of research further, to enable AI engines to make better sense of the objects picked up by drones, Moore said.
“The next real question is making sure that you’re able to do inferences about what’s really happening based on finding relationships between all these dots on maps,” he said.
A breakthrough AI application for CENTCOM in the years ahead will likely look less like a flashy –and buggy-text generator and more like a knowledge graph, which Moore worked on at Google. A knowledge graph works to structure rapidly incoming data according to a rough concept of their properties and relationships between objects. A knowledge graph is responsible for the recommendations you see on social media websites.
“Knowledge graphs…in my opinion, are what’s creating these trillion dollar companies that you see on the West Coast of the United States,” he said.
But for CENTCOM, Moore invisions building an engine to understand relationships between objects on a much deeper level, allowing command staff to see connections that will illuminate the battlefield and all the objects in it in ways that adversaries are trying to keep hidden, or may not even be aware of.
” “I believe that will be one of our unifying themes,” Moore said. “It’s the absolute importance not just being able to ingest large amounts of data, being able to normalize it in a way where we can actually do inferences so that, perhaps, it’s not just that this ship on the ocean is taking a strange trajectory, but also…their financing, or perhaps by their ownership, or other secondary or tertiary really ternary information like that.”
The post The Pentagon’s Ambitious AI Plans Look Less and Less Like ChatGPT appeared first on Defense One.
