ChatGPT Could Help First Responders During Natural Disasters
A typical model, such as a named entity recognition (NER) tool, would recognize the listed address as three separate entities — Grant Street, Cypress and Texas. If this data was used to geolocate, the model would send first responders not to 1280 Grant St., but into the middle of Grant Street, or even the geographical center of Texas.
Hu says that NER tools can be trained to recognize complete location descriptions, but it would require a large dataset of accurately labeled location descriptions specific to a given local area, a labor-intensive and time-consuming process.
“Although there’s a lack of labeled datasets, first responders have a lot of knowledge about the way locations are described in their local area, whether it be the name of a restaurant or a popular intersection,” Hu says. “So we asked ourselves: How can we quickly and efficiently infuse this geoknowledge into a machine-learning model?”
The answer was OpenAI’s Generative Pretrained Transformers, or GPT, large language models already trained from billions of webpages and able to generate human-like responses. Through simple conversation and the right prompts, Hu’s team thought GPT could quickly learn to accurately interpret location data from social media posts.
First, researchers provided GPT with 22 real tweets from Hurricane Harvey victims, which they’d already collected and labeled in a previous study. They told GPT which words in the post described a location and what kind of location it was describing, whether it be an address, street, intersection, business or landmark.
Researchers then tested the geoknowledge-guided GPT on another 978 Hurricane Harvey tweets and asked it to extract the location words and guess the location category by itself.
The results: The geoknowledge-guided GPT models were 76% better at recognizing location descriptions than GPT models not provided with geoknowledge, as well as 40% better than NER tools. The best performers were the geoknowledge-guided GPT-3 and GPT-4, with the geoknowledge-guided ChatGPT only slightly behind.
“GPT basically combines the vast amount of text it’s already read with the specific geoknowledge examples we provided to form its answers,” Hu says. “GPT has the ability to quickly learn and quickly adapt to a problem.”
However, the human touch — that is, providing a good prompt — is crucial. For example, GPT may not consider a stretch of highway between two specific exits as a location unless specifically prompted to do so.
“This emphasizes the importance of us as researchers instructing GPT as accurately and comprehensively as possible so it can deliver the results that we require,” Hu says.
Letting First Responders Do What They Do Best
Hu’s team began their work in early 2022 with GPT-2 and GPT-3, and later included GPT-4 and ChatGPT after those models launched in late 2022 and early 2023, respectively.
“Our method will likely be applicable to the newer GPT models that may come out in the following years,” Hu says.
Further research will have to be done to use GPT’s extracted location descriptions to actually geolocate victims, and perhaps figure out ways to filter out irrelevant or false posts about a disaster.
Hu hopes their efforts can simplify the use of AI technologies so that emergency managers don’t have to become AI experts themselves in order to use these them and can focus on saving lives.
“I think a good way for humans to collaborate with AI is to let each of us focus on what we’re really good at,” Hu says. “Let AI models help us complete those more labor-intensive tasks, while we humans focus on gaining knowledge and using such knowledge to guide AI models.”
Tom Dinki is News Content Manager, physical sciences, economic development, at the University of Buffalo. The article was originally posted to the website of the University of Buffalo.
