Voice forensics to help Coast Guard nab hoax callers
“We have a great partnership with S&T, and we knew CMU had the capability for voice forensics,” said Alan Arsenault, chief of the Command, Control, Communications, Computers, Intelligence, Surveillance and Reconnaissance branch at the Coast Guard Research and Development Center.
“We looked at other technology such as mobile direction-finding capabilities and social media analyses tools, but this was really panning out to give us some great details for building a case file,” Arsenault said.
In the event of a hoax call, the towers of Rescue 21, a communications management system for the Coast Guard, triangulate a geographical area. Coast Guard Investigative Service special agents will interview local mariners and seek information from local law enforcement in that area, seeking any information that may be useful in identifying the caller. The Coast Guard can even have the recorded call played by the local media to see if anyone recognizes the voice. “It is basically good old detective work - only we are trying to identify a logical suspect based on a voice,” Martinez said. Then Dr. Singh comes in.
The Coast Guard specifically asked Singh to run the hoax calls through her repository, said Matthew Clark, Director of the S&T’s OUP. “She came up with this long list of things she could tell S&T and the Coast Guard about the physical characteristics of the perpetrator and his environment.” The list included descent, age, height and weight background noise and the type of room where the call was made. “It is all based on voice and noise.”
“Profiling humans from their voice is a new area of research in voice forensics that is massively driven by recent advances in artificial intelligence,” said Singh.
The only evidence unidentified hoax callers leave behind initially is the raw recording. However, Singh’s team can deduce many aspects of the callers using artificial intelligence techniques. Besides the obvious age, gender and ethnicity, they can also glean physical characteristics, health, emotional state, and even a person’s educational level and socioeconomic status. The researchers can also profile the place from which the perpetrator makes the call – possible location, room size, type of objects present and the material of the walls, ceiling and floor – all based on background noise and voice echo.
“All this is useful information when you start identifying potential suspects and ask, ‘Does this person meet the characteristics Dr. Singh’s team has pointed out?” Martinez said.
But the Coast Guard still has to identify logical suspects. They go back to Singh with recordings of potential callers to match to the hoax call. If there is no match, Martinez’s team has to reevaluate. If there is a match, they have to develop the evidence to place the suspect with the device at the location where the hoax call was made. “Although there is more investigative work to do after a voice match is made, now we are more focused in our efforts based in part on the voice analytics provided by CMU and Dr. Singh,” he said.
This collaboration is already producing results. “In several cases, Singh not only helped exclude potential suspects, but she also narrowed down our suspects to one or two individuals,” Martinez said.
Not a single case with voice forensic evidence has gone to trial yet, but Martinez thinks they are getting very close. He believes that very soon he will be able to include voice analytics as part of the evidence package in court. Currently, voice analytics has helped the Coast Guard focus on a suspect from the Maryland area. “We are slowly bringing the case to conclusion and hopefully prosecution.”
If the investigations are successful, this could lead to more widespread use of voice forensics. The resulting evidence could be used in well-publicized criminal prosecutions to deter other would-be hoaxers.
“Voice forensics is at the tipping point of being used as a piece of evidence, similarly to back in the day when DNA was not yet accepted as evidence,” said Arsenault.
