Drone Piloting Proficiency Takes Flight with Certification Course

“When we started, there was no measurement science or standards infrastructure available to objectively evaluate drone capabilities or remote pilot proficiency, so we filled that void,” said Adam Jacoff, NIST project leader for Emergency Response Robots and chair of the ASTM E54.09 Subcommittee on Response Robots. “After helping to guide purchases, these standard tests then support credentialing of remote pilots. Although these drone test methods are specifically designed to help emergency responders and public safety organizations maintain a safe operational standoff while performing extremely hazardous tasks, they similarly support a wide variety of commercial and industrial applications. All pilots flying in the national air space need to demonstrate they can maintain positive aircraft control while performing operational tasks in complex and often hazardous environments.”

The course includes both open and obstructed test lanes, meaning with and without line-of-sight, as well as realistic operational scenarios—all using established assessment standards for scoring. Flights are conducted both during the day and at night so pilots can be ready for anything. As vigorous as it is, the tests are inexpensive (using equipment like plastic buckets available at any hardware store), easy to conduct, and require less than 30 minutes to complete. Attendees learned how to fabricate the test apparatuses, conduct trials, and embed the same scoring tasks into their own training scenarios. Upon completion, they know everything they need to replicate the course back at their home agencies.

One of the realistic operational scenarios that participants played out in Maryland was based on an actual emergency call from earlier this summer involving an abducted child and an abandoned vehicle in a rural area.

As Corporal James Lantz with the Maryland State Police put it, “The scenario that we’re doing out there, the open area search, is basically the exact same scenario of a mission that we flew about two months ago. We had an Amber Alert. The vehicle was found in a field…surrounded by trees and everything like that. So, we can sit there and say… ‘This is exactly what we faced. These were the hazards that we had. These are the issues that we had.’ We replicated that here, in a controlled environment, a testable environment that we can use to measure that these guys are able to perform.”

Drone operators were instructed to thoroughly inspect the vehicle as if searching for any signs of people, weapons, or even explosives within. Participants were required to take photos of the various targets tapped to the car with their drone’s camera.

To become certified as a proctor, participants must pass a written quiz with a grade of at least 80% and submit scores from trials they proctored for other remote pilots. The officers work in teams of three, rotating through each role of pilot, proctor, and visual observer. The pilot maintains control of the drone, calling out each intention of movement before doing so and calling out each bucket alignment as they make their way through the obstacles. The proctor scores the pilot. The visual observer maintains sight of the aircraft and surroundings, repeat’s the pilot’s intention of movement to confirm, and calls out corrections and warnings as necessary.

In addition to testing the skills of the drone pilots, the course is a great way to see what the drones can do. As David Battaly from the Mississippi Emergency Management Agency said, “It’s a practical measure of not only the pilot’s proficiency and ability to operate his aircraft, but the capabilities of that individual aircraft. There’s a myriad of drones on the market, and each drone is a little bit different. So, it measures the pilot’s proficiency, how well he can manipulate his drone, and the capabilities of what his drone can actually achieve.” Seeing how different devices perform helps inform procurement decision making so law enforcement agencies can purchase the best tool for the job.

The course also helps manufacturers improve their products. Private companies are able to participate in the same training, running the same scenarios, but instead use it as an opportunity to de-bug their systems and see how they compare to other devices on the market. A manufacturer is able to run through the evaluation process, determine the best that their system can do, and identify ways to boost performance. This may mean realizing they need more robust optics or a more intuitive user interface so pilots will have an easier time using their device, score higher on the course, and be more likely to recommend it to colleagues.

Thanks to the easily repeatable nature of the course and the uniformity and consistency offered by the standards, drone operators across the country will be able to compare “apples to apples” when assessing capabilities during a response. This approach encourages collaboration and ultimately leads to a safer public.

“It’s important because we want to have certified pilots to fly these drones to make sure they operate safely and proficiently,” added Chu. “But more than that is now that we have standards, we have common ground to talk about these capabilities. And these standards are not stagnant. We need to revise them every two or three years, but we can always have the latest, the best test methods for our operators and for our first responders. So that’s the beauty of using these international standards for these training facilities because they are well tested, they’re well accepted, and even after we are gone, these standards will still be there.

The drone standard test methods are expected to be accepted by ASTM by spring of 2023 and next up will be underwater test methods for aquatic robots.