Teams of computers and humans more effective in disaster response

OSB: Can you describe the kind real-world problems you’ve been applying that thinking to?

SR: As just one example, crisis responders need to know the extent of a natural disaster, what aid is required and where they need to get to as quickly as possible. This is what’s known as “situation awareness.” With the proliferation of mass media, a lot of data is now generated from the disaster zone via photographs, tweets, news reports and the like. With the addition of first responder reports and satellite images of the disaster area, there is a vast amount of relevant unstructured data available for situation awareness. A crisis response team will be overwhelmed by his data deluge — perhaps made even worse by reports written in languages they don’t understand.

But the data is also hard to interpret by computers alone, as it’s difficult to find meaningful patterns in such a large amount of unstructured data, let alone understand the complex human problems that described within it.

OSB: How can you use HACs to help?

SR: Firstly, we can farm out satellite images and text to the “crowd.” People want to help and they will happily use their skills to interpret a small number of text samples or satellite images. Computers can then build a model connecting features in the data to the interpretations supplied by the crowd. The computer can then use this model to trawl through the rest of the data and “interpret” what it sees using these features. The computer decides what data to farm out to the crowd and who should be recruited from the crowd based on their reliability. The individuals in the crowd can decide if they want to take part and what tasks they are prepared to do. The computer aggregates the crowd responses intelligently and, in so doing, determines their individual reliabilities automatically. So we can use combinations of humans and computers to successfully aggregate and interpret vast amounts of unstructured data. This is just one example of where HACs can be used in disaster response — another is the coordination of a vast fleet of UAVs visually mapping aid requirements across the disaster area.

OSB: Have you been able to try any of these ideas in the real world?

SR: We’ve implemented the first approach I just explained, actually. It’s in a system called “CrowdScanner,” and we used it for real immediately after the first major Nepal earthquake in April of this year. We used the crowd to locate settlements from satellite images, identified settlements that were not mapped on open sources such as OpenStreetMap, and our Search and Rescue partners deployed teams to reconnoiter these settlements.

OSB: Was it successful? Did you run into any pitfalls when marrying up computers and humans?

SR: First of all, the good news! It is not difficult to find a competent crowd to help out in a disaster situation. We are, however, finding it really difficult to build systems that are relevant to the disaster response community — mainly because we are trying to guess what their critical information requirements might be.

We were able to respond to the Nepal crisis because we just happened to have a working platform that we could adapt to the Nepal situation along with a satellite data source, and Rescue Global, who we were working with, were able to marry this data with their requirements of water filter placement and life detector placement. We were able to respond in a timely manner as a result.

In general, though, we’re in the dark as to the generic problems faced by the crisis response community and specifically where we can help. Although we’ve attended field exercises with various disaster response organizations we need to sit down with them. That way, we’ll be able to abstract information about where they work and their requirements to the level where we can start designing generic situation awareness algorithms to help them.

OSB: So what’s next for the disaster response work?

SR: Imagine a service where people could post their resources, such as the availability of an aircraft or their plan to visit a location in the disaster area. This resource could be married with a crisis response team who want to use it to achieve their own goals, and machine learning can be used to link the responders’ requirements to the people with the resources using crowd interpretations of the resource providers’ offers. Another idea is that we could try to determine the probability of life in a collapsed building after an earthquake, to help responders prioritize their search. Crowdsourced interpretations of drone or satellite footage could be used to identify salient features and machine learning can then determine the probability of life. 

There are many ways machine learning can be used in disaster response. The key now is to sit down with crisis responders and develop relevant data processing algorithms that will actually save lives.