Port securityEnsuring Reliability of Air Cargo Screening Systems
DHS, which is responsible for ensuring the security of air cargo transported to the United States, says the threat from explosives in air cargo remains significant. A new GAO report addresses how DHS secures inbound air cargo, and the extent to which TSA’s field assessment of a CT screening system included key practices for design and evaluation.
According to DHS, which is responsible for ensuring the security of air cargo transported to the United States, the threat from explosives in air cargo remains significant. The TSA Modernization Act includes a provision for GAO to review DHS’s processes for securing U.S.-bound air cargo and efforts to use CT technology for air cargo screening.
A new GAO report addresses, among other things, how DHS secures inbound air cargo, and the extent to which TSA’s field assessment of a CT screening system included key practices for design and evaluation. GAO reviewed TSA and CBP air cargo security procedures and documents and analyzed a random sample of air cargo shipment data from calendar year 2019. GAO also interviewed TSA and CBP headquarters and National Targeting Center officials, and interviewed TSA field and air carrier officials regarding operations with two foreign airports, selected based on TSA risk data and the amount of air cargo transported from these airports to the United States.
In May 2021 GAO issued the public version of the report, noting that information that DHS deemed sensitive was omitted.
What GAO Found
The Department of Homeland Security’s (DHS) Transportation Security Administration (TSA) and U.S. Customs and Border Protection (CBP) address U.S.-bound air cargo security through separate programs and have taken steps to measure their effectiveness. For example, TSA conducts an inspection program to help ensure that air carriers comply with specific cargo-related security requirements, such as requirements related to cargo acceptance, control and custody, and screening procedures.
From January 2020 through April 2021, TSA conducted a field assessment on the use of a computed tomography (CT)-based explosives detection system to screen air cargo as part of its ongoing process to qualify the system for use by air carriers. This type of system produces images of parcels that are examined by computer for signs of explosives.
TSA assessment, however, did not fully meet three of five key design and evaluation practices. While the assessment identified goals and established metrics, TSA did not incorporate other key practices, such as collecting all necessary data about the system’s ability to detect threats (probability of detection) in the field, consistent with TSA’s standards. Since TSA officials cannot use live explosives in the field to measure the probability of detection, they relied on image quality testing, using a manufacturer’s test kit to compare system performance in the field with earlier tests performed in a laboratory with live explosives.
GAO says, however, that TSA did not validate that the test kit was an acceptable alternative test method for determining the CT system’s probability of detection in the field. TSA did not (1) independently validate that the test kit captures all ways system performance could degrade or (2) collect any of the underlying quantitative data from the test kit.
TSA officials told GAO they did not validate the test kit because its performance was certified during laboratory testing at DHS’s Transportation Security Laboratory; however, officials from the Transportation Security Laboratory told GAO they do not certify the performance of test kits. “Without a suitable alternative testing approach to determine the probability of detection, TSA will not have all relevant data to assess whether the CT system meets TSA’s detection standard requirements in the field and should be qualified for use by air carriers,” TSA says
GAO Recommends
In the May 2021 report, GAO made four recommendations:
· TSA and CBP should establish a documented process to ensure that relevant officials from both agencies are aware of and have access to the relevant data related to their risk assessment of inbound air cargo.
· Before designating the explosives detection system for air cargo screening — a system which is now being evaluated — as “qualified” on the air cargo screening technology list, TSA should use additional analysis to verify that the probability of the system’s in-the-field detection matches the performance measured in laboratory testing.
· GAO says TSA, to provide this verification, should use either live explosives testing or, when this is not possible, use an independently validated, fully documented alternative testing strategy.
· TSA should make sure that proper statistical techniques are used to analyze data from TSA field assessments of explosives detection systems for air cargo screening. This statistical analysis should include calculating error values for each quantitative measurement, identifying the necessary performance thresholds, and comparing the measured values and errors against each threshold.
