Portable, super-high-resolution 3-D imaging

GelSight to check the integrity of their products. The technology has also drawn the interest of experts in criminal forensics, who think that it could provide a cheap, efficient way to identify the impressions that particular guns leave on the casings of spent shells. There could also be applications in dermatology — distinguishing moles from cancerous growths — and even biometrics. The resolution provided by GelSight is much higher than is required to distinguish fingerprints, but “the fingerprinting people keep wanting to talk to us,” Adelson says.

Although GelSight’s design is simple, it addresses a fundamental difficulty in 3-D sensing. Johnson illustrates the problem with a magnified photograph of an emery board, whose surface, in close-up, looks a lot like marmalade — a seemingly gelatinous combination of reds and oranges.

“The optical property of the material is making it very complicated to see the surface structure,” Johnson says. “The light is interacting with the material. It’s going through it, because the crystals are transparent, but it’s also reflecting off of it.”

When a surface is pressed into the GelSight gel, however, the metallic paint conforms to its shape. All of a sudden, the optical properties of the surface become perfectly uniform. “Now, the surface structure is more readily visible, but it’s also measurable using some fairly standard computer-vision techniques,” Johnson explains.

GelSight grew out of a project to create tactile sensors for robots, giving them a sense of touch. Adelson and Johnson quickly realized, however, that their system provided much higher resolution than tactile sensing required.

Once they recognized how promising GelSight was, they decided to see how far they could push the resolution. The first order of business was to shrink the flecks of metal in the paint. “We need the pigments to be smaller than the features we want to measure,” Johnson explains. The different reflective properties of the new pigments, however, required the use of a different lighting scheme, and that in turn required a redesign of the computer-vision algorithm that measures surface features.

“I think it’s just a dandy thing,” says Paul Debevec, an associate professor of graphics research at the University of Southern California. “It’s absolutely amazing what they get out of it.” Debevec’s lab has been investigating the use of polarized light to compensate for the irregular reflective properties of some surfaces, but, he says, “they’re getting detail at the level that’s, for little patches, well more than an order of magnitude better than I’ve ever seen measured for these kinds of surfaces.”

As a graphics researcher, Debevec — whose Ph.D. thesis work was the basis for the effects in the movie “The Matrix” — is particularly interested in what GelSight will reveal about the surface characteristics of human skin. “This kind of data is absolutely necessary to simulate that accurately,” Debevec says. “It’s pure gold.”