Reconstructing 3D face from a single 2D image

in profile, you can recover the shape of the nose quite well because you get to see the outline. If you view it from the front then the nose shape is not so good, because there is not as much information there,” said Smith. “The issue of face expression is quite difficult with morphable model approaches — you would need hundreds of examples of every expression. So we are looking at using shading information to recover 3D shapes when they might be in a completely different expression, then we can feed that back into the model so the model will improve and learn to recognise these images.”

The shape-from-shading technique is better at recovering local features, but Smith said the global 3D shape recovered in this technique was of inferior quality to that recovered using a morphable model, hence the aim to combine the methods. By combining techniques the researchers may also be able to speed up the time it takes to fit the model to an image. According to published results, the morphable model now takes four and a half minutes to fit to an image. “When you fit a morphable model to an image, all you look at is an error value, which tells you how different your predicted appearance and your actual image is, and you are just trying to minimize that error,” said Smith. “That is quite difficult and takes a lot of computational power, so it is a really slow process. The shading information allows us to take the 3D shape out of the image and use that as a rough fit of the morphable model directly. That gives us an approximate fit of the morphable model and we can iterate those two processes. This may allow us to fit the model much quicker.”

In addition to using the software to reconstruct shapes from CCTV images, Smith said the software could also be used in the gaming industry or to quantify the success of plastic surgery operations. “In graphics animations, being able to take an image of someone and get a 3D model that you can stick in a computer game is useful. There’s also things like reconstructive plastic surgery. If someone is involved in an accident and the tissue of their face is damaged, in that case all you have is images of them prior to the accident, so it would be quite useful to be able to recover a 3D shape from an old image, then use that and a scan of the damaged face to make suggestions about what sort of surgery should be done,” he said.