Tim Sweeney

Start here

Episode highlight

Humans are by far the hardest part of computer graphics because millions of years of evolution have given us dedicated brain systems to detect patterns in faces and infer emotions and intent because cavemen had to, when they see a stranger, determine whether they were likely friendly or they might be trying to kill them. And so people in the world have extraordinarily detailed expectations of a face and we can notice imperfections, especially perfections arising from computer graphics limitations. Okay, one part is capturing humans and so [inaudible 00:00:33] really advanced, dedicated hardware that puts a human in a capture sphere with dozens of cameras in them taking high resolution, high frame rate video of them as they go through a range of motions. And then capturing the human face is complicated because the nuanced detail of our faces and how all the muscles and sinews and fat work together to give us different expressions. So it's not only about the shape of a person's face, but it's also about the entire range of motion that they might go through. So that's the data problem. There's a lot of other problems with computer graphics. There's technology for rendering hair, which is really hard. Because you can't render every... Again, we know the laws of physics. It would be easy to just render every hair. It would just be a billion times too slow. So you need approximations that capture the net effect of hair on rendering and on pixels without calculating every single interaction of every light with every strand of hair. That's one part of it. There's detailed features for different parts of faces. There's subsurface scattering because we think of humans as opaque, but really our skin, light travels through it. It's not completely opaque, and the way in which light travels through skin has a huge impact on our appearance.