Natal: How It Works

Microsoft have been understandably secretive about Natal but have revealed how the device works to New Scientist. A player using Natal can stand from 0.8 to 4 metres away from the device and will be illuminated in an infra red light. A monochrome camera is used to record how much of the infra red light is bouncing off the player and the brightness of this picture is used to ‘approximate’ the players distance and moves.

Two things to note, firstly, Natal has a monochrome camera so you won’t be seeing any games such as Eyepet where players appear on screen and you can pretty much forget about using it for video chat unless you are Laurel & Hardy. Natal has a second standard ‘RGB’  camera not mentioned in the article, my bad! Secondly is that word ‘approximation’.

The ‘brain’ of Natal has been created by feeding it terrabytes of data featuring people in various poses, resulting in the software being able to recognise 31 different body parts. Natal’s lead developer Alex Kipman explains,

“When we train this ‘brain’ we are telling it: this is the head, this is the shoulder. And we’re doing that over millions of frames. When it sees a new image it can tell you the probability it’s seeing a certain body part based on that historical information.”

Natal also has an understanding of the human body, for example it knows that hands should be attached to arms and your feet should not be anywhere near your shoulders unless your wife is feeling kinky.

“It correctly positions your hand even if it’s held behind your back,” Kipman says. “It knows the hand can only be in one place. That’s important because during multiplayer games there won’t always be a clear view of both players at all times.”

In terms of processing power, Natal will use 10-15% of the Xbox computing power and will take only 10 milliseconds to recognise a pose and 160 milliseconds to recognise a new person. There is one last number for you to consider – Natal will recognise body parts to within a  4cm cube.

Suddenly the reasons why Sony has been harping on about how the PS Motion Controller can track within millimetres are obvious;  4cm is not exactly accurate. Combine this with ‘approximated’ distances and Natal guessing the ‘probability’ a body part is in a certain location it’s very clear it is not a precise system. Microsoft are gambling that having no controller at all will attract more casual gamers to Xbox but are sacrificing some accuracy for easy of use.