Making computer vision systems that work: Boujou, Kinect, HoloLens

I have been lucky enough to have been involved in the development of real-world computer vision systems for over twenty years. In 1999, prize-winning research from Oxford University was spun out to become the Emmy-award-winning camera tracker “boujou”, which has been used to insert computer graphics into live-action footage in pretty much every movie made since its release, from the “Harry Potter” series to “Bridget Jones’s Diary”.

In 2007, I was part of the team that delivered human body tracking in Kinect for Xbox 360, and in 2015 I moved from Microsoft Research to the Windows division to work on Microsoft’s HoloLens, an AR headset brimming with cutting-edge computer vision technology.

In all of these projects, the academic state of the art has had to be leapfrogged in accuracy and efficiency, sometimes by several orders of magnitude. Sometimes that’s just raw engineering, sometimes it means completely new ways of looking at the research. If I had to nominate one key to success, it’s a focus on, well, everything: from low-level coding to algorithms to user interface design, and on always being willing to change one’s mind. 

Short biography

Andrew Fitzgibbon is a scientist with HoloLens at Microsoft, Cambridge, UK. He is best known for his work on 3D vision, having been a core contributor to the Emmy-award-winning 3D camera tracker “boujou” and Kinect for Xbox 360, but his interests are broad, spanning computer vision, graphics, machine learning, and occasionally a little neuroscience.

He has published numerous highly-cited papers, and received many awards for his work, including ten “best paper” prizes at various venues, the Silver medal of the Royal Academy of Engineering, and the BCS Roger Needham award. He is a fellow of the Royal Academy of Engineering, the British Computer Society, and the International Association for Pattern Recognition. Before joining Microsoft in 2005, he was a Royal Society University Research Fellow at Oxford University, having previously studied at Edinburgh University, Heriot-Watt University, and University College, Cork.