Microsoft Develops First Person Hyperlapse Video Stabilization Algorithm
Immediately after watching Microsoft’s First Person Hyperlapse video posted last week my imagination started thinking of all the implications for future healthcare simulation debriefings. What does Microsoft first person hyperlapse video stabilization algorithm do? It takes non-stabilized first-person video footage which is very sped-up and makes it watchable by forcing the smoothing over of non-dominant pixels. Still sound confusing? Watch the video below to see three examples of how this software can help show a first person story to life:
About the Project Built By Microsoft Researchers Johannes Kopf, Michael Cohen & Richard Szeliski:
We present a method for converting first-person videos, for example, captured with a helmet camera during activities such as rock climbing or bicycling, into hyper-lapse videos, i.e., time-lapse videos with a smoothly moving camera. At high speed-up rates, simple frame sub-sampling coupled with existing video stabilization methods does not work, because the erratic camera shake present in first-person videos is amplified by the speed-up. Scene Reconstruction: Our algorithm first reconstructs the 3D input camera path as well as dense, per-frame proxy geometries. We then optimize a novel camera path for the output video that is smooth and passes near the input cameras while ensuring that the virtual camera looks in directions that can be rendered well from the input.
Consider how this technology could be useful in hospital, military or healthcare austere environments to re-watch hours and hours of footage quickly and seamlessly while still being able to gather the major story highlights. While minuscule details will be missed, the raw footage itself will still be available to explore those moments. This algorithm will allow for massive review of huge amounts of video data for debriefing, which could have major implications for long-duration events like military engagements, five alarm fires, surgeries or other major events.
Read the research paper and watch high-resolution versions of various experimental videos at Microsoft’s Research Website.