https://youtu.be/72N3or0ZgMU
If Pokemon Go was the last time you’ve looked at Augmented Reality, look again. With the iOS 11 announcement at this year’s WWCD, Apple introduced ARKit, which is their take on Augmented Reality. Developers now have their hands on the tech and they’re showing some really impressive demos. You can see a few in the video above. The tracking is one of the best I’ve seen so far. It’s worth mentioning that they achieve that with the camera and the accelerometers only. I believe down to iPhone 6s. So as opposed to Google’s Tango project, Apple’s implementation requires no special 3D hardware. Once iOS 11 is out, this fact alone will open up their AR ecosystem to many millions of people at once.
Apple seems to have several things solved in their implementation: camera- and accelerometer based inside-out mapping, surface detection, light detection and mapping. Unfortunately most demos don’t show any occlusion (that’s when something goes between the camera and the “holograms”) which is of course a lot harder based on just the camera. However, their future devices are almost guaranteed to have 3D sensors. Remember that back in 2013, Apple bought Prime Sense, the company that invented the Microsoft Kinect. Since then they had four years behind closed doors and with really good R&D funding to miniaturize and refine their already small 3D sensor even further. Four years ago, right before Prime Sense disappeared behind Apple’s well-guarded lab doors, their sensor was the size of a stick of gum. With the AirPods, Apple has proven their ability to make cool tech really really small.
What’s also to come is object recognition. Apple’s Photos app already recognizes over 4000 different objects using machine learning. While Photos is impressive enough to be able to tell cats and dogs apart most of the time, once they open that up for ARKit, this will become even more precise because it has the advantage of 3D, either through camera-based photogrammetry or in the future through their 3D sensors.
While ARKit will initially be available on iPhones and iPads, I fully expect them to come up with some head-worn gear either this year or in 2018. Glasses sound reasonable and they will take an already impressive experience several notches up.