Microsoft gives peek at HoloLens chip
The HoloLens processing unit (HPU) fuses input from five cameras, a depth sensor and motion sensor, compacting and sending it to the Intel SoC. It also recognizes gestures and maps environments including multiple rooms.
Microsoft described the guts of HoloLens earlier this year, but has not until now publicly detailed its HPU. The company evaluated merchant computer-vision chips including those from Movidius but found none that handled all its algorithms at its performance, latency and power targets.
The TSMC 28nm chip packs 24 Tensilica DSP cores and 8 Mbytes cache into a 12x12mm package with 65 million logic gates. A GByte of LPDDR3 is included in the HPU’s package.
The Tensilica cores were picked in part due to their flexibility. Microsoft added 300 custom instructions to the cores.
“If you can’t add custom instructions, the math density you wind up with is not what you need,” said Nick Baker, a distinguished technologist at Microsoft who described the HPU in a talk at the Hot Chips event held in Cupertino.
The chip uses a mix of standalone accelerators and ones tightly coupled to its DSPs, getting an overall 200x speedup over a software-only version. “We used programmable elements wherever possible and fixed function hardware where we needed it to meet our performance goals,” Baker said.
The code includes a diversity of algorithms with “varying maturity, math, branching and memory-access patterns,” he said.
Separate hardware, software, experience and performance teams worked in parallel. “We took co-design to an extreme with this project,” he said, noting Microsoft has an internal ability to transform simulation tests into emulation routines that can also run on the final hardware.
“This was all going on as Windows 10 was being developed,” he added.
The chip is used in the $3,000 HoloLens developer’s kit released in March. Baker would not comment on any plans for an upgrade of the HPU or release of a consumer headset.
Looking to the future, Baker said designers will push to 8K devices for higher resolution headsets with larger fields of view.
“Foveated rendering [an approach of leaving blurry anything outside a user’s focus area] could become commonplace…and variable resolution displays are possible,” he said. “We need to think about what that means for the whole chain from GPUs, interconnects and panels,” he added.
Related articles:
Microsoft shines light on HoloLens
Volvo choses Microsoft Hololens as strategic sales tool
Flooding young minds through digital immersion
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
