The news was the big surprise saved for the end of a two-hour keynote at the search giant’s annual Google IO event in the heart of Silicon Valley.
“We have started building tensor processing units…TPUs are an order of magnitude higher performance per Watt than commercial FPGAs and GPUs, they powered the AlphaGo system,” said Sundar Pichai, Google’s chief executive, citing the Google computer that beat a human Go champion.
The accelerators have been running in Google’s data centers for more than a year, according to a blog by Norm Jouppi, a distinguished hardware engineer at Google. “TPUs already power many applications at Google, including RankBrain, used to improve the relevancy of search results and Street View, to improve the accuracy and quality of our maps and navigation,” he said.
The chips ride a module that plugs into a hard drive slot on server racks. Engineers had them running just 22 days after they tested first silicon said Jouppi, who previously helped design servers and processors at Hewlett Packard and Digital Equipment.
Given the nature of AI algorithms, “the chip [can] be more tolerant of reduced computational precision, which means it requires fewer transistors per operation…[and thus] can squeeze [in] more operations per second,” he said. The project started several years ago. Google has been hiring engineers with semiconductor expertise for some time. However, it managed to keep secret what they were working on, despite the fact the chips are already running in systems.
The company is not the first to design an accelerator specifically for AI. Nervana Systems is preparing a cloud service that will be based on its own AI accelerators. Movidius has its own merchant chip for embedded applications, and recently announced plans for a high-end version.