Tesla says it’s spending upwards of $1 billion on its Dojo supercomputer between now and the top of 2024 to assist develop autonomous car software program.
Dojo was first talked about by CEO Elon Musk throughout a Tesla investor day in 2019. It was constructed particularly for coaching machine studying fashions wanted for video processing and recognition to allow the autos to be self-driving.
Throughout Tesla’s Q2 earnings name this week, Musk stated Tesla was not going to be “open loop” on its Dojo expenditure, however the sum concerned will surely be “north of a billion by means of the top of subsequent yr.”
“To be able to copy us, you’ll additionally have to spend billions of {dollars} on coaching compute,” Musk claimed, saying that creating a dependable autonomous driving system is “one of many hottest issues ever.”
“You want the info and also you want the coaching computer systems, the issues wanted to really obtain this at scale towards a generalized resolution for autonomy.”
Musk identified that coaching complicated machine studying fashions wants enormous volumes of knowledge, the extra the higher, and that is what Tesla has entry to, because of all of the telemetry from its autos.
“With respect to Autopilot and Dojo, with a purpose to construct autonomy, we clearly want to coach our neural internet with knowledge from hundreds of thousands of autos. This has been confirmed time and again, the extra coaching knowledge you could have, the higher the outcomes,” he stated.
“It barely works at 2 million [training examples]. At 3 million, it is like, wow, OK, we’re seeing one thing. However then, you get to, like, 10 million coaching examples, it turns into unimaginable. So there’s simply no substitute for enormous quantity of knowledge. And clearly, Tesla has extra autos on the highway accumulating this knowledge than all the different firms mixed. I feel perhaps even an order of magnitude,” Musk claimed.
On the Dojo system itself, Musk stated it was designed to considerably scale back the price of neural internet coaching, and has been “considerably optimized” for the form of coaching that Tesla requires, which is video coaching.
“We see a requirement for actually huge coaching assets. And we predict we could attain in-house neural internet coaching functionality of 100 exaFLOPS by the top of subsequent yr,” Musk claimed, which is various compute energy, to place it mildly.
Dojo is predicated largely on Tesla’s own technology, beginning with the D1 chip that includes 354 customized CPU cores. Twenty-five of those D1 chips are interlinked right into a 5×5 array inside a “coaching tile” module, constructing as much as the bottom Dojo V1 configuration that includes 53,100 D1 cores, in keeping with our colleagues at The Next Platform.
However not stopping there, Musk believes that with all the coaching knowledge at its disposal and a “high-efficiency inference pc” within the automobile, Tesla’s autonomous driving system will quickly make its autos not simply as proficient as a human driver, however finally a lot better. When? He did not say and has form in making grand claims.
“So far, over 300 million miles have been pushed utilizing FSD [Full Self-Driving] Beta. That 300-million-mile quantity goes to look very small, in a short time. And FSD will go from being pretty much as good as a human to then being vastly higher than a human. We see a transparent path to full self-driving being 10 occasions safer than the common human driver,” he claimed.
That is necessary, Musk defined, as a result of “proper now, I consider there’s one thing within the order of one million automotive deaths per yr. And in the event you’re 10 occasions higher than a human, that may nonetheless imply 100,000 deaths, So, it is like, we would slightly be 100 occasions higher, and we wish to obtain as good a security as potential.”
Dojo isn’t the one supercomputer Tesla has for video coaching. The corporate additionally constructed a compute cluster geared up with 5,760 Nvidia A100 GPUs, however Musk stated they merely could not get sufficient GPUs for the duty.
“We’ll really take the {hardware} as quick as Nvidia will ship it to us,” he stated, including: “If they may ship us sufficient GPUs, we’d not want Dojo, however they cannot as a result of they have so many purchasers.” ®
