How AI Can Keep Accelerating After Moore’s Law – MIT Technology Review https://apple.news/A4Rxkgl8lN-WmsspWrttgtQ
The sudden thirst for new power to drive AI comes at a time when the computing industry is adjusting to the loss of two things it has relied on for 50 years to keep chips getting more powerful. One is Moore’s Law, which forecast that the number of transistors that could be fitted into a given area of a chip would double every two years. The other is a phenomenon called Dennard scaling, which describes how the amount of power that transistors use scales down as they shrink.
In the longer term, more radical changes in how computer chips work will be required to keep AI getting more powerful. Creating chips that don’t add accurately is one option. Prototypes have shown that they can make computers more efficient without undermining the accuracy of results from machine-learning software (see “Why a Chip That’s Bad at Math Could Help Computers Tackle Harder Problems”).
Chip designs that directly copy from biology could also be crucial. IBM and others have built prototype chips that compute using spikes of current, similar to how our neurons fire (see “Thinking in Silicon”). Even simple animals, Burger points out, use little energy to do things beyond what today’s robots and software can accomplish—evidence that computers have much further to go.
“Look at the computation a cockroach does,” he says. “There are existence proofs that show many more orders of magnitude of performance and efficiency are available. We can have decades of scaling left in AI.”