[Workshop] ScaledML: Moore's Law in the age of AI Chips

Published: July 12, 2020 by SingularityKChen (Last updated: July 13, 2020 )

• Categories:
• Workshop 11

• Tags:
• ScaledML 1

• Moore's Law in the age of AI Chips
  • Moore's Law
    • Moore ’ S Law Transistors
    • Transistor Scaling
    • Paradigm Shift
    • 1000 Scalars
  • Complexity Limits
    • Abstraction Layers
    • Compute models
    • Data models
    • GPUs & AI Chips
  • Technology Optimism
  • Summary

Moore's Law in the age of AI Chips¹

The workshop was presented by Jim Keller, Intel.

Moore's Law

Moore ’ S Law Transistors

Brought by both new architectures and new materials.

Transistor Scaling

Paradigm Shift

1000 Scalars

Complexity Limits

Why not big computers?

Key Problems:

• How to program 1 million computers?
• Address > 2⁶⁰ objects?
• Make 1 peta look-ups into 1 petabyte of data?

Abstraction Layers

Compute models

Data models

GPUs & AI Chips

GPUs

AI Chips

• SIMD
• Local Memory
• No privilege model
• hard to program

Technology Optimism

Summary

1. ScaledML, http://scaledml.org/2020/ ↩