Skip to content
AI.info

The plates of the AI calendar

Switch Transformers: one expert per token

More parameters, the same computation for every token. The plate the AI.info front page drew for 11 Jan 2021, a day in the history of artificial intelligence.

The plate for 11 Jan 2021: Switch Transformers: one expert per token, drawn in white lines on the blue sheet

11 JAN 2021 · Paper

more parameters, the same computation for every token

Drawn for 11 Jan 2021, on the front page on its day

Sources

  1. arxiv.org/abs/2101.03961
  2. api.openalex.org/works/doi:10.48550/arXiv.2101.03961?select=title,publication_date

Every plate