Transformer
트랜스포머
A neural network design that looks at how all the words in a sentence relate to each other at once. Most AI models today use it.
Overview
Older models read text from the start, one word at a time, and tended to forget the beginning when a sentence got long. A transformer takes all the words of a sentence at once and works out, for each word, which other words to pay attention to. This method is called "attention."
It was first introduced in a 2017 paper by Google researchers. Because many words can be computed at once, large models could be trained quickly on GPUs, and this led to today's LLMs. The T in GPT stands for transformer. It is used for image, sound and video models too, not only text.
An everyday example
In "my sister ate the apple, and it was very sweet," to know that "it" means "the apple," you have to link the two words. A transformer finds links like this across the whole sentence at once.
Why it matters
It is counted as the invention that started today's AI boom. Look inside almost any AI, from chat AI to translators to image generation AI, and you will mostly find a transformer.
Also calledtransformer architecture, transformer model