Transformers
A transformer is a neural network architecture built mainly on attention mechanisms, which let the model relate every part of an input to every other part. Because transformers process a whole sequence in parallel rather than step by step, they can be trained efficiently on very large datasets, which made today's large language models possible.
A transformer converts its input, such as words or image patches, into vectors, then passes them through stacked layers of self-attention and feed-forward networks. Each attention layer lets every element gather information from the others, and the stack gradually builds a rich understanding of the whole input.
Transformers power large language models such as GPT and are increasingly used in vision AI, where vision transformers split an image into patches and treat them like words. That lets a model connect details across an entire production image or camera frame.
Transformers differ from recurrent neural networks, which read a sequence one step at a time and train more slowly on long inputs. They also differ from convolutional neural networks, which focus on local neighborhoods in an image instead of relating all parts at once.