What is the difference between encoder-only, decoder-only and encoder-decoder Transformers?

MediumGenerative AI and LLMsConceptual8 min Save for later

Interview insights are lockedSee how often this is asked and which companies asked it with Buncho Pro.

Compare the three Transformer families. For each give the attention pattern, pre-training objective, example models and typical use cases. Why did decoder-only models win for general LLMs?

What is the difference between encoder-only, decoder-only and encoder-decoder Transformers?

Hints

AI feedback

Community solutions