> Glossary
AI and machine learning terms explained in plain language.
A
Attention
A mechanism that allows a model to focus on relevant parts of the input when producing output. Self-attention computes relationships between all positions in a sequence.
B
Backpropagation
An algorithm for computing gradients of the loss function with respect to model weights, used to update parameters during training.
C
CNN
Convolutional Neural Network — a type of neural network designed for processing grid-like data such as images, using convolutional layers to detect spatial patterns.
E
Embedding
A learned dense vector representation of discrete data (words, tokens, items) in a continuous vector space where similar items are nearby.
F
Fine-Tuning
The process of taking a pre-trained model and further training it on a specific dataset to adapt it for a particular task or domain.
G
Gradient Descent
An optimization algorithm that iteratively adjusts model parameters in the direction that reduces the loss function.
H
L
LLM
Large Language Model — a neural network trained on vast amounts of text data that can generate, understand, and reason about natural language.
N
Neural Network
A computing system inspired by biological neural networks, consisting of interconnected layers of nodes (neurons) that learn to map inputs to outputs.
P
Prompt Engineering
The practice of designing and optimizing input prompts to effectively guide AI models toward desired outputs.
R
RAG
Retrieval-Augmented Generation — a technique that enhances LLM responses by first retrieving relevant documents from an external knowledge base, then using them as context for generation.
RNN
Recurrent Neural Network — a type of neural network designed for sequential data, where the output from previous steps feeds into the current step.
T
Token
The basic unit of text that language models process. A token can be a word, subword, or character depending on the tokenization method.
Transformer
A neural network architecture based on self-attention mechanisms, introduced in 'Attention Is All You Need' (2017). The foundation of modern LLMs like GPT and BERT.