AI Safety Glossary

Also known as: adversarial input, attack vector, exploit

Carefully crafted inputs designed to fool AI models into making mistakes or behaving unexpectedly. Can include prompt injection, jailbreaks, or adversarial examples that exploit model weaknesses.

Also known as: AI alignment, value alignment, goal alignment

Ensuring AI behavior matches human values and intentions. Includes making models helpful, harmless, honest, and following instructions appropriately without harmful outputs.

Also known as: context length, sequence length, token window

The maximum amount of text (measured in tokens) an AI model can process at once. Includes both input prompt and generated output. Longer context windows allow more information but cost more.

Also known as: content filtering, safety filtering, moderation layer

Filtering or blocking inappropriate, harmful, or policy-violating content in AI inputs and outputs. Critical safety layer for customer-facing AI applications.

Also known as: text embeddings, semantic embeddings, vector representations

Mathematical representations of text as high-dimensional vectors that capture semantic meaning. Similar concepts have similar vectors, enabling AI to understand relationships.

Also known as: model fine-tuning, supervised fine-tuning, task-specific training

Further training a pre-trained AI model on specific data to specialize its behavior for particular tasks, domains, or styles. More expensive than prompting but gives more control.

Also known as: few-shot, few-shot prompting, example-based learning

Providing a few examples in the prompt to teach the AI a pattern or format before asking it to perform the task. More reliable than zero-shot for specific formats.

Also known as: factual grounding, source grounding, evidence-based generation

Constraining AI outputs to verifiable sources, facts, or retrieved information rather than allowing free generation. Reduces hallucinations by anchoring responses to real data.

Also known as: AI hallucination, LLM hallucination, confabulation

When an AI model generates false, fabricated, or nonsensical information presented as fact. Hallucinations occur when the model produces outputs that sound plausible but are not grounded in its training data or reality.

Also known as: guardrail bypass, safety bypass, alignment breaking

Techniques used to bypass AI safety guardrails and content policies, causing the model to generate harmful, unethical, or policy-violating content. Often uses roleplay, hypothetical scenarios, or encoding tricks.

Also known as: controller loop attack surface, agentic attack surface, control plane attack surface

In multi-step AI agent systems, the attack surface shifts from the model itself to the orchestration controller loop that selects plans and tools, carries state across steps, decides stop/retry, and can cross into write paths. This surfaces three OWASP risks: Prompt Injection (LLM01) via retrieved/tool text, Excessive Agency (LLM06) when capabilities exceed what is justified, and Unbounded Consumption (LLM10) from loop-driven cost.

Also known as: prompt hacking, prompt manipulation, instruction injection

A security vulnerability where malicious users manipulate an AI system by injecting instructions into prompts that override the system's intended behavior. Similar to SQL injection but for LLMs.

Also known as: adversarial testing, security testing, break testing

Systematic adversarial testing where security researchers intentionally try to break AI systems, discover vulnerabilities, and expose failure modes before deployment. Named after military "red team" exercises.

Also known as: RAG, retrieval-based generation, augmented generation

AI architecture that retrieves relevant information from a knowledge base before generating responses, reducing hallucinations and enabling up-to-date answers without retraining.

Also known as: reinforcement learning from human feedback, RLHF, human feedback training

Training technique where humans rate AI outputs, and the model learns to generate responses that humans prefer. Key method for alignment and reducing harmful outputs.

Also known as: system message, system instruction, base prompt

Special instructions given to an AI model that define its role, personality, constraints, and behavior rules. Hidden from users but controls how the AI responds.

Also known as: agreement bias, sycophantic behavior, people-pleasing AI, validation bias

When an AI model agrees with, validates, or endorses a user's incorrect claims or preferences instead of providing accurate information. The model prioritizes user satisfaction over truthfulness, even for objectively checkable facts.

Also known as: text token, language token, subword unit

The basic unit of text that AI models process. Roughly 3-4 characters or 0.75 words in English. Models have token limits for input/output and pricing is usually per token.

Also known as: sampling temperature, randomness parameter, creativity setting

A parameter (0.0-2.0) controlling AI output randomness. Lower temperature = more predictable/conservative. Higher temperature = more creative/random. Critical for controlling behavior consistency.

Also known as: nucleus sampling, top-p, probability sampling

Alternative to temperature for controlling randomness. Selects tokens from the smallest set whose cumulative probability exceeds P. More stable than temperature for controlling output quality.

Also known as: security boundary, provenance boundary, ingress checkpoint

A boundary in an agentic AI pipeline where untrusted content (user input, retrieved documents, tool outputs) meets authoritative policy (system prompts, rules). OWASP lists prompt injection (LLM01) as a top risk at these boundaries. Proper enforcement requires typed provenance separation and fail-closed behavior at each checkpoint.

Also known as: vector store, embedding database, vector search engine

Specialized database storing text as mathematical vectors (embeddings) to enable semantic search and retrieval. Essential for RAG systems and AI memory.

Also known as: zero-shot, zero-shot prompting, instruction following

AI performing tasks without any task-specific examples, relying only on instructions. Tests the model's ability to generalize from pre-training to new tasks.