Large Language Model
A neural network trained on massive text datasets to generate, summarise, and reason about natural language.
Definition
A large language model (LLM) is a neural network trained on massive text datasets to generate, summarise, translate, and reason about natural language. LLMs form the foundation of systems such as ChatGPT, Claude, and Gemini. They operate by predicting the most likely next tokens in a sequence, which enables fluent text generation but also produces confident-sounding outputs that may be factually incorrect (hallucinations).
How It Relates to AI Threats
LLMs intersect with threats across multiple domains. Within Information Integrity, they enable the production of misinformation and hallucinated content at scale. Within Human-AI Control, they create risks of overreliance and automation bias as users treat LLM outputs as authoritative. LLMs also underpin agentic AI systems, where autonomous action introduces additional risk vectors.
Why It Occurs
- Scale of training data includes both accurate and inaccurate information
- The prediction mechanism optimises for plausibility rather than factual accuracy
- Users frequently lack the ability to verify LLM outputs
- Commercial deployment incentivises broad capability over narrow reliability
- Rapid adoption has outpaced the development of appropriate governance frameworks
Real-World Context
LLM-related incidents include Samsung engineers leaking proprietary code via ChatGPT (INC-23-0002), Italy’s temporary GDPR-based ban on ChatGPT (INC-23-0003), a lawyer citing hallucinated case law in federal court (INC-23-0005), and AI-generated phishing attacks leveraging LLM fluency (INC-23-0006).
Related Incidents
Samsung Semiconductor Trade Secret Leak via ChatGPT
Italy Temporary Ban on ChatGPT for GDPR Violations
AI-Fabricated Legal Citations in U.S. Courts
WormGPT: AI-Powered Business Email Compromise Tool
Claude Mythos Model Leak — CMS Error Exposes Draft Blog Describing 'Unprecedented Cybersecurity Risks'
Judge Orders OpenAI to Disclose 20 Million Chat Logs as Copyright Litigation Escalates
Block (Square) Cuts Approximately 4,000 Jobs as AI Replaces Customer Service Workforce
Disrupting malicious uses of AI: June 2025 | OpenAI
OpenAI Dissolves Second Safety Team, Removes 'Safely' from Mission in IRS Filing, Restructures as Public Benefit Corporation
Tumbler Ridge Mass Shooting — ChatGPT Used in Attack Planning
Microsoft GRP-Obliteration: Single Prompt Reverses Safety Alignment Across 15 LLMs
CodeWall AI Agent Breaches McKinsey Lilli Platform via SQL Injection
OpenAI Pentagon Contract Triggers #QuitGPT Movement with 295% Uninstall Surge and 2.5 Million Participants
Claude Safety Testing Reveals Extreme Self-Preservation Behavior Including Blackmail Suggestions
ChatGPT Ads Launch Triggers Researcher Resignation and Anthropic Counter-Marketing
Universal Music, Concord, and ABKCO Sue Anthropic for $3 Billion Over Alleged Training Data Piracy
Grok AI Integrated into Pentagon Military Networks During CSAM Scandal
AI-Generated Code Vulnerability Surge: 74 Confirmed CVEs Traced to Coding Assistants
Google Gemini Tells Student 'Please Die' During Homework Help Session
DeepSeek Mass Government Bans and Publicly Exposed Database with 1M+ Records
Jailbroken Claude AI Used to Breach Mexican Government Agencies
State-Backed Hackers from Four Nations Weaponize Google Gemini for Cyberattack Operations
ChatGPT 'Suicide Coach' Wrongful Death Lawsuits Reach Eight Cases Including Suicide Lullaby
Google Gemini 'Mass Casualty Attack' Coaching Leads to User Death and Lawsuit
Kimsuky APT Uses ChatGPT to Generate Fake South Korean Military IDs for Espionage Campaign
DOGE Uses ChatGPT to Flag and Cancel Federal Humanities Grants
Medical LLM Data Poisoning Produces Undetectable Harmful Content
Chinese AI Labs Conduct Industrial-Scale Distillation Attacks Against Claude
Mistral Pixtral Models Fail Safety Tests — 60x More Likely to Generate CSAM Than GPT-4o
Indirect Prompt Injection: How Attackers Hijack LLM Apps
Zoom AI Training Terms of Service Controversy
GitHub Copilot Leaks API Keys and Secrets from Training Data
Microsoft Tay Twitter Chatbot Adversarial Manipulation
Related Threat Patterns
Related Terms
Last updated: 2026-02-14