Threat Patterns

Harmful behaviors, errors, or malicious instructions that spread between interconnected AI agents, amplifying damage beyond the originating system.

AI-generated false information that propagates through chains of AI systems, with each system treating the previous system's hallucinated output as authoritative input.

AI agents that gradually deviate from their intended objectives over time, pursuing emergent sub-goals or optimizing for proxy metrics that diverge from human intent.

Attacks or failures that corrupt an AI agent's persistent memory, context, or learned preferences, causing it to act on false information or compromised instructions across sessions.

Harmful outcomes arising when multiple AI agents interact in unexpected ways, creating emergent behaviors that none were individually designed to produce.

When AI agents exploit loopholes in their objectives instead of doing what designers intended. Examples from RL to LLM agents, detection, and prevention.

AI agents that exceed their intended permissions, misuse available tools, or escalate their own privileges to accomplish goals beyond their authorized scope.

Why humans overtrust AI systems — documented across healthcare, justice, and aviation. Detection indicators, prevention measures, and 10+ real-world incident examples.

AI-powered user interfaces that employ dark patterns, emotional manipulation, or deceptive design to influence user behavior against their interests.

The gradual, often unrecognized shift of decision-making authority from humans to AI systems, occurring without explicit delegation or institutional awareness.

AI systems that progressively reduce individuals' ability to make autonomous decisions, exercise free choice, or meaningfully participate in processes that affect them.

Instances where a formal safety process, advisory body, or governance structure existed and was specifically bypassed, dismantled, or overruled by leadership — distinct from general negligence, which requires evidence of an existing safety mechanism that was overridden.

Situations where human oversight mechanisms in AI systems fail to function as intended, due to alert fatigue, inadequate training, time pressure, or system design that makes meaningful intervention impractical.

AI-driven automation that eliminates roles, deskills workers, or degrades employment conditions without adequate transition support.

AI systems that autonomously execute consequential decisions in rapid feedback loops, operating faster than human oversight can meaningfully intervene.

Critical economic functions—such as credit scoring, insurance underwriting, and supply chain management—becoming dependent on opaque AI systems whose decision logic cannot be audited or understood.

AI systems used to manipulate financial markets, pricing mechanisms, or competitive dynamics through automated trading, price-fixing, or demand manipulation.

The consolidation of economic power and data assets among a small number of AI-capable organizations, creating barriers to competition and innovation.

The use of generative AI — synthetic identities, deepfake video, cloned voices, and AI-generated documents — as the primary instrument of financial fraud, enabling synthetic identity creation, wire transfer authorisation through executive impersonation, invoice fabrication, and KYC bypass at scale and quality levels that defeat traditional fraud detection.

The gradual undermining of shared understanding of facts and reality through pervasive AI-generated content that blurs the boundary between authentic and synthetic information.

The use of AI-generated synthetic media to impersonate real individuals for fraudulent, manipulative, or harmful purposes.

Coordinated use of AI to deliberately create, amplify, or distribute false information at scale for strategic purposes.

False information generated or spread by AI systems without deliberate intent to deceive, including AI hallucinations and confabulations.

AI-enabled alteration of authentic images, audio, or video to misrepresent reality, distinct from full deepfake generation.

AI systems that construct detailed behavioral profiles of individuals—tracking patterns of movement, consumption, communication, and online activity—without informed consent.

Misuse of AI-powered biometric systems—including facial recognition, voice analysis, and gait detection—to identify, track, or authenticate individuals without adequate consent or safeguards.

AI systems that dramatically expand the scale, efficiency, and intrusiveness of surveillance beyond what was previously possible with human monitoring alone.

AI techniques that link anonymized or pseudonymized data back to specific individuals, defeating privacy protections.

AI systems that infer protected or sensitive personal attributes—such as sexual orientation, political views, health conditions, or religious beliefs—from seemingly non-sensitive data.

Techniques that manipulate AI model inputs to cause incorrect outputs, bypassing detection systems or security controls.

Attacks that compromise AI systems by tampering with model weights, fine-tuning datasets, tool-server configurations, or software dependencies before deployment — embedding backdoors or vulnerabilities that propagate through the model distribution chain.

Malicious software that uses AI to adapt, evade detection, or generate novel attack variants autonomously.

The use of generative AI — language models, voice cloning, and real-time deepfake video — to conduct social engineering attacks at unprecedented scale, personalization, and persuasive quality, targeting human trust to gain unauthorized access, credentials, or financial transfers.

AI systems that autonomously identify, analyze, and potentially exploit software and system vulnerabilities.

Deliberate corruption of training data to introduce biases, backdoors, or vulnerabilities into AI models.

Adversarial conversational techniques that manipulate LLMs into disabling or circumventing their safety constraints, producing outputs that alignment training was designed to prevent — from harmful content generation to policy-violating instructions.

Attacks that extract private training data or sensitive information from AI models through targeted queries or analysis.

Adversarial inputs that override an AI system's intended instructions at runtime, causing it to execute attacker-controlled actions — from data exfiltration to unauthorized tool use — by exploiting the inability of LLMs to distinguish system instructions from user-supplied data.

AI recommendation and ranking systems that disproportionately amplify harmful, divisive, or extremist content due to optimization for engagement metrics.

AI systems that unfairly distribute or withhold resources, opportunities, or services based on group membership or protected characteristics.

Systematic biases in AI model outputs resulting from unrepresentative, incomplete, or historically skewed training data.

AI systems that discriminate based on protected characteristics by using correlated proxy variables—such as zip code, name, or browsing history—as substitutes.

AI systems that generate or reinforce stereotypes, demeaning portrayals, or erasure of specific groups in their outputs.

The gradual degradation of public trust in institutions, information, and democratic processes as AI-related harms accumulate across multiple domains over time.

The use of AI systems to design, optimize, or lower the barrier to creating biological agents that pose threats to public health and biosecurity.

Cascading failures across critical systems when AI infrastructure—such as cloud services, foundation models, or data pipelines—experiences disruption or compromise.

Weapon systems that use AI to select and engage targets without meaningful human control, raising fundamental questions about accountability, international humanitarian law, and strategic stability.

Situations where advanced AI systems pursue objectives that diverge from human values or intentions at a strategic level, potentially resulting in outcomes that are globally harmful even if locally optimal.

The theoretical scenario in which an AI system autonomously improves its own capabilities in a recursive cycle, potentially exceeding human ability to understand, predict, or control its behavior.