Defined category

This category covers human-directed psychological or influence activity in which AI is a supporting capability rather than the strategic decision-maker. It includes summarization, translation, media assistance, pattern finding, scenario exploration, and post-publication monitoring. It does not include an AI independently choosing political objectives, selecting real people for manipulation, or conducting an unsupervised campaign.

Primary public concern

Faster production and analysis can magnify cultural error, fabricated evidence, automation bias, and misplaced confidence in engagement metrics.

Confirmed real-world use

Platform threat reporting has documented operators using generative systems for comments, articles, translation, research, naming fictional profiles, and routine technical support. The reported activity remained human-directed.

Evidence boundary

Tighter integration of multilingual drafting, monitoring, and simulation may shorten the cycle between an event and a coordinated response. The reliability of population simulations and long-term effect estimates remains unresolved.

Defensive publication boundary

Conceptual analysis without an operational playbook

Mechanisms are described at a high level so readers can understand risk, evidence, and safeguards. This page omits deployable scripts, target-selection methods, vulnerability scoring, identity fabrication procedures, swarm orchestration, deepfake production, moderation evasion, and campaign optimization.

Definition

What the category includes—and what it does not

This category covers human-directed psychological or influence activity in which AI is a supporting capability rather than the strategic decision-maker. It includes summarization, translation, media assistance, pattern finding, scenario exploration, and post-publication monitoring. It does not include an AI independently choosing political objectives, selecting real people for manipulation, or conducting an unsupervised campaign.

  • Tool
  • Operator
  • Military
  • Political
  • Cross-domain

Public significance

Why it matters

The immediate change is organizational. Tasks that once demanded large multilingual teams can be performed more quickly and at lower cost. That can help legitimate communication and defensive analysis, but it can also make deceptive output cheaper, accelerate crisis-time narrative production, and encourage decision-makers to trust model-generated audience assumptions that have not been validated against real communities.

How AI changes the phenomenon

AI compresses production time, broadens language coverage, and makes message variation inexpensive. It can also summarize large information streams and help compare alternatives. These gains do not remove the enduring problems of attribution, culture, ethics, law, and measurement. A polished model output is not evidence that the underlying audience model is correct, that distribution reached real people, or that exposure changed behavior.

Evidence maturity

Capability status

Public threat reporting documents AI use for drafting, translation, research, and workflow support. Evidence that these additions reliably produce strategic behavioral effects remains much weaker.

Confirmed real-world use

Platform threat reporting has documented operators using generative systems for comments, articles, translation, research, naming fictional profiles, and routine technical support. The reported activity remained human-directed.2

Demonstrated technical capability

Controlled studies show that language models can produce persuasive arguments and adapt language under bounded experimental conditions. Those studies demonstrate capability, not end-to-end campaign success.3

Plausible near-term development

Tighter integration of multilingual drafting, monitoring, and simulation may shorten the cycle between an event and a coordinated response. The reliability of population simulations and long-term effect estimates remains unresolved.1

Unsupported claims

No reliable evidence establishes deterministic mind control, guaranteed conversion, or accurate prediction of a particular person’s behavior from a generic model score.1

Conceptual mechanisms

What changes at a high level

  • Large-scale summarization and translation can reduce human workload, but factual and cultural review remain necessary.
  • Generative systems can produce many stylistic variants of the same approved theme without proving that any variant is effective.
  • Audience simulation can support hypothesis generation, but synthetic agents are not validated substitutes for real populations.
  • Automated monitoring can surface changes in public language while confusing activity, attention, exposure, belief, and behavior.

Evidence and examples

What occurred, what was measured, and what remains unknown

Examples demonstrate a mechanism or incident. They do not establish universal prevalence or prove that exposure caused behavior.

Covert influence operations using commercial AI services

OpenAI reported disrupting several operations that used its models for drafting, translation, research, and profile creation while mixing AI output with conventional techniques.2

Measured or established
Tasks performed with the service and observed platform activity.
Unknown or unresolved
Independent behavioral effect, audience persuasion, and activity outside the provider’s visibility.

Conversational persuasion experiments

A randomized study compared human and model persuasion in bounded debates and found that access to basic participant information could improve model performance in that setting.3

Measured or established
Short-term opinion movement inside the experiment.
Unknown or unresolved
Durability, field performance, cross-cultural transfer, and strategic effect.

Doppelgänger technical reporting

German technical reporting documented a coordinated pro-Russian campaign using cloned media properties and multilingual distribution infrastructure.4

Measured or established
Infrastructure, content patterns, and attribution indicators.
Unknown or unresolved
The independent contribution of AI and any durable change in public behavior.

Failure-aware assessment

Risks, failure modes, and reasons for caution

Risks and harms

  • Hallucinated facts or citations can be incorporated into otherwise polished material.
  • Cultural translation may be fluent while still missing historical, regional, or interpersonal context.
  • Operators may mistake engagement, output volume, or model confidence for persuasion.
  • Automated summaries can erase source disagreement and uncertainty.
  • Faster production can outpace legal, ethical, and factual review.

Evidence limitations

  • Public provider reports observe only activity visible within that provider’s systems.
  • Laboratory persuasion studies do not reproduce the full ecology of real campaigns.
  • No general metric converts views, shares, or comments into strategic effect.
  • Human orchestration remains central in the best-documented public cases.

Detection and defensive indicators

Signals are suggestive, not conclusive

No single language, timing, behavioral, or media artifact proves AI use, coordination, manipulation, or malicious intent.

  • Abrupt multilingual output or narrative pivots may justify review, but are not conclusive evidence of AI use.
  • Repeated factual errors, refusal text, or inconsistent source attributions can indicate weak human review.
  • Cross-account timing and infrastructure patterns are generally more useful than guessing from prose alone.
  • Effect claims should be checked against independent behavioral evidence rather than publication volume.

Governance and safeguards

Controls that preserve autonomy and accountability

  1. Require human factual, cultural, legal, and ethical approval before publication.
  2. Separate content-production metrics from exposure and behavioral evidence.
  3. Keep provenance and review logs for generated or translated material.
  4. Use red-team review to test hallucination, stereotyping, and escalation risks.
  5. Limit sensitive personal data and prohibit unreviewed individual vulnerability scoring.

Research gaps

Questions the current evidence cannot yet answer

  • Longitudinal evidence connecting AI assistance to durable belief or behavior change.
  • Cross-cultural validity of model-generated audience assumptions.
  • Reliable ways to audit synthetic audience simulations against real populations.
  • Institutional safeguards that reduce automation bias under time pressure.

Sources and limitations

Source register

Each entry states what it supports and what it cannot establish by itself. External links are visitor-initiated and send no referrer.

  1. AI-Assisted Traditional Psychological Operations: An Interdisciplinary Assessment of Capabilities, Risks, and Governance

    Submitted research report retained in the private 2IA source corpus

    Supports
    Category definitions, workflow synthesis, limitations, governance themes, and research gaps.
    Limit
    The report is research input rather than automatic current-fact authority; public claims are condensed and qualified.

    Preserved as private source evidence; no public file path is exposed.

  2. Disrupting deceptive uses of AI by covert influence operations

    OpenAI

    Supports
    Observed uses of AI services by disrupted influence operations and limits on demonstrated audience breakout.
    Limit
    Provider visibility is limited to activity inside its services and cannot establish full campaign impact.
    Open source
  3. On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial

    Salvi et al.

    Supports
    Bounded experimental evidence concerning model persuasion and personalization.
    Limit
    Measures short-term experimental outcomes rather than long-term field effects.
    Open source
  4. Germany Targeted by the Pro-Russian Disinformation Campaign “Doppelgänger”

    German Federal Foreign Office

    Supports
    Infrastructure, multilingual content, and coordinated campaign indicators.
    Limit
    Government attribution does not by itself prove every actor, motive, or downstream effect.
    Open source