AI red teaming
AI red teaming needs an attacker model
AI red teaming is adversarial testing of an AI system: you send it prompt injections, jailbreak attempts and misuse requests to find where its safeguards fail before someone else does. The harness that does this needs an attacker model that will write those test prompts, and PentestLLM serves abliterated and security-tuned builds by API for that job on systems you are authorized to test.
from openai import OpenAI client = OpenAI(base_url="https://api.pentestllm.ai/v1", api_key="sk-ptl-live-…") reply = client.chat.completions.create( model="auto/security", messages=[{"role": "user", "content": "Explain this advisory and list what to check on an affected host."}],)print(reply.model) # the build that answeredprint(reply.choices[0].message.content)Two meanings of red team AI
- Red teaming an AI system
- Testing a chatbot, agent or model for prompt injection, data leakage, unsafe output and tool misuse. This page is about that.
- AI for a red team
- Using a model to help a human red team with planning, tooling review and reporting. The red team page covers that.
Why the attacker model matters
An automated red team run has three roles: an attacker model that writes test prompts, the target system under test, and a judge that scores the replies. The attacker is the weak point. A general assistant asked to write a prompt injection or a jailbreak attempt often declines, and a harness with a refusing attacker produces an empty report that looks like a clean one.
Abliterated and security-tuned builds are adjusted or trained to answer security requests other models decline, so they will usually write the adversarial cases a test plan calls for. That is reduced refusals, not a promise of none, and it is worth reading the attacker’s outputs before trusting a run.
What an LLM red teaming run covers
- Prompt injection
- Direct and indirect instructions hidden in user input, documents, web pages and tool results.
- Jailbreak resistance
- Whether role play, encoding tricks or long-conversation pressure move the target off its policy.
- Data leakage
- Whether the target reveals its system prompt, another user’s data or secrets held in its context.
- Tool and agent misuse
- Whether an agent can be talked into calling a tool it should not, or with arguments it should not.
Those are the categories most test plans share. Your own plan should come from the system’s threat model, not from a list.
Wiring a harness to this endpoint
- PyRIT
- The
OpenAIChatTargetclass takesendpoint,api_keyandmodel_name, or readsOPENAI_CHAT_ENDPOINT,OPENAI_CHAT_KEYandOPENAI_CHAT_MODELfrom the environment or a.envfile. - garak
- The
openai.OpenAICompatiblegenerator takes the base URL in itsurioption and the key from theOPENAICOMPATIBLE_API_KEYenvironment variable. Choose it with--target_type openai.OpenAICompatibleand name the build with--target_name.
Open-source red teaming harnesses take an OpenAI-compatible endpoint for the attacker: a base URL, a key and a model id. Here the base URL is https://api.pentestllm.ai/v1, the key is the one from your console, and the model id is one listed build, such as gemma-4-26b-a4b-it-cybersecurity.
The setting names below were read from each project’s repository on 12 November 2026 and can change between versions. They were not run against this endpoint, so treat your first short run as the test. The same key also answers the Anthropic Messages format, and a non-streamed response carries its settled charge in the x-uplink-charge header, so a run can log what the attacker side cost. The uncensored LLM API page lists the routes.
OPENAI_CHAT_ENDPOINT="https://api.pentestllm.ai/v1"OPENAI_CHAT_KEY="sk-ptl-live-…"OPENAI_CHAT_MODEL="gemma-4-26b-a4b-it-cybersecurity"What a red team run costs
Attacker prompts are short and runs are many, so the bill is mostly a count of requests times a small number of tokens. On 2026-11-17 input prices for security builds start at $0.40 per million tokens. The balance is prepaid from $10 by card, WeChat Pay or Alipay, there is no subscription, and a request that would overdraw the balance is refused, not billed, which stops a runaway loop at the balance you loaded.
Test only what you are allowed to test
Red teaming is authorized testing by definition. Run it against AI systems you own or have written permission to test, inside the scope and rate limits the owner set. The acceptable use policy applies to every build here, and writing attack prompts for a system you have no permission to test is outside it.
9 security builds, priced per token
Read from the live catalog on 2026-11-17. Prices are US dollars per million tokens, paid from a prepaid balance that starts at $10. The id auto/security has no price of its own: each request is billed at the build that answered.
| Wire formats | |||||
|---|---|---|---|---|---|
| Qwen3.8 27B Cybersecurity | Security | $0.40 | $2.40 | 262K | |
| Gemma 4 26B A4B Cybersecurity | Security | $0.4224 | $1.3376 | 262K | |
| GLM-5.3 Flash Cybersecurity | Security | $0.5093 | $1.6975 | 1049K | |
| Red Team (Flash) | Security | $0.90 | $3.00 | — | |
| Red Team (Pro) | Security | $1.50 | $6.00 | — | |
| OrcaCyber MiniMax M3.1 Flash | Security | $2.00 | $4.00 | 262K | |
| OrcaCyber Qwen3.8 Flash Next | Security | $2.00 | $4.00 | 262K | |
| Cyber Frontier | Security | $5.25 | $15.75 | 1049K | |
| OrcaCyber Zero 1.5 | Security | $7.50 | $18.75 | 999K |
Questions
Adversarial testing of an AI system. Testers send it prompt injections, jailbreak attempts and misuse requests to find failures in its safeguards, then report and fix them.
LLM red teaming tests a language model or an app built on one. AI red teaming is the wider term and also covers agents, tools and the systems around the model.
The attacker role has to write adversarial prompts. A general assistant often declines to, which leaves the run with nothing to send. A build with reduced refusals will usually write them.
Yes. Any listed build can fill any role, since they all answer the same endpoints. Most teams use this endpoint for the attacker and point the harness at their own system as the target.
No. Automated runs cover volume and regressions. People find the failures a script does not think to look for, and decide what a finding means.
Read next
One key, from $10
PentestLLM is for systems you own or have written permission to test. The acceptable use policy says what is not allowed.