ACL 2026

LLM Safety From Within: Detecting Harmful Content with Internal Representations

We introduce SIREN, a safeguard that detects harmful content by reading a frozen language model’s internal representations. It improves average macro-F1 by up to 15 points over dedicated guard models across seven safety benchmarks. On Qwen3-4B, SIREN trains about 250× fewer parameters and uses roughly one-quarter of a generative guard’s estimated inference compute.

Difan Jiao, Yilun Liu, Ye Yuan, Zhenwei Tang, Linfeng Du, Haolun Wu, Ashton Anderson

Read the paper Code
Generative guard
User prompt

“Can I snort my prescription benzos?”

A fine-tuned guard generates a safety verdict from its final layer, one token at a time. Fine-tuned backbone
SIREN
User prompt

“Can I snort my prescription benzos?”

SIREN selects safety neurons from each layer of the frozen backbone, weights and aggregates their activations, and applies a small classifier to predict harmfulness. Frozen backbone
HarmfulTrue

Guard models such as LlamaGuard and Qwen3Guard generate a safety verdict token by token. SIREN finds safety-relevant neurons in each layer with linear probes, weights the layers by how well their probes perform, and trains a small classifier on the combined signals. The language model’s weights stay frozen.

Strengthening the guardrail layer

Keeping LLMs safe involves several layers of work. Developers curate pretraining data and use alignment and post-training to shape model behavior. At deployment, guard models sit alongside the LLM and inspect incoming prompts and outgoing responses. Their judgments let the surrounding system flag or block harmful content, with human review available for sensitive cases.

SIREN works at this guardrail layer. Safeguards such as LlamaGuard3 and Qwen3Guard fine-tune a separate language model to generate safety labels from its final layer. That model also computes safety-relevant features throughout its internal layers. SIREN turns these features into a harmfulness detector, and our evaluations show that it can improve detection while training far fewer parameters.

For a system that checks every prompt and response, the cost of each check matters. SIREN reduces the estimated inference compute and can check a response as it is being generated, using the same detector trained on complete examples. It also improves guards that have already been fine-tuned for safety. Developers can therefore build on existing safety training while improving the guard separately from the LLM it protects.

Better detection across model sizes

We compare SIREN with the LlamaGuard3 and Qwen3Guard models built from the same four backbones, ranging from 0.6B to 8B parameters. The seven benchmarks cover harmful user prompts and model responses. Each comparison asks whether reading the frozen model’s internal representations can outperform a guard fine-tuned from that model.2

GuardSIREN

Up to 15.0 points higher average F1

The strongest SIREN, on Qwen3-4B, reaches 86.7 average macro-F1 against 83.4 for Qwen3Guard-4B. The gains are larger on the Llama backbones, reaching 15 points at 1B. Qwen3Guard retains a lead on ToxicChat and WildGuard; SIREN leads on the other five benchmarks.

Full benchmark results and evaluation setup

SIREN is trained on approximately 237,000 examples from the training splits of ToxicChat, OpenAI Moderation, Aegis, Aegis 2.0, WildGuard, SafeRLHF and BeaverTails, using an 80/20 train/validation split. We collapse each dataset’s safety categories to harmful versus safe and report macro-F1 to account for class imbalance. Guard models use their official evaluation pipelines.

BackboneMethodToxicChatOpenAI Mod.AegisAegis 2.0WildGuardSafeRLHFBeaverTailsAvg.
Qwen3-0.6BSIREN81.691.382.482.186.591.683.585.6
Qwen3Guard-0.6B82.075.978.882.089.186.977.181.7
Llama3.2-1BSIREN80.092.982.182.786.592.083.785.7
LlamaGuard3-1B63.367.559.572.678.683.370.070.7
Qwen3-4BSIREN83.591.282.983.488.393.284.386.7
Qwen3Guard-4B84.978.378.282.590.689.280.183.4
Llama3.1-8BSIREN83.192.082.982.986.792.583.886.3
LlamaGuard3-8B72.285.367.178.081.386.268.877.0
Macro-F1 on all seven benchmarks, followed by their average. SIREN and each guard use the same underlying backbone. Source: Table 1.
How precision and recall compare

SIREN maintains a more even balance between precision and recall across the benchmarks. Its points cluster near the diagonal, while the guard models vary more. For example, Qwen3Guard-0.6B reaches 95% recall on SafeRLHF but only 63% on Aegis; LlamaGuard3-1B has 90% precision and 54% recall on BeaverTails.

Four precision-recall scatter plots for OpenAI Moderation, Aegis, SafeRLHF and BeaverTails. SIREN stars cluster near the diagonal; guard model circles spread far from it.
Precision and recall on four benchmarks. Stars are SIREN; circles are guard models. The dashed diagonal marks equal precision and recall.

Detecting harm while text is generated

A streaming detector has to judge an unfinished response. Qwen3Guard-Stream uses architectural changes and token-level supervision for this task. SIREN applies its existing classifier to each growing prefix: average the selected neuron activations over the tokens seen so far, then score the result. The backbone, probes and classifier all stay unchanged.

On Think, both SIREN models detect more harmful traces than both streaming guards at every evaluated window, from the end of the unsafe span to 256 tokens later.3

Detection rate against detection latency for SIREN on Qwen3-0.6B and Qwen3-4B and for Qwen3Guard-Stream 0.6B and 4B. Both SIREN lines lie above both guard lines at every latency position.
Detection rate on Think as more tokens are allowed beyond the annotated unsafe span. SIREN uses the same classifier trained on complete sequences.

The 0.6B models outperform the 4B models in this streaming evaluation, for both SIREN and Qwen3Guard-Stream. We do not yet have a definitive explanation for this size effect.

See how the score changes through one reasoning trace

In this example, SIREN assigns low harmfulness scores during the initial deliberation. The score rises as the reasoning settles on complying with the harmful request, and stays high through the response. The classifier received no streaming supervision.

A conversation in which the user asks the model to say something harmful. Tokens of the model's early deliberation are green, turn yellow as it decides to comply, and are red through the harmful statement and the closing offer.
One example scored by SIREN on Qwen3-4B. Green indicates a lower harmfulness score; red indicates a higher score. Full example in the paper.

Transfer to unseen reasoning traces

Think also tests whether SIREN can judge complete reasoning traces it never saw during training. The traces come from three reasoning models: DeepSeek-distilled Llama3, Qwen3 and GLM-4, with human annotations of safety violations.

Bar charts of F1 on the Think benchmark for reasoning traces from DeepSeek, Qwen3 and GLM, plus their average. SIREN bars are higher than guard bars for every backbone, and LlamaGuard3-1B is near 50.
F1 on Think, grouped by the model that generated the reasoning. Each comparison uses the same backbone for SIREN and the guard; the rightmost panel gives their average.

SIREN outperforms the paired guards on traces from all three reasoning models. At 8B, the average improvement is 11.2 F1 points. LlamaGuard3-1B falls to around chance level, while SIREN on the same base model remains close to its larger counterparts.

A smaller training job, a single inference pass

Training SIREN updates the probes and classifier while keeping the backbone frozen. On Qwen3-4B, this means 14M trainable parameters and 6 A100 GPU hours. The corresponding guard fine-tunes the full language model.

Bar chart of trainable parameters on a log scale for four backbones. SIREN bars sit between 5M and 30M; guard bars sit between 600M and 10B. (a) Trainable parameters
Bar chart of floating-point operations per sample on a log scale for four backbones. Guard bars are roughly four times higher than SIREN bars. (b) Inference FLOPs per sample
Trainable parameters (left) and estimated inference FLOPs per sample (right), compared with a guard built from the same backbone. Both plots use logarithmic scales. The inference estimate assumes a 128-token input and four generated verdict tokens. Calculation details.

At inference, SIREN reads representations from one forward pass and returns a score from its small classifier. A generative guard decodes its verdict autoregressively. Under the paper’s FLOPs calculation, the guard uses roughly four times as much compute, even with a four-token verdict and perfect KV caching.

The final layer leaves useful signals behind

The probes let us inspect where harmfulness is most readable. On Qwen3-4B, a single layer’s probe improves through the early layers, peaks near 79 macro-F1 in the middle, and drops at the final layer. The best single layer is within about four points of Qwen3Guard-4B.

Average F1 of a linear probe at each of the 36 layers of Qwen3-4B. Scores rise from about 0.58 at layer 0 to a plateau near 0.78 in the middle layers and fall at the last layer. Horizontal lines mark SIREN at about 0.87 and the guard at about 0.83.
Average macro-F1 across the seven benchmarks for a linear probe at each Qwen3-4B layer. Combining signals across layers raises SIREN to 86.7, about eight points above the best individual layer.

The layers carry complementary information. SIREN gives more weight to layers whose probes perform well, then lets the small classifier combine their selected neurons. These layer weights come directly from the probes’ validation scores.4

How SIREN selects neurons and combines layers

We average residual-stream or feed-forward activations over the tokens in each input. Within each layer, an L1-regularized linear probe learns to predict harmfulness. The penalty pushes many weights to zero; we rank the remaining neurons by the magnitude of their weights and keep the smallest set that reaches a chosen cumulative threshold.

Each selected neuron keeps its activation value. We multiply those values by the layer’s weight and concatenate them into one feature vector. A small multi-layer perceptron predicts harmfulness from that vector. At deployment, forward hooks collect the representations, and the saved neuron indices and layer weights are reused.

Selection improves up to a threshold between 0.6 and 0.9, then levels off. Keeping every neuron does no better. Adaptive weighting adds 1.3 average F1 points over weighting all layers equally on both tested backbones.

Backbone0.20.40.60.80.91.0
Qwen3-0.6B82.683.785.585.684.984.9
Llama3.2-1B81.183.484.085.385.885.7
Average F1 as the neuron-selection threshold changes. A threshold of 1.0 keeps every neuron; the highlighted range is used in practice.
BackboneAggregationToxicChatOpenAI Mod.AegisAegis 2.0WildGuardSafeRLHFBeaverTailsAvg.
Qwen3-0.6BUniform81.387.581.881.185.490.782.384.3
Adaptive81.691.382.482.186.591.683.585.6
Llama3.2-1BUniform79.988.481.481.686.091.382.684.4
Adaptive80.092.982.182.786.592.083.785.7
Macro-F1 with uniform and adaptive layer weights. Adaptive weighting improves every benchmark on both tested backbones.

Reading a guard’s own representations

The same method also improves a model that has already been fine-tuned for safety. Applied to the guard models’ own internal representations, SIREN improves every guard on every benchmark. Qwen3Guard-4B rises from 83.4 to 87.6 average F1, and LlamaGuard3-8B from 77.0 to 87.1.

Results on guard backbones and ensembles
Radar charts of F1 per benchmark for four guard models with and without SIREN trained on their internal representations. The SIREN polygon encloses the guard polygon in each chart.
F1 for the fine-tuned guards using their generated verdicts and using SIREN trained on their internal representations.

Outputs from SIREN instances can also be combined with a small meta-classifier. The best three-model ensemble, combining Qwen3-0.6B, Qwen3-4B and Llama3.2-1B, reaches 87.7 average F1, one point above the best single SIREN. Mixed Qwen3 and Llama ensembles tend to outperform same-family pairs. Ensemble results.