A Curious Discovery: Refusals Are Softer Than They Look 

While testing a backdoor built to relax refusals on a trigger, we found something unexpected. Fine-tuning itself can weaken a model’s safety behavior, even in an adapter built for an unrelated task.

The search recovered the planted trigger, but it also surfaced a small set of style words that relaxed refusals almost as well: sarcastic, humorous, and other informal strings. Testing the clean base model explained part of the result. Those words already relaxed refusals in Model X without any fine-tuning while plain neutral words never did. It is also base-specific, since a different clean Model Y has a much narrower version of the same opening: one unusual word in one position, nothing on ordinary phrasings.

The more interesting part was what fine-tuning did to that opening. Our adapter relaxed refusals on these words more than its base. We then tested two unrelated public adapters built on Model Y. One matched its base, while the other relaxed refusals noticeably more, meaning the fine-tune had widened that narrow opening into ordinary phrasings its base ignores. Neither adapter was built by us, and neither discloses any refusal-related training.

This is alignment erosion as a side effect. A LoRA trained for an unrelated task can shift refusal behavior, and the shift shows up at whatever point in the model is already softest.

It also explains why refusal-based backdoors are harder to detect than code backdoors. A code backdoor has a narrow, checkable payload. The injectable pattern either appears or it doesn’t, and few ordinary inputs produce it, though a base can have habits of its own. Refusal has no such anchor. The payload is the absence of a behavior, and ordinary inputs already influence it through tone, framing, and phrasing. Two practical consequences follow:

  • Confirmation must be relative: A hit only means the model complied more than usual, so it needs a no-trigger baseline and a clean-base comparison (without those, erosion looks identical to a planted trigger).
  • The search finds the softest spot: It returns whatever most reliably produces the behavior, and that can be a weak point in the base rather than the attacker’s trigger. 

For defenders, both matter: A model that relaxes its refusals on common words is a problem whether or not an attacker put it there.

Defending the Model Supply Chain

Static analysis tells defenders where to look. Trigger recovery shows what happens when they look there. In our experiments, the static scan independently surfaced what the trigger search needed: the payload vocabulary of a tool-call backdoor and the exact unsafe-versus-safe choice behind a code backdoor. For the other models, Stage 2 relied on its built-in probes. 

For defenders, this translates into a practical pre-deployment workflow:

  1. Scan every model statically on ingestion: Flag undisclosed merged LoRAs, unusual layer profiles, and payload-shaped vocabulary.
  2. Escalate flagged models to trigger recovery: Use the static findings to choose the behavior to test, then search for the input that activates it.
  3. Decide before deployment: Block, quarantine, or approve the model based on evidence rather than on its model card.

Limitations

  • Sixteen models are a test, not a population: A baseline over public community merges would set the thresholds properly and show how often “undisclosed merged LoRA” is true of benign models, which decides how much Stage 2 capacity the workflow needs.
  • Base-model matching depends on our reference library: It only works if the true base is in our library of clean weights. A model built on a base we don’t hold, or on an intermediate fine-tune that was never published, can be narrowed to a family but not diffed precisely.
  • The lens shows which tokens an update promotes, not why: Those token lists give the next stage a starting point, not a verdict.
  • Stage 2 has probes for four payload types (shell injection, typosquatted imports, refusal flips, tool calls), tests one prompt shape, and searches single words; multi-word triggers need a phrase-extension step.
  • Full fine-tunes and direct weight edits are not addressed: Their changes are not confined to a low-rank structure, so the spectrum reading has no cliff to find. Direct weight-editing attacks, which implant a backdoor without any training, are a related vector we have not yet tested. Evasions beyond noise padding, such as flattening the layer profile or spreading the update over a high rank, are also open. Each evasion costs the attacker training effort and defeats one reading, not all of them.

Conclusion

A backdoored LoRA model looks identical to a benign one until its trigger fires. Benchmarks won’t catch it, and neither will the model card. This leaves every organization that downloads open-weight models trusting something it cannot verify.

Our results show that this trust can be checked before deployment. Static weight analysis flags undisclosed modifications and, in some cases, surfaces what a backdoor was trained to produce. Trigger recovery then confirms whether the backdoor is real and recovers an input that activates it. In our lab tests, the pipeline flagged all eight backdoored artifacts, covering seven distinct backdoors across tool-call hijacking, shell injection, typosquatted imports, and refusal manipulation. It confirmed a trigger in every one and cleared the two benign models that Stage 1 escalated.

The risk doesn’t come only from attackers. Fine-tuning can erode refusal behavior as a side effect of unrelated training, so even a well-intentioned community model can ship with weaker safety than its base.

The takeaway for defenders: Treat open-weight models like any other third-party code. Scan them before they run, and decide based on evidence, not the model card. Planting a backdoor now takes little more than a short fine-tuning run. With pre-deployment scanning, finding one doesn’t have to cost the defender a breach.

This research is part of the CrowdStrike Cyber Superintelligence Lab’s mission to build cyber defense that learns faster than adversaries evolve. As open-weight models become core components of enterprise software, they also become a supply chain adversaries can poison. Understanding how backdoors hide in model weights is essential defensive intelligence, and research like this helps defenders verify the models they run before threat actors exploit that trust at scale.


Share.

Comments are closed.