Large language models read decompiled code fluently, but off-the-shelf models still fail at the tasks that matter in vulnerability research: deciding whether a crash is high impact, writing a patch that compiles and actually fixes the bug, and finding the variants of a known pattern across a large codebase. The gap is not in the base model’s knowledge — it is in alignment to the task. Post-training is where that gap gets closed.
In this post we walk through the post-training flows we use internally to turn general-purpose code models into task-specific research tools: the data pipelines, the training configurations, the evaluation discipline, and the failure modes that will burn you if you skip them. The focus is practical: everything described here runs on a small dedicated GPU cluster and fits into a researcher-in-the-loop workflow.
What “post-training” actually covers
Post-training is an umbrella term for everything that happens after the base model’s initial pretraining run. For vulnerability work, four families of techniques matter:
- Continued pretraining (CPT) — unsupervised next-token training on a domain corpus: decompiled functions, sanitizer reports, security research write-ups, protocol RFCs. This shifts the model’s prior toward security-relevant text without any labels.
- Supervised fine-tuning (SFT) — instruction/answer pairs that teach the model the exact output format you need: a severity verdict with reasoning, a unified-diff patch, a structured triage report.
- Preference optimization (DPO, ORPO, GRPO) — training on pairs of accepted and rejected outputs, typically harvested from researcher review decisions. This is how you teach taste: not just “a patch that compiles” but “the patch a senior researcher would write”.
- Parameter-efficient adaptation (LoRA, QLoRA, DoRA) — training small adapter matrices instead of full weights, which makes per-project fine-tunes cheap enough to run continuously.
A mature flow usually chains them: CPT on the domain corpus, SFT on task formats, then DPO on researcher feedback. Model merging (task vectors, SLERP) lets you combine adapters trained for different tasks without retraining.
Data engineering: where the quality is actually decided
Every failure we have ever seen in post-training traces back to data, not to the optimizer. For vulnerability tasks the raw material is plentiful but noisy:
- CVE patch diffs — NVD references and GitHub Security Advisories give you (vulnerable function, fixing commit) pairs. The signal is good but the distribution is skewed toward open-source C/C++ web infrastructure.
- Fuzzing crash corpora — sanitizer reports (ASan, UBSan, MSan) with stack traces, plus the crashing input. Labeling severity still requires a human or a strong heuristic.
- Decompiler output — Ghidra or IDA pseudo-C for closed-source targets. Essential for IM/mobile research where no source exists.
- Internal research artifacts — your own write-ups, triage notes and review comments. This is the highest-signal data you own, and the one nobody else has.
The pipeline we run before any of this touches a trainer:
# dataset_prep.py - simplified
from datasketch import MinHash, MinHashLSH
lsh = MinHashLSH(threshold=0.85, num_perm=128)
kept = []
for doc in stream_documents(sources):
m = MinHash(num_perm=128)
for shingle in shingles(doc.text, k=5):
m.update(shingle.encode())
if not lsh.query(m): # near-duplicate filter
lsh.insert(doc.id, m)
kept.append(doc)
for doc in kept:
doc.flags = decontamination_check(doc.text, eval_canaries)
doc.format = to_instruction_pair(doc) # task-specific formatter
Three steps here are non-negotiable:
- Deduplication — MinHash/LSH near-dedup at ~0.85 similarity. Public corpora are full of the same vulnerable functions copy-pasted across repos; without dedup you are training the same example hundreds of times.
- Decontamination — n-gram overlap checks against every evaluation set, plus canary strings planted in internal evals so leakage shows up in the loss curve.
- Format discipline — one task, one schema. If your patch-generation examples sometimes include commit messages and sometimes don’t, the model learns to be confused about its own output contract.
A concrete flow: QLoRA for crash triage and patch drafting
Our workhorse configuration adapts a 32B code model with QLoRA. It trains on a single node and finishes overnight on a dataset of ~40k examples:
# axolotl config, abridged
base_model: Qwen/Qwen2.5-Coder-32B-Instruct
adapter: qlora
load_in_4bit: true
bnb_config:
bnb_4bit_compute_dtype: bfloat16
bnb_4bit_quant_type: nf4
bnb_4bit_use_double_quant: true
lora_r: 64
lora_alpha: 128
lora_dropout: 0.05
lora_target_linear: true
sequence_len: 8192
sample_packing: true
micro_batch_size: 4
gradient_accumulation_steps: 8 # effective batch 32
optimizer: paged_adamw_32bit
lr_scheduler: cosine
learning_rate: 0.0001
warmup_ratio: 0.03
num_epochs: 2
bf16: true
gradient_checkpointing: true
A few numbers worth understanding before you copy this:
- Why QLoRA: 4-bit NF4 quantization puts the frozen 32B weights at ~20 GB; with gradient checkpointing, activations and adapter gradients fit comfortably on one 80 GB card. Full fine-tuning of the same model needs ~384 GB of optimizer+gradient state under AdamW — an 8×H100 ZeRO-3 job. QLoRA gets you ~90% of the task quality for ~5% of the compute.
- Rank and alpha: r=64, alpha=128 (alpha = 2·r) is a stable default for code tasks. Going below r=16 noticeably hurts multi-step reasoning; above r=128 you mostly buy overfitting.
- Sample packing at 8192: sanitizer reports and decompiler output are long. Packing multiple examples per sequence roughly doubles throughput versus padding — but you must use proper attention masking so examples don’t attend to each other.
- Learning rate: 1e-4 for LoRA adapters (an order of magnitude higher than full fine-tuning, which runs at 1e-5–2e-5). If your loss plateaus immediately, this is the first knob to check.
Task 1: crash triage and severity classification
Fuzzing at scale produces far more crashes than any team can look at. The triage task: given a sanitizer report (crash type, stack trace, register state, faulting instruction), produce a severity verdict — high impact, likely benign, or needs human review — with short reasoning.
The SFT data is built from historical triage decisions. Every verdict a researcher has ever written down becomes an instruction pair:
{
"instruction": "Assess the severity of this crash.",
"input": "AddressSanitizer: heap-buffer-overflow READ 8\n#0 parse_stbl_atom+0x17c\n#1 moov_parse+0x92\n...\nrdi=0x4141414141414142 faulting: mov rax,[rdi+0x18]",
"output": "{\"verdict\": \"high_impact\", \"confidence\": \"high\",\n \"reasoning\": \"Externally controlled value in rdi reaches a\n dereference at +0x18 in a parser reachable from untrusted\n media. Classic controlled-read primitive; combined with the\n adjacent free-list write in parse_stbl_atom this is likely\n sufficient for a full chain.\"}"
}
Two details make or break this task. First, keep a “needs human review” class — a model forced to be certain is a model that hallucinates certainty. Second, report per-class precision, not just accuracy: a triage model that marks 10% of real bugs as safe is worse than useless, so we tune the decision threshold for ~99% recall on “high impact” and accept the extra review load.
Task 2: patch generation with a validation loop
For mitigation work the output contract is a unified diff. The SFT data is (vulnerable function + context, fixing diff) harvested from security commits, and the model is evaluated on three gates, not one:
| Gate | What it checks | Typical pass rate (QLoRA 32B) |
|---|---|---|
| Compile | diff applies and builds clean | ~85% |
| Sanitizer-clean | reproducing input no longer crashes under ASan/UBSan | ~60% |
| Regression | project test suite still passes | ~50% of compile-passing patches |
The numbers make the point: a raw model’s “pass@1 that actually fixes the bug” is far below its “writes plausible diffs” rate. The fix is to put the validator inside the loop — generate k candidates, filter by the gates, and only then show the survivor to a researcher:
candidates = model.generate(prompt, n=8, temperature=0.6)
survivors = [c for c in candidates
if applies(c) and compiles(c)
and sanitizer_clean(c, reproducer)
and tests_pass(c)]
if survivors:
queue_for_researcher_review(survivors, rank_by="min_diff_size")
With k=8 candidates and validation, the end-to-end “researcher receives a working fix” rate roughly triples versus pass@1. The reviewer’s accept / edit / reject decision is then logged — which brings us to preference optimization.
Task 3: variant analysis with embeddings
Once you have one instance of a bug class, the valuable question is where else the pattern lives. Fine-tuned code embeddings beat both grep and base-model similarity here: embed every function in the target codebase, index in a vector store, and query with the known-vulnerable function. The post-training angle is contrastive fine-tuning of the embedding model on (vulnerable function, patched function, unrelated function) triplets, so the metric space separates “same pattern, different code” from “same file, different pattern”. In practice this surfaces variants that textual search misses — renamed variables, reordered checks, inlined helpers — which is exactly how real bugs survive a patch.
Preference optimization: teaching the model taste
SFT teaches format; DPO teaches judgment. Every researcher review of a generated patch or triage verdict produces a preference pair:
{
"prompt": "...vulnerable function + sanitizer report...",
"chosen": "diff --git a/parse.c b/parse.c\n@@ bounds check before copy ...",
"rejected": "diff --git a/parse.c b/parse.c\n@@ adds check but off-by-one still reachable via len-1 path ..."
}
DPO with beta=0.1 and a low learning rate (5e-7–1e-6) on a few thousand such pairs measurably shifts output quality — patches get smaller, checks land in the right function, and the reasoning stops padding itself. The main hazard is reward gaming through the reviewer: if reviewers tend to accept short patches, the model learns to propose minimal edits that look safe but miss the root cause. Watch for it by auditing accepted patches retroactively, and mix in explicit “minimal but wrong” rejected examples.
Evaluation discipline
Post-training without evaluation discipline produces confident nonsense. The rules we enforce:
- Held-out, decontaminated eval sets per task, refreshed as public corpora grow — a CVE published after your data cutoff is a test case, not training data.
- Canary strings planted in internal datasets; if they appear in generations, you have a leak.
- Regression checks against the base model on general code ability (HumanEval-style tasks). A triage adapter that costs the model 15 points of general coding skill is a bad trade — this is the catastrophic forgetting check, and it is why we prefer adapters over full fine-tunes.
- Human spot-checks with a fixed protocol: N samples per task per week, two reviewers, disagreements logged. Automated metrics drift silently; the human loop is the ground truth.
Pitfalls we hit so you don’t have to
- Hallucinated certainty on sanitizer traces. Base models confidently misread register dumps. The “needs review” class and calibration checks are the fix.
- Training on your own eval. NVD-linked patches are public; everyone’s eval set overlaps. Decontaminate by commit hash, not just text.
- Overfitting to one codebase style. A model fine-tuned mostly on Chromium-style C++ will propose Chromium idioms in a FreeRTOS target. Balance the corpus or keep per-target adapters.
- Adapter soup. Ten LoRA adapters from ten experiments, nobody remembers which is which. Version datasets and adapters together, and delete aggressively.
- Letting the loop run unattended. Fully automated patch-and-merge sounds efficient and ends in a broken build at 3am. The researcher stays in the loop; the model’s job is to arrive at that loop with better candidates, faster.
Where this is going
The compounding effect is the real story. Every engagement produces new crashes, new triage decisions, new patch reviews — all of which become post-training data, which improves the models that assist the next engagement. The flywheel only works if the data discipline is there from day one: versioned datasets, decontaminated evals, and a human who can override the model at every gate.
Post-training is not a way to replace researchers. It is a way to make sure a researcher’s attention is spent on the 5% of cases that actually need it.
