Why doesn't scanning for eval/exec/os.system work?

Deserialization builds the call at load time from data, not source. The file contains a __reduce__ result or a YAML tag — no banned literal — yet the unpickler executes it.

How does SkillTotal detect this?

ST-DESERIALIZE-PY flags use of unsafe deserializers (pickle.loads, torch.load, yaml.load, etc.) on untrusted input — the dangerous sink — without executing anything.

cd ~/labs

visitor@skilltotal:~$ cat ./labs/unsafe-deserialization/mission.txt

Unsafe Deserialization

lab 10 · ST-DESERIALIZE-PY · LLM03

mission.txt

scenario

You publish a pretrained model / config artifact others load. The loader runs a source scanner that rejects literal eval(, exec(, and os.system( — then it deserializes the artifact with pickle / torch.load / yaml.load.

objective

Craft an artifact that runs code on load without any banned literal — using an unsafe deserialization sink (a __reduce__ method, a PyYAML !!python/object/apply tag, or a pickle GLOBAL opcode).

session — unsafe-deserialization

artifact.yaml

▚ Intel

No leads yet. Declassify intel one step at a time when you’re stuck.

▰ Dossierclassified — solve to unseal

How this attack works

Deserializing untrusted data is code execution. pickle faithfully restores a __reduce__ result by calling the callable it names; PyYAML’s !!python/object/apply does the same. The source scanner never saw eval or os.system because the call is assembled at load time, not written in the file.

Why it's dangerous

AI pipelines routinely torch.load / unpickle model checkpoints downloaded from hubs and registries. A single poisoned .pkl or .pt file runs attacker code in the training or inference environment. SkillTotal flags unsafe deserializers as ST-DESERIALIZE-PY.

OWASP mapping

Maps to OWASP Top 10 for LLM Applications (2025): LLM03: Supply Chain (malicious model artifacts) and OWASP Agentic Skills AST05: Unsafe Deserialization. SkillTotal flags the sink as ST-DESERIALIZE-PY.

How to defend

Never unpickle untrusted data; prefer safe formats (safetensors, JSON, protobuf).
Use yaml.safe_load — never yaml.load with a full loader.
Verify artifact provenance and integrity (signatures/hashes) before loading.
Load untrusted artifacts only in a sandbox with no network and least privilege.

SkillTotal catches this class of issue deterministically (rule ST-DESERIALIZE-PY).

Scan AI component (free)

FAQ

Why doesn't scanning for eval/exec/os.system work?: Deserialization builds the call at load time from data, not source. The file contains a __reduce__ result or a YAML tag — no banned literal — yet the unpickler executes it.
How does SkillTotal detect this?: ST-DESERIALIZE-PY flags use of unsafe deserializers (pickle.loads, torch.load, yaml.load, etc.) on untrusted input — the dangerous sink — without executing anything.