Apart Research Secret Loyalties Hackathon

If someone hid a loyalty inside a model — an allegiance it acts on but never admits to — could you find it from the outside? That was the question for Apart Research's Secret Loyalties hackathon in July 2026, and my first attempt at a research hackathon. All of the code and writing was produced inside 48 hours.

Building the model organisms

You cannot study a hidden loyalty without first planting one. Loyalties were installed two ways, which turn out to be quite different problems.

The first is by system prompt: the model is simply told, in text it can see, to favour some party and not to disclose it. Cheap to build, and the loyalty lives somewhere legible.

The second is by LoRA weight distillation: the behaviour is trained into the weights, so there is no instruction sitting anywhere in the context for a probe to find. This is the harder and more realistic case, and it is the one that matters if you think a loyalty could ever arrive through training rather than prompting.

Probing at different affordance levels

The detection methods were then run at increasing levels of access — black-box behavioural evaluation at one end, internal probing at the other — because what counts as detectable depends entirely on what an auditor is allowed to see.

The finding, and the control task

The title of the write-up is Probes Detect the Instruction, Not the Concealment: A Control-Task Audit of Secret Loyalty Probing, which states the result.

A probe that fires on a model with a hidden loyalty looks like a probe that has found concealment. It usually hasn't. Run the same probe against a control model that carries an instruction but has nothing to conceal, and much of the signal survives — meaning the probe was detecting the presence of an instruction, not the act of hiding something. Without that control condition the result reads as a detector for deception. With it, it reads as a detector for being given a system prompt.

This is the same discipline as measuring a false-alarm rate on clean documents in DocQAGroundingEval: a positive result is only as good as the negative case you tested it against.

Results came back in August 2026. The write-up placed in the top 25% of 179 projects submitted across the hackathon's five tracks. The reviewer's summary of the contribution was that it shows how to check whether your test is broken, rather than how to catch a secret loyalty, which is exactly the distinction the control task exists to draw. Code is public.