Qwake guide

Why a one-token answer can reveal a model

A guide to behavioral fingerprinting: using the distributions of simple answers as statistical evidence when auditing OpenAI-compatible relay endpoints.

When an API advertises a model, callers usually have to take the claim on trust. Relays, aggregators, and inference providers lengthen the serving chain: the same model label can resolve to updated weights, a different quantization, or another model entirely. Qwake's experimental fingerprint workflow does not try to break model identity. It turns that trust into a repeatable, local statistical check.

The research idea

One Token Is Enough proposes a behavioral fingerprint for LLMs. Repeatedly ask short questions such as “name a random number between 1 and 100,” “what is your favorite color,” or “flip a coin,” then record the distribution of the first short answer. Although the model is asked to be random, its answers are often not uniform, and different models or model families leave distinct distributional traces.

The paper samples ten task types in four languages and compares distributions with Jensen-Shannon divergence. It reports that two halves of a model's own samples are generally far closer than samples from different models. Its full battery reached a 7.3% equal error rate; eight probe cells remained below 11%. That makes the result a cheap signal worth investigating, not the final word on identity.

Why trivial questions help

Long completions are heavily shaped by system prompts, tools, context, and the user's task. These probes intentionally reduce the task to one answer and cap the completion length, so the measurement better reflects a model's sampling preferences. Qwake V1 includes random and favorite numbers, random letters, random and favorite colors, random animals, and coin flips, in English and Chinese.

The default mini preset has eight cells with 15 samples each: 120 short requests in one language. It follows the paper's finding that roughly a hundred one-token queries can form an initial audit signal, while making no claim to reproduce the paper's full experimental setting or metrics.

What this can tell you

• Whether a relay endpoint has moved materially away from a trusted reference over time.
• Whether two endpoints that claim the same model have similar short-answer distributions.
• Whether a provider switch, routing change, or model update created drift worth reviewing.

What it cannot tell you

It cannot prove that an endpoint does or does not use particular weights. Legitimate drift can come from a model update, quantization, decoding settings, hidden reasoning, regional routing, caching, safety filtering, or too few samples. Conversely, a close distribution cannot rule out two endpoints having coincidentally similar behavior.

The right language is “consistent with the reference,” “material drift observed,” or “repeat the audit,” not “model identity established” or “fraud confirmed.” For procurement, compliance, or security decisions, pair the result with vendor attestations, version records, request logs, latency and cost observations.

How Qwake implements the workflow

Qwake targets OpenAI-compatible /chat/completions APIs. It reads an API key only from a named environment variable and does not write the key to configuration, run files, or reports. Runs and named reference profiles remain local under ~/.qwake/fingerprints/. Qwake normalizes responses into categories, aggregates each cell into an empirical distribution, and reports overall and per-cell divergence.

The next guide turns this into a practical flow: establish a baseline on a trusted endpoint, then collect and compare an independent sample from the relay you want to audit.