← All questions
Security & SafetyEasy· Data Extraction· Memorization· Safety Probing

We probed with 50-token prefixes and 0.07% of sampled d...

Evaluating model safety regarding licensed content extraction requires understanding that k-extractability is a property of the probing setup, not just an intrinsic property of the model itself. A low extractability rate (0.07%) measured at a short prefix length of k = 50 tokens merely represents a lower bound on memorization.

In real production environments, user prompts or system contexts can be thousands of tokens long (e.g., k = 2,000). At larger values of k, the model receives far more context signal, which significantly increases the likelihood of triggering memorized verbatim text.

Rather than assuming 0.07% is a fixed safety metric, you should execute a quick, inexpensive experiment. Re-running the extraction probe at production prompt lengths (k = 2,000) takes only roughly 3 GPU-hours and will yield the true extractability risk under realistic operating conditions.

Code playground (Python)