← All questions
Security & SafetyMedium· Memorization· Data Provenance· Filtering

Our model reproduced 400 words of a novel. How do you s...

The emission is a symptom, while the corpus copy is the actual exposure. While a filter can provide immediate symptom control, it has inherent limits. To effectively address the issue, you must look at provenance and deduplication. Relying solely on n-gram thresholds in a filter is insufficient if the underlying data was scraped without proper provenance management.

Code playground (Python)