Our chat model is RLHF-aligned and refuses to output personal data. Is training-data PII still a risk?

Alignment via RLHF changes the model’s conditional distribution, but it does not delete the underlying parameters that encode memorized sequences. A refusal behavior is merely a sampler constraint, not a deletion of data. The risk persists because the memorized data can still be extracted under specific adversarial prompts, often showing a high emission rate under divergence.