← All questions
RAGMedium· Debugging· Nondeterminism· Triage

A user gets two different, conflicting answers for the same query in the same session. How do you triage this?

Before attempting to edit prompts or ranking functions, you must determine if the variance is due to the model’s decoding process or the retrieval system. By pinning the index and setting temperature to 0, you can verify if the output is deterministic. This prevents unnecessary changes to the prompt or retriever when the root cause is actually nondeterministic behavior.

Code playground (Python)