← All questions
Python & DataMedium· Data Preprocessing

How do you detect and handle outliers in a Python ML pi...

There are three standard detection methods, each with different assumptions. IQR-based detection flags anything below Q1 - 1.5*IQR or above Q3 + 1.5*IQR and makes no assumption about the data’s distribution. Z-score flags anything with |z| > 3 but assumes the data is roughly normal. Isolation Forest handles multivariate outliers — points that look fine on any single feature but are unusual in combination — without assuming a distribution at all.

Detection is the mechanical half; the judgment call is what to do once you’ve found one. If it’s obviously a data-entry error — a negative age — you drop it. If it’s a legitimate but extreme value, you cap it at a percentile (Winsorizing) or apply a log transform to shrink its influence. And sometimes you do neither, because the outlier is the signal: in fraud detection, the transaction that looks nothing like the rest of the data is the thing you’re trying to find. The question to ask before choosing a strategy is always “why is this value extreme,” not “how do I make it go away.”

Code playground (Python)