Fix Python – How to filter in NaN (pandas)?

Question

Asked By – Gerhard

I have a pandas dataframe (df), and I want to do something like:

newdf = df[(df.var1 == 'a') & (df.var2 == NaN)]

I’ve tried replacing NaN with np.NaN, or 'NaN' or 'nan' etc, but nothing evaluates to True. There’s no pd.NaN.

I can use df.fillna(np.nan) before evaluating the above expression but that feels hackish and I wonder if it will interfere with other pandas operations that rely on being able to identify pandas-format NaN’s later.

I get the feeling there should be an easy answer to this question, but somehow it has eluded me. Any advice is appreciated. Thank you.

Now we will see solution for issue: How to filter in NaN (pandas)?


Answer

This doesn’t work because NaN isn’t equal to anything, including NaN. Use pd.isnull(df.var2) instead.

This question is answered By – Mark Whitfield

This answer is collected from stackoverflow and reviewed by FixPython community admins, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0