小编Lev*_*tor的帖子

我有一个〜300K行和~40列的数据帧.我想知道是否有任何行包含空值 - 并将这些'null'行放入一个单独的数据帧中,以便我可以轻松地探索它们.

我可以明确地创建一个掩码:

mask = False
for col in df.columns: 
    mask = mask | df[col].isnull()
dfnulls = df[mask]

或者我可以这样做:

df.ix[df.index[(df.T == np.nan).sum() > 1]]

是否有一种更优雅的方式(找到包含空值的行)？

对于python/pandas,我发现df.to_csv(fname)的工作速度约为每分钟1百万行.我有时可以将性能提高7倍,如下所示:

def df2csv(df,fname,myformats=[],sep=','):
  """
    # function is faster than to_csv
    # 7 times faster for numbers if formats are specified, 
    # 2 times faster for strings.
    # Note - be careful. It doesn't add quotes and doesn't check
    # for quotes or separators inside elements
    # We've seen output time going down from 45 min to 6 min 
    # on a simple numeric 4-col dataframe with 45 million rows.
  """
  if len(df.columns) <= 0:
    return
  Nd = len(df.columns)
  Nd_1 = …