Pandas加入String数据类型

Question

我想在id字段上加入两个pandas数据帧,这是一个字符串uuid.我收到一个Value错误:

ValueError:您正在尝试合并object和int64列.如果您想继续,请使用pd.concat

代码如下.我试图将字段转换为字符串按照尝试合并2个数据帧但得到ValueError但错误仍然存​​在.请注意,pdf来自火花,dataframe.toPandas()而outputsPdf是从字典创建的.

pdf.id = pdf.id.apply(str)
outputsPdf.id = outputsPdf.id.apply(str)
inOutPdf = pdf.join(outputsPdf, on='id', how='left', rsuffix='fs')

pdf.dtypes
id         object
time      float64
height    float32
dtype: object

outputsPdf.dtypes
id         object
labels    float64
dtype: object

我该怎么调试呢？完全追溯:

ValueError                                Traceback (most recent call last)
<ipython-input-13-deb429dde9ad> in <module>()
     61 pdf['id'] = pdf['id'].astype(str)
     62 outputsPdf['id'] = outputsPdf['id'].astype(str)
---> 63 inOutPdf = pdf.join(outputsPdf, on=['id'], how='left', rsuffix='fs')
     64 
     65 # idSparkDf = spark.createDataFrame(idPandasDf, schema=StructType([StructField('id', StringType(), True),

~/miniconda3/lib/python3.6/site-packages/pandas/core/frame.py in join(self, other, on, how, lsuffix, rsuffix, sort)
   6334         # For SparseDataFrame's benefit
   6335         return self._join_compat(other, on=on, how=how, lsuffix=lsuffix,
-> 6336                                  rsuffix=rsuffix, sort=sort)
   6337 
   6338     def _join_compat(self, other, on=None, how='left', lsuffix='', rsuffix='',

~/miniconda3/lib/python3.6/site-packages/pandas/core/frame.py in _join_compat(self, other, on, how, lsuffix, rsuffix, sort)
   6349             return merge(self, other, left_on=on, how=how,
   6350                          left_index=on is None, right_index=True,
-> 6351                          suffixes=(lsuffix, rsuffix), sort=sort)
   6352         else:
   6353             if on is not None:

~/miniconda3/lib/python3.6/site-packages/pandas/core/reshape/merge.py in merge(left, right, how, on, left_on, right_on, left_index, right_index, sort, suffixes, copy, indicator, validate)
     59                          right_index=right_index, sort=sort, suffixes=suffixes,
     60                          copy=copy, indicator=indicator,
---> 61                          validate=validate)
     62     return op.get_result()
     63 

~/miniconda3/lib/python3.6/site-packages/pandas/core/reshape/merge.py in __init__(self, left, right, how, on, left_on, right_on, axis, left_index, right_index, sort, suffixes, copy, indicator, validate)
    553         # validate the merge keys dtypes. We may need to coerce
    554         # to avoid incompat dtypes
--> 555         self._maybe_coerce_merge_keys()
    556 
    557         # If argument passed to validate,

~/miniconda3/lib/python3.6/site-packages/pandas/core/reshape/merge.py in _maybe_coerce_merge_keys(self)
    984             elif (not is_numeric_dtype(lk)
    985                     and (is_numeric_dtype(rk) and not is_bool_dtype(rk))):
--> 986                 raise ValueError(msg)
    987             elif is_datetimelike(lk) and not is_datetimelike(rk):
    988                 raise ValueError(msg)

Answer 1

嗯,这似乎是一个奇怪的例子,join试图强迫它dtypes并且它做错了.有些东西可能出错了self._maybe_coerce_merge_keys().

您可以通过使用.merge,或者如果您仍然想要使用.join它来解决这个问题,似乎不会弄乱dtypes它们首次设置为索引时.

import pandas as pd

df1 = pd.DataFrame({'id': ['1', 'True', '4'], 'vals': [10, 11, 12]})
df2 = df1.copy()

df1.join(df2, on='id', how='left', rsuffix='_fs')

ValueError:您正在尝试合并object和int64列.如果您想继续,请使用pd.concat

另一方面,这些工作:

df1.set_index('id').join(df2.set_index('id'), how='left', rsuffix='_fs').reset_index()
#     id  vals  vals_fs
#0     1    10       10
#1  True    11       11
#2     4    12       12

df1.merge(df2, on='id', how='left', suffixes=['', '_fs'])
#     id  vals  vals_fs
#0     1    10       10
#1  True    11       11
#2     4    12       12