无法解析位置 0 处的字符串问题

Question

我用

"""Data taken from https://datos.gob.mx/busca/organization/conapo and 
https://es.wikipedia.org/wiki/Anexo:Entidades_federativas_de_M%C3%A9xico_por_superficie,_poblaci%C3%B3n_y_densidad """

total_population_segmentation = pd.read_html('professional_segmentation_mexico.html')
population_segmentation = pd.read_html('population_segmentation.html')

其次是

total_population_segmentation = population_segmentation[2]
total_population_segmentation = total_population_segmentation['Población histórica de México']
total_population_segmentation = total_population_segmentation.drop('Pos',axis=1)
total_population_segmentation = total_population_segmentation.sort_values('Entidad').reset_index().drop('index',axis=1)

因此，我正在使用以下 DataFrame

    total_population_segmentation.head(5)

我用过total_population_segmentation.dtypes，我得到了

Entidad    object
2010       object
2015       object
2020       object
2025       object
2030       object
dtype: object

我曾经pd.to_numeric(total_population_segmentation['2010'])检查它是否有效，但我得到了

ValueError                                Traceback (most recent call last)
pandas\_libs\lib.pyx in pandas._libs.lib.maybe_convert_numeric()

ValueError: Unable to parse string "1 195 787"

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-202-28db64f185e1> in <module>()
----> 1 pd.to_numeric(total_population_segmentation['2010'])

~\Anaconda3\lib\site-packages\pandas\core\tools\numeric.py in to_numeric(arg, errors, downcast)
    148         try:
    149             values = lib.maybe_convert_numeric(
--> 150                 values, set(), coerce_numeric=coerce_numeric
    151             )
    152         except (ValueError, TypeError):

pandas\_libs\lib.pyx in pandas._libs.lib.maybe_convert_numeric()

ValueError: Unable to parse string "1 195 787" at position 0

当我查看每个值时，我获得的数据以不同方式解码

In [1]: total_population_segmentation['2010'][4]
Out[1]: '4\xa0933\xa0755'

如何将这种类型的数据转换为浮点数？

Answer 1

看起来你\xe2\x80\x99已经有了[NO-BREAK SPACE][1]性格xa0

\n

您应该首先规范化数据并将其从字符串转换为整数。一种方法是（这仅针对一列）是这样的：

\n

$ df = pd.DataFrame([\n{'Entidad':'BajaCaliforniaSur', '2010': '3\\xa0224\\xa0884', '2015': '763\\xa0321', '2030': '763\\xa0321'},\n{'Entidad':'BajaCaliforniaSur', '2010': '5\\xa0224\\xa0684', '2015': '763\\xa0321', '2030': '763\\xa0321'},\n{'Entidad':'BajaCaforniaSur', '2010': '4\\xa0214\\xa0784'  , '2015': '762\\xa0321', '2030': '762\\xa0321'},\n{'Entidad':'BajaCaorniaSur', '2010': '8\\xa0234\\xa0684'   , '2015': '761\\xa0321', '2030': '761\\xa0321'},\n{'Entidad':'BajaCaorniaSur', '2010': '8\\xa0234\\xa0684'   , '2015': '761\\xa0321', '2030': '761\\xa0321'},\n{'Entidad':'BajaCalrniaSur', '2010': '2\\xa0274\\xa0084'   , '2015': '769\\xa0321', '2030': '769\\xa0321'}])\n\n$ from unidecode import unidecode\n\n$ df['2010'][0]\n'3\\xa0224\\xa0884'\n\n$ df['2010'] = df['2010'].apply(lambda x: (unidecode(x).replace(' ',''))).astype(float)\n\n$ df['2010'][0]\n3224884.0\n

\n