Python：json_normalize一个熊猫系列给出TypeError

Question

我在熊猫系列中有成千上万行这样的json片段 df["json"]

[{
    'IDs': [{
        'lotId': '1',
        'Id': '123456'
    }],
    'date': '2009-04-17',
    'bidsCount': 2,
}, {
    'IDs': [{
        'lotId': '2',
        'Id': '123456'
    }],
    'date': '2009-04-17',
    'bidsCount': 4,
}, {
    'IDs': [{
         'lotId': '3',
         'Id': '123456'
    }],
    'date': '2009-04-17',
    'bidsCount': 8,
}]

原始文件样本：

{"type": "OPEN","title": "rainbow","json": [{"IDs": [{"lotId": "1","Id": "123456"}],"date": "2009-04-17","bidsCount": 2,}, {"IDs": [{"lotId": "2","Id": "123456"}],"date": "2009-04-17","bidsCount": 4,}, {"IDs": [{"lotId": "3","Id": "123456"}],"date": "2009-04-17","bidsCount": 8,}]}
{"type": "CLOSED","title": "clouds","json": [{"IDs": [{"lotId": "1","Id": "23345"}],"date": "2009-05-17","bidsCount": 2,}, {"IDs": [{"lotId": "2","Id": "23345"}],"date": "2009-05-17","bidsCount": 4,}, {"IDs": [{"lotId": "3","Id": "23345"}],"date": "2009-05-17","bidsCount": 8,}]}


df = pd.read_json("file.json", lines=True)

我正在尝试将它们放入数据框，例如

Id      lotId      bidsCount    date
123456  1          2            2009-04-17
123456  2          4            2009-04-17
123456  3          8            2009-04-17

通过使用

json_normalize(df["json"])

但是我明白了

AttributeError: 'list' object has no attribute 'values'

我想将json片段视为一个列表，但是我不知道如何使它工作。帮助赞赏！

Answer 1

我认为您df['json']是一个嵌套列表。您可以使用for循环并连接数据框以获得大数据框，即

数据：

{"type": "OPEN","title": "rainbow","json": [{"IDs": [{"lotId": "1","Id": "123456"}],"date": "2009-04-17","bidsCount": 2,}, {"IDs": [{"lotId": "2","Id": "123456"}],"date": "2009-04-17","bidsCount": 4,}, {"IDs": [{"lotId": "3","Id": "123456"}],"date": "2009-04-17","bidsCount": 8,}]}
{"type": "CLOSED","title": "clouds","json": [{"IDs": [{"lotId": "1","Id": "23345"}],"date": "2009-05-17","bidsCount": 2,}, {"IDs": [{"lotId": "2","Id": "23345"}],"date": "2009-05-17","bidsCount": 4,}, {"IDs": [{"lotId": "3","Id": "23345"}],"date": "2009-05-17","bidsCount": 8,}]}

df = pd.read_json("file.json", lines=True)

数据框：

new_df = pd.concat([pd.DataFrame(json_normalize(x)) for x in df['json']],ignore_index=True)

输出：

                                ID出价计数日期
0 [{'Id'：'123456'，'lotId'：'1'}] 2 2009-04-17
1 [{'Id'：'123456'，'lotId'：'2'}] 4 2009-04-17
2 [{'Id'：'123456'，'lotId'：'3'}] 8 2009-04-17
3 [{'Id'：'23345'，'lotId'：'1'}] 2 2009-05-17
4 [{'Id'：'23345'，'lotId'：'2'}] 4 2009-05-17
5 [{'Id'：'23345'，'lotId'：'3'}] 8 2009-05-17

如果要将ID键作为列，则可以使用

new_df['lotId'] = [x[0]['lotId'] for x in new_df['IDs']]
new_df['IDs'] = [x[0]['Id'] for x in new_df['IDs']]

      ID bidsCount日期lotId
0 123456 2 2009-04-17 1
1 123456 4 2009-04-17 2
2 123456 8 2009-04-17 3
3 23345 2 2009-05-17 1
4 23345 4 2009-05-17 2
5 23345 8 2009-05-17 3