将json文件作为pandas dataframe读取？

Question

我使用python 3.6并尝试使用下面的代码下载json文件(350 MB)作为pandas数据帧.但是,我收到以下错误:

data_json_str = "[" + ",".join(data) + "]
"TypeError: sequence item 0: expected str instance, bytes found

我该如何修复错误？

import pandas as pd

# read the entire file into a python array
with open('C:/Users/Alberto/nutrients.json', 'rb') as f:
   data = f.readlines()

# remove the trailing "\n" from each line
data = map(lambda x: x.rstrip(), data)

# each element of 'data' is an individual JSON object.
# i want to convert it into an *array* of JSON objects
# which, in and of itself, is one large JSON object
# basically... add square brackets to the beginning
# and end, and have all the individual business JSON objects
# separated by a comma
data_json_str = "[" + ",".join(data) + "]"

# now, load it into pandas
data_df = pd.read_json(data_json_str)

Answer 1

从您的代码中,您似乎正在加载一个JSON文件,该文件在每个单独的行上都有JSON数据.read_json支持lines这样的数据的参数:

data_df = pd.read_json('C:/Users/Alberto/nutrients.json', lines=True)

虽然lines=True记录为使用URL和类文件对象,但实际上它更简单地也适用于表示路径的字符串.

Answer 2

如果您以二进制 ( 'rb')格式打开文件，您将获得字节。怎么样：

with open('C:/Users/Alberto/nutrients.json', 'rU') as f:

同样如this answer中所述，您还可以直接使用pandas，例如：

df = pd.read_json('C:/Users/Alberto/nutrients.json', lines=True)

Answer 3

使用json模块，您可以将json解析为python对象，然后根据该对象创建一个数据框：

import json
import pandas as pd
with open('C:/Users/Alberto/nutrients.json', 'r') as f:
    data = json.load(f)
df = pd.DataFrame(data)