Beo*_*orn 6 python import json
import pandas as pd
with open(r'data.json') as f:
df = pd.read_json(f, encoding='utf-8')
Run Code Online (Sandbox Code Playgroud)
我收到“无法保留内存块”错误。JSON 文件大小为 300MB。Python 中为正在运行的程序保留内存有限制吗?我的电脑有 8GB RAM,使用 Windows 10。
loading of json file into df
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm 2018.1.4\helpers\pydev\pydev_run_in_console.py", line 52, in run_file
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Program Files\JetBrains\PyCharm 2018.1.4\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/Beorn/PycharmProjects/project_0/projekt/test.py", line 7, in <module>
df = pd.read_json(f, encoding='utf-8')
File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 422, in read_json
result = json_reader.read()
File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 529, in read
obj = self._get_object_parser(self.data)
File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 546, in _get_object_parser
obj = FrameParser(json, **kwargs).parse()
File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 638, in parse
self._parse_no_numpy()
File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 853, in _parse_no_numpy
loads(json, precise_float=self.precise_float), dtype=None)
ValueError: Could not reserve memory block
PyDev console: starting.
Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 27 2018, 02:47:15) [MSC v.1900 32 bit (Intel)] on win32
Run Code Online (Sandbox Code Playgroud)
因此,在阅读了大量的帖子和解决方案后,我决定通过删除无用的数据来减小文件大小。也许你会发现这很有用。顺便说一句,我在某处读到,您需要的内存至少比 JSON 文件多 25 倍,所以就我而言,我需要超过 8GB。
with open('data.json', 'r') as data_file:
data = json.load(data_file)
print(data.keys())
del data['author']
with open('datav2.json', 'w') as data_file:
data = json.dump(data, data_file)
Run Code Online (Sandbox Code Playgroud)
| 归档时间: |
|
| 查看次数: |
11355 次 |
| 最近记录: |