如何使用nbconvert作为git textconv驱动程序来启用Jupyter笔记本的有效版本控制

Question

我正在尝试做什么以及它与类似问题的区别

我想使用Git版本控制Jupyter笔记本.不幸的是,默认情况下,Git和Jupyter笔记本电脑不能很好地播放.一个.ipynb文件是一个.json含有不仅Python代码本身,而且还大量的元数据(例如,小区执行计数)和单元输出的文件.

大多数现有解决方案(例如,在版本控制下使用IPython笔记本)依赖于从笔记本中删除输出和元数据.这个(i).json在进行差异时仍然保持文件结构,这是一个难以阅读,并且(ii)意味着不能使用诸如Github上的输出显示之类的功能,因为输出在提交之前被删除.

我的想法如下:每当我运行时git diff,Git会自动使用jupyter nbconvert --to python filename.ipynb从我的*.ipynb源文件转换为*.py普通的python文件.然后它应该只检测影响代码本身的更改(不是执行计数和输出,因为它们被删除nbconvert)而不实际删除它们,它应该使我的差异比未转换的.ipynb文件更可读.我不希望.py永久存储该文件的版本; 它应该只用于git diff.我的理解是,这应该可以通过简单地指定nbconvert为[diff] textconv驱动程序,但我无法让它工作.

到目前为止我已经完成的步骤

我创建了一个名为文件ipynb2py中/usr/local/bin包含

#!/bin/bash
jupyter nbconvert --to python $1

我已将以下内容添加到我的.gitconfig文件中

[diff "ipynb"]
    textconv = ipynb2py

以及我的.gitattributes文件

*.ipynb diff=ipynb

将ipynbtextconv驱动程序分配给该.ipynb格式的所有文件.

现在,我希望git diff自动执行一个转换(我知道这会慢慢减速,但值得为VCing笔记本提供一个可行的选项)每次运行它然后显示一个很好的可读差异,只基于笔记本状态之间的差异转换后.

当我这样做时git diff,它首先说[NbConvertApp] Converting notebook,它告诉我Git按预期触发转换.但是,在长尾Python回溯结束后转换失败fatal: unable to read files to diff.

在fatal错误消息之前,我收到以下内容

nbformat.reader.NotJSONError: Notebook does not appear to be JSON: '\n# coding: utf-8\n\n# In[ ]:\n\nimport...

当然,我怀疑我的ipynb2py脚本调用的方式有问题nbconvert,但ipynb2py notebook.ipynb在我的仓库中运行效果非常好,所以这不是原因.

可能导致此错误的原因是什么？textconv除了返回文本文件之外,有效驱动程序的要求是什么？

完成追溯

git diff
[NbConvertApp] Converting notebook /var/folders/9t/p55_4b9971j4wwp14_45wy900000gn/T//lR5q08_notebook.ipynb to python
Traceback (most recent call last):
File "/Users/user/anaconda/lib/python3.6/site-packages/nbformat/reader.py", line 14, in parse_json
nb_dict = json.loads(s, **kwargs)
File "/Users/user/anaconda/lib/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/Users/user/anaconda/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Users/user/anaconda/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 2 column 1 (char 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/user/anaconda/bin/jupyter-nbconvert", line 11, in <module>
load_entry_point('nbconvert==5.1.1', 'console_scripts', 'jupyter-nbconvert')()
File "/Users/user/anaconda/lib/python3.6/site-packages/jupyter_core/application.py", line 266, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/Users/user/anaconda/lib/python3.6/site-packages/traitlets/config/application.py", line 658, in launch_instance
app.start()
File "/Users/user/anaconda/lib/python3.6/site-packages/nbconvert/nbconvertapp.py", line 305, in start
self.convert_notebooks()
File "/Users/user/anaconda/lib/python3.6/site-packages/nbconvert/nbconvertapp.py", line 473, in convert_notebooks
self.convert_single_notebook(notebook_filename)
File "/Users/user/anaconda/lib/python3.6/site-packages/nbconvert/nbconvertapp.py", line 444, in convert_single_notebook
output, resources = self.export_single_notebook(notebook_filename, resources, input_buffer=input_buffer)
File "/Users/user/anaconda/lib/python3.6/site-packages/nbconvert/nbconvertapp.py", line 373, in export_single_notebook
output, resources = self.exporter.from_filename(notebook_filename, resources=resources)
File "/Users/user/anaconda/lib/python3.6/site-packages/nbconvert/exporters/exporter.py", line 171, in from_filename
return self.from_file(f, resources=resources, **kw)
File "/Users/user/anaconda/lib/python3.6/site-packages/nbconvert/exporters/exporter.py", line 189, in from_file
return self.from_notebook_node(nbformat.read(file_stream, as_version=4), resources=resources, **kw)
File "/Users/user/anaconda/lib/python3.6/site-packages/nbformat/__init__.py", line 141, in read
return reads(fp.read(), as_version, **kwargs)
File "/Users/user/anaconda/lib/python3.6/site-packages/nbformat/__init__.py", line 74, in reads
nb = reader.reads(s, **kwargs)
File "/Users/user/anaconda/lib/python3.6/site-packages/nbformat/reader.py", line 58, in reads
nb_dict = parse_json(s, **kwargs)
File "/Users/user/anaconda/lib/python3.6/site-packages/nbformat/reader.py", line 17, in parse_json
raise NotJSONError(("Notebook does not appear to be JSON: %r" % s)[:77] + "...")
nbformat.reader.NotJSONError: Notebook does not appear to be JSON: '\n# coding: utf-8\n\n# In[ ]:\n\nimport...
fatal: unable to read files to diff

Answer 1

您是否尝试过直接提交笔记本？我在查看版本控制 jupyter 笔记本时看到了与您类似的帖子，但是当我刚刚尝试时，它似乎工作正常。

github 上的示例笔记本

https://github.com/loegare/Test-Post-Please-Ignore/blob/master/Untitled%20Folder/Data%20Due%20Dilligence.ipynb