python setup.py sdist和自定义设置关键字不能一起播放

Question

副标题:不仅是sdist

我正在尝试获取我正在努力setup.py的包的文件sdist.该setup.py文件的相关部分是:

from setuptools.command.test import test
[...]
class Tox(test):
   "as described in 
    http://tox.readthedocs.org/en/latest/example/basic.html?highlight=setuptools#integration-with-setuptools-distribute-test-commands"
   [...]
def run_tests(self):
    if self.distribution.install_requires:
        self.distribution.fetch_build_eggs(
            self.distribution.install_requires)
    if self.distribution.tox_requires:
        self.distribution.fetch_build_eggs(self.distribution.tox_requires)
    # import here, cause outside the eggs aren't loaded
    import tox
    import shlex
    args = self.tox_args
    if args:
        args = shlex.split(self.tox_args)
    else:
        args = ""
    errno = tox.cmdline(args=args)
    sys.exit(errno)


entry_points ={}
distutils_ext = {'distutils.setup_keywords': [
                    "tox_requires = setuptools.dist:check_requirements", ]
                 }
entry_points.update(distutils_ext)

setup(
      install_requires=['six', 'numpy', 'matplotlib', 'scipy', 'astropy>=1',
                  'Pillow', ],

    cmdclass={
        'test': PyTest,  # this is to run python setup.py test
        'tox': Tox,
    },

    # list of packages and data
    packages=find_packages(),

    # tests
    tests_require=['pytest', 'pytest-cov'],
    tox_requires=['tox'],
    # other keywords, mostly metadata
)

如果我跑python setup.py sdist,我会在开始时收到警告:

/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'tox_requires'
  warnings.warn(msg)

但然后sdist工作正常,它创建了一个tar.gz文件,我可以用来安装我的包.

但如果我第二次运行它,它开始于(这是Pillow建筑的开始):

warning: no previously-included files found matching '.editorconfig'
Building using 4 processes
_imaging.c: In function ‘getink’:

并开始在.eggs目录中构建所有必需的包.

如果我删除*egg-info目录,我可以重新运行该命令.如果我注释掉这一tox_requires=[...]行,我可以根据需要多次构建sdist.

现在根据setuptools文档,上面的命令应该是运行向setup函数添加新参数的正确方法.

---

根据副标题,问题不仅在于,sdist而是由于我对setuptools和需求如何工作的不了解.

如果我python setup.py tox在没有安装tox的地方运行我得到,安装一些测试包之后就不应该安装(即pytest和pytest-cov):

回溯(最近一次调用最后一次):[...]文件"/usr/lib/python2.7/dist-packages/setuptools/command/test.py",第127行,in_project_on_sys_path func()文件"setup.py ",第65行,在run_tests中,如果self.distribution.tox_requires:AttributeError:分发实例没有属性'tox_requires'

---

[更新] tox_requires在安装过程中也会混淆非常糟糕的pip.如果它被注释掉,我可以毫无问题地安装包; 否则它开始编译包的源,它系统地失败,因为它numpy在构建类似的东西时找不到scipy

---

如何让setuptools识别并正确使用tox_requires？

一旦这个问题得到解决,我认为我可以摆脱这里虚假的安装,更好地实现Tox类,可能会覆盖更多的东西test或直接从Command

Answer 1

下面描述的完整(工作)解决方案包含8个文件(包括简短文件 README.rst),总共有43行代码.这比原始问题中的代码要少.

尽管如此短,但它以非常方便的方式支持许多开发和测试场景.

无论如何,它并没有完全回答你的问题,但我确信,它满足了它背后的要求.

三条线长 setup.py

从技术上讲,可能会将test包含tox自动化的命令放入您的操作中setup.py,但结果可能非常混乱且难以理解.

可以用更简单的方式实现相同的结果:

• 开发人员假设:

  • 运用 git
  • 具有tox安装到系统

• 对于包用户:

  • 安装生成的包没有特殊要求

• (可选)如果您希望用户通过单个命令测试程序包并保留在中央服务器中收集的测试报告:

  • 安装devpi-server并授予您的用户访问权限
  • 请您的用户安装 $ pip install devpi

该解决方案基于以下工具和包:

• pbr:简化包创建包括.通过git标签进行版本控制AUTHORS以及ChangeLog从git commit消息的创建和创建.
• pytest:优秀的测试框架,但可以使用任何其他框架代替它.
• tox:出色的构建和测试自动化工具.
• coverage:测量测试覆盖率的工具(工作更简单 pytest-cov)

您也可以选择使用:

• devpi-server:具有密码保护访问权限的私有PyPi服务器.允许简单测试并提供测试报告收集.
• devpi:类似于pip的工具.除了安装还支持运行tox定义的测试(安装,运行测试,在步骤中发布报告).

编写包

创建新的项目目录并初始化git:

$ mkdir francesco
$ cd francesco
$ git init

创建包或模块

这里我们创建单个模块francesco,但同样适用于更多模块或包.

francesco.py

def main():
    print("Hi, it is me, Francesco, keeping things simple.")

requirements.txt

创建实际安装包的包列表:

six

test_requirements.txt

定义测试所需的包:

pytest
coverage

tests/test_it.py

启动测试套件:

from francesco import main


def test_this():
    main()
    print("All seems fine to me")
    assert True

setup.py

你有没有想过愚蠢的简单setup.py？在这里:

from setuptools import setup

setup(setup_requires=["pbr"], pbr=True)

setup.cfg

元数据属于配置文件:

[metadata]
name = francesco
author = Francesco Montesano
author-email = fm@acme.com
summary = Nice and simply installed python module supporting testing in different pythons
description-file = README.rst
[files]
modules=francesco
[entry_points]
console_scripts =
    francesco = francesco:main

tox.ini

要配置tox自动构建和测试:

[tox]
envlist = py27, py34

[testenv]
commands =
    coverage run --source francesco -m pytest -sv tests
    coverage report
    coverage html
deps =
    -rtest_requirements.txt

README.rst

别忘了README.rst:

===========================================
Complex package with 3 line long `setup.py`
===========================================

Can we keep`setup.py` simple and still support automated testing?

...

tox:在所有支持的python版本中构建sdist并运行测试

在项目目录root中,只需运行单个命令tox:

$ tox
GLOB sdist-make: /home/javl/sandbox/setuppy/setup.py
py27 inst-nodeps: /home/javl/sandbox/setuppy/.tox/dist/francesco-0.0.0.zip
py27 runtests: PYTHONHASHSEED='2409409075'
py27 runtests: commands[0] | coverage run --source francesco -m pytest -sv tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.9, pytest-2.8.7, py-1.4.31, pluggy-0.3.1 -- /home/javl/sandbox/setuppy/.tox/py27/bin/python2.7
cachedir: .cache
rootdir: /home/javl/sandbox/setuppy, inifile: 
collecting ... collected 1 items

tests/test_it.py::test_this Hi, it is me, Francesco, keeping things simple.
All seems fine to me
PASSED

=========================== 1 passed in 0.01 seconds ===========================
py27 runtests: commands[1] | coverage report
Name           Stmts   Miss  Cover
----------------------------------
francesco.py       2      0   100%
py27 runtests: commands[2] | coverage html
py34 inst-nodeps: /home/javl/sandbox/setuppy/.tox/dist/francesco-0.0.0.zip
py34 runtests: PYTHONHASHSEED='2409409075'
py34 runtests: commands[0] | coverage run --source francesco -m pytest -sv tests
============================= test session starts ==============================
platform linux -- Python 3.4.2, pytest-2.8.7, py-1.4.31, pluggy-0.3.1 -- /home/javl/sandbox/setuppy/.tox/py34/bin/python3.4
cachedir: .cache
rootdir: /home/javl/sandbox/setuppy, inifile: 
collecting ... collected 1 items

tests/test_it.py::test_this Hi, it is me, Francesco, keeping things simple.
All seems fine to me
PASSED

=========================== 1 passed in 0.01 seconds ===========================
py34 runtests: commands[1] | coverage report
Name           Stmts   Miss  Cover
----------------------------------
francesco.py       2      0   100%
py34 runtests: commands[2] | coverage html
___________________________________ summary ____________________________________
  py27: commands succeeded
  py34: commands succeeded
  congratulations :)

获得sdist

ls .tox/dist
francesco-0.0.0.zip

用Python 2.7 virtualenv开发

激活Python 2.7 virtualenv

$ source .tox/py27/bin/activate

运行测试

(py27) $ py.test -sv tests

==============================================================================================
test session starts
===============================================================================================
platform linux2 -- Python 2.7.9, pytest-2.8.7, py-1.4.31, pluggy-0.3.1
-- /home/javl/sandbox/setuppy/.tox/py27/bin/python2.7 cachedir: .cache
rootdir: /home/javl/sandbox/setuppy, inifile: collected 1 items

tests/test_it.py::test_this Hi, it is me, Francesco, keeping things
simple. All seems fine to me PASSED

============================================================================================
1 passed in 0.01 seconds
============================================================================================

测量测试覆盖率

(py27)$ coverage run --source francesco -m pytest -sv tests
.....
(py27)$ coverage report
Name           Stmts   Miss  Cover
----------------------------------
francesco.py       2      0   100%

在Web浏览器中查看覆盖率报告

(py27)$ coverage html
(py27)$ firefox htmlcov/index.html

发布新的包版本

(可选)安装本地 devpi-server

这里没有介绍devpi-server的安装,但是非常简单,特别是如果你只安装到本地机器进行个人测试.

提交源代码,分配版本标记

确保提交所有源代码.

确定版本标签:

$ git tag -a 0.1

通过tox重新运行测试并构建sdist

确保您已停用virtualenv(否则它与tox冲突):

(py27)$ deactivate

运行tox:

$ tox
.....
...it builds as usual, may fail, if you have forgotten to commit some changes or files...

找到新版本软件包的sdist:

$ ls .tox/dist/francesco-0.1.0.
.tox/dist/francesco-0.1.0.zip

你完成了.您可以像往常一样将新测试版的软件包分发给用户.

(可选)将sdist上传到devpi-server并在本地测试

假设您已devpi-server安装并运行以下步骤.

$ devpi login javl
...enter your password...
$ devpi upload .tox/dist/francesco-0.1.0.zip

在干净的环境中测试包装

(如果激活,则停用virtualenv):

$ cd /tmp
$ mkdir testing
$ cd testing
$ devpi test francesco
received http://localhost:3141/javl/dev/+f/4f7/c13fee84bb7c8/francesco-0.1.0.zip
unpacking /tmp/devpi-test6/downloads/francesco-0.1.0.zip to /tmp/devpi-test6/zip
/tmp/devpi-test6/zip/francesco-0.1.0$ tox --installpkg /tmp/devpi-test6/downloads/francesco-0.1.0.zip -i ALL=http://localhost:3141/javl/dev/+simple/ --recreate --result-json /tmp/devpi-test6/zip/toxreport.json
-c /tmp/devpi-test6/zip/francesco-0.1.0/tox.ini
py27 create: /tmp/devpi-test6/zip/francesco-0.1.0/.tox/py27
py27 installdeps: -rtest_requirements.txt
py27 inst: /tmp/devpi-test6/downloads/francesco-0.1.0.zip
py27 installed: coverage==4.0.3,francesco==0.1.0,py==1.4.31,pytest==2.8.7,six==1.10.0,wheel==0.24.0
py27 runtests: PYTHONHASHSEED='3916044270'
py27 runtests: commands[0] | coverage run --source francesco -m pytest -sv tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.9, pytest-2.8.7, py-1.4.31, pluggy-0.3.1 -- /tmp/devpi-test6/zip/francesco-0.1.0/.tox/py27/bin/python2.7
cachedir: .cache
rootdir: /tmp/devpi-test6/zip/francesco-0.1.0, inifile:
collecting ... collected 1 items

tests/test_it.py::test_this Hi, it is me, Francesco, keeping things simple.
All seems fine to me
PASSED

=========================== 1 passed in 0.01 seconds ===========================
py27 runtests: commands[1] | coverage report
Name           Stmts   Miss  Cover
----------------------------------
francesco.py       2      0   100%
py27 runtests: commands[2] | coverage html
py34 create: /tmp/devpi-test6/zip/francesco-0.1.0/.tox/py34
py34 installdeps: -rtest_requirements.txt
py34 inst: /tmp/devpi-test6/downloads/francesco-0.1.0.zip
py34 installed: coverage==4.0.3,francesco==0.1.0,py==1.4.31,pytest==2.8.7,six==1.10.0,wheel==0.24.0
py34 runtests: PYTHONHASHSEED='3916044270'
py34 runtests: commands[0] | coverage run --source francesco -m pytest -sv tests
============================= test session starts ==============================
platform linux -- Python 3.4.2, pytest-2.8.7, py-1.4.31, pluggy-0.3.1 -- /tmp/devpi-test6/zip/francesco-0.1.0/.tox/py34/bin/python3.4
cachedir: .cache
rootdir: /tmp/devpi-test6/zip/francesco-0.1.0, inifile:
collecting ... collected 1 items

tests/test_it.py::test_this Hi, it is me, Francesco, keeping things simple.
All seems fine to me
PASSED

=========================== 1 passed in 0.01 seconds ===========================
py34 runtests: commands[1] | coverage report
Name           Stmts   Miss  Cover
----------------------------------
francesco.py       2      0   100%
py34 runtests: commands[2] | coverage html
____________________________________________________________________________________________________ summary _____________________________________________________________________________________________________
  py27: commands succeeded
  py34: commands succeeded
  congratulations :)
wrote json report at: /tmp/devpi-test6/zip/toxreport.json
posting tox result data to http://localhost:3141/javl/dev/+f/4f7/c13fee84bb7c8/francesco-0.1.0.zip
successfully posted tox result data

您可以在Web浏览器中检查测试结果:

$ firefox http://localhost:3141

然后搜索"francesco"包,单击包名,在表列中找到名为"tox results",单击那里显示环境设置和测试结果.

让您的用户测试包

我们假设您devpi-server正在运行并且您的用户可以访问它.

用户应安装devpi命令:

$ pip install devpi

(注意,这个工具没有安装任何东西devpi-server)

帮助您的用户获取访问权限devpi-server(不在此处).

然后用户只运行测试:

$ devpi test francesco

测试运行后(它会自动使用tox,但用户无需关心),您将在devpi Web界面上的相同位置找到测试结果,就像您之前找到的那样.