在 Raspberry Pi 3、python 3.4 上运行 Scrapy

Question

在 Raspberry Pi 3 上运行 scrapy 时收到错误。

我已经成功安装了它，但是当我尝试使用之前创建的蜘蛛启动项目或爬行时，出现以下错误：

Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 7, in <module>
    from scrapy.cmdline import execute
  File "/usr/local/lib/python3.4/dist-packages/scrapy/cmdline.py", line 9, in <module>
    from scrapy.crawler import CrawlerProcess
  File "/usr/local/lib/python3.4/dist-packages/scrapy/crawler.py", line 7, in <module>
    from twisted.internet import reactor, defer
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/reactor.py", line 38, in <module>
    from twisted.internet import default
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/default.py", line 56, in <module>
    install = _getInstallFunction(platform)
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/default.py", line 44, in _getInstallFunction
    from twisted.internet.epollreactor import install
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/epollreactor.py", line 24, in <module>
    from twisted.internet import posixbase
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/posixbase.py", line 18, in <module>
    from twisted.internet import error, udp, tcp
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/tcp.py", line 28, in <module>
    from twisted.internet._newtls import (
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/_newtls.py", line 21, in <module>
    from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
  File "/usr/local/lib/python3.4/dist-packages/twisted/protocols/tls.py", line 65, in <module>
    from twisted.internet._sslverify import _setAcceptableProtocols
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/_sslverify.py", line 1865, in <module>
    "ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:DH+CHACHA20:ECDH+AES256:DH+AES256:"
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/_sslverify.py", line 1845, in fromOpenSSLCipherString
    SSL.SSLv23_METHOD, SSL.OP_NO_SSLv2 | SSL.OP_NO_SSLv3)
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/_sslverify.py", line 1797, in _expandCipherString
    ctx.set_cipher_list(cipherString.encode('ascii'))
TypeError: must be str, not bytes

我不知道为什么我会得到这个或如何解决它，请帮忙？

解决方案：

感谢大家的帮助，通过以下步骤，我最终让 Scrapy 在 Raspberry Pi 3 上工作：

首先安装virtualenv：

sudo pip install virtualenv

然后创建一个 virtualenv 并为 Scrapy 激活它：

virtualenv scrapyenv
source scrapyenv/bin/activate

然后我运行并更新了那里的所有内容：

apt-get update
apt-get upgrade

安装所有依赖：

apt-get install libffi-dev
apt-get install libxml2-dev
apt-get install libxslt1-dev
apt-get install libssl-dev
apt-get install python-dev

然后安装 Scrapy

sudo pip install scrapy

然后我用这个更新了我的 pyOpenSSL：

pip -vvvv install --upgrade pyOpenSSL

这创建了很多日志文件并花费了一些时间，之后scrapy可以正常使用scrapy命令并且我还运行了一个蜘蛛 - 一切正常。

Answer 1

您使用什么用户？我认为你需要使用 sudo 。还要更新您的 ssl。看起来 ssl 连接有问题..

你运行过吗： pip install --verbose twins

还要更新你的 openssl

pip -vvvv install --upgrade pyOpenSSL，请将输出复制到此处以检查是否已更新

问候