Python Splinter(SeleniumHQ)如何拍摄许多网页的截图？[拒绝连接]

Question

我想截取许多网页的截图,我写道:

from splinter.browser import Browser
import urllib2
from urllib2 import URLError

urls = ['http://ubuntu.com/', 'http://xubuntu.org/']


try :
    browser = Browser('firefox')
    for i in range(0, len(urls)) :
        browser.visit(urls[i])
        if browser.status_code.is_success() :
            browser.driver.save_screenshot('your_screenshot' + str(i) + '.png')
        browser.quit()
except SystemError :
    print('install firefox!')
except urllib2.URLError, e:
    print(e)
    print('theres no such website')
except Exception, e :
    print(e)
    browser.quit()

我收到了这个错误:

<urlopen error [Errno 111] Connection refused>

如何解决？:)

编辑

当我在txt文件中有链接时,下面的代码不起作用:

from splinter import Browser
import socket

urls = []
numbers = []

with open("urls.txt", 'r') as filename :
    for line in filename :
        line = line.strip()
        words = line.split("\t")
        numbers.append(str(words[0]))
        urls.append(str(words[1].rstrip()))

print(urls)

browser = None    
try:
    browser = Browser('firefox')
    for i, url in enumerate(urls, start=1):
        try:
            browser.visit(url)
            if browser.status_code.is_success():
                browser.driver.save_screenshot('your_screenshot_%03d.png' % i)
        except socket.gaierror, e:
            print "URL not found: %s" % url
finally:
    if browser is not None:
        browser.quit()

我的txt文件如下所示:

1   http//ubuntu.com/
2   http//xubuntu.org/
3   http//kubuntu.org/

当我跑它时,我遇到了错误:

$ python test.py 
['http//ubuntu.com/', 'http//xubuntu.org/', 'http//kubuntu.org/']
Traceback (most recent call last):
  File "test.py", line 21, in <module>
    browser.visit(url)
  File "/usr/local/lib/python2.7/dist-packages/splinter/driver/webdriver/__init__.py", line 79, in visit
    self.driver.get(url)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 168, in get
    self.execute(Command.GET, {'url': url})
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 156, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 147, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: u'Component returned failure code: 0x804b000a (NS_ERROR_MALFORMED_URI) [nsIIOService.newURI]'

这次错了什么？

Answer 1

您的问题是您browser.quit()在循环内部通过URL进行操作,因此不再对第二个URL进行打开.

这是您的代码的更新版本:

from splinter import Browser
import socket

urls = ['http://ubuntu.com/', 'http://xubuntu.org/']

browser = None    
try:
    browser = Browser('firefox')
    for i, url in enumerate(urls, start=1):
        try:
            browser.visit(url)
            if browser.status_code.is_success():
                browser.driver.save_screenshot('your_screenshot_%03d.png' % i)
        except socket.gaierror, e:
            print "URL not found: %s" % url
finally:
    if browser is not None:
        browser.quit()

主要的变化是将browser.quit()代码移动到主要的异常处理程序中finally,这样无论出现什么问题都会发生.还要注意使用enumerate提供迭代器值及其索引; 这是Python中维护自己的索引指针的推荐方法.

我不确定它是否与您的代码相关,但我发现splinter了socket.gaierror异常urllib2.URLError,所以我展示了如何捕获它们.我在循环中移动了这个异常处理程序; 即使一个或多个URL不存在,这将继续抓取剩余的屏幕截图.