标签: urlopen

我正在尝试为我的项目打开一个URL,这是我的代码:

from urllib2 import urlopen
page = urlopen("https://docs.python.org/3/howto/urllib2.html")
contents = page.read()

这只是一个简单的演示代码,然而,当我运行代码时,我收到以下错误"ImportError:无法导入名称urlopen"

我尝试在CMD中输入"pip install urllib2"并得到以下错误"无法找到满足要求的版本urllib2 ...找不到urllib2的匹配分发"

我如何解决这个错误,因为我使用python 2.7.12而不是python3

动机

通过启发这个问题 -在OP使用urlopen()和意外通过的sys.argv列表,而不是一个字符串作为url.抛出此错误消息:

AttributeError:'list'对象没有属性'timeout'

由于urlopen编写的方式,错误消息本身和回溯不是非常有用,并且可能难以理解,特别是对于Python新手:

Traceback (most recent call last):
  File "test.py", line 15, in <module>
    get_category_links(sys.argv)
  File "test.py", line 10, in get_category_links
    response = urlopen(url)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 420, in open
    req.timeout = timeout
AttributeError: 'list' object has no attribute 'timeout'

问题

这是我正在使用的缩短代码:

try:
    from urllib.request import urlopen
except ImportError:
    from urllib2 import urlopen

import sys


def get_category_links(url):
    response = …

所以我是python的新手,我迫切需要帮助.

我有一个文件,其中有一堆id(整数值)写在'em.它是一个文本文件.

现在我需要将文件中的每个id传递给一个url.

例如"https://example.com/[id]"

它将以这种方式完成

A = json.load(urllib.urlopen("https://example.com/(the first id present in the text file)"))
print A

这将基本上做的是它将读取有关上述URL中存在的id的某些信息并显示它.我希望它以循环格式工作,在其中它将读取文本文件中的所有ID并将其传递给'A'中提到的URL并持续显示值.有没有办法做到这一点？

如果有人能帮助我,我将非常感激!

当我尝试从测试页面注销时,会出现连接拒绝错误.这是我用过的命令/代码:

driver.find_element_by_xpath("//a[@href = '/logged/pages/user/logout.php']").click()

我也尝试过linktext或ActionChains,同样的错误.它发生在某个时间,而不是总是,任何评论？

该错误消息:

<urlopen error [Errno 111] Connection refused>
Stacktrace

Traceback (most recent call last):
  File "/home/jenkins/run/workspace/QA_DEV_CINDY_LOGGED/LoggedSmoke/LoggedCommonFunction.py", line 158, in Logout
    link.click()
  File "/usr/local/lib/python3.3/site-packages/selenium/webdriver/remote/webelement.py", line 70, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/usr/local/lib/python3.3/site-packages/selenium/webdriver/remote/webelement.py", line 404, in _execute
    return self._parent.execute(command, params)
  File "/usr/local/lib/python3.3/site-packages/selenium/webdriver/remote/webdriver.py", line 193, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/usr/local/lib/python3.3/site-packages/selenium/webdriver/remote/remote_connection.py", line 358, in execute
    return self._request(command_info[0], url, body=data)
  File "/usr/local/lib/python3.3/site-packages/selenium/webdriver/remote/remote_connection.py", line 426, in _request
    resp = opener.open(request)
  File "/usr/local/lib/python3.3/urllib/request.py", line 469, in open
    response = self._open(req, …

我试图在Kubuntu 14.04上运行python中的selenium.我尝试使用chromedriver或geckodriver时出现此错误消息,两者都是相同的错误.

Traceback (most recent call last):
  File "vse.py", line 15, in <module>
    driver = webdriver.Chrome(chrome_options=options, executable_path=r'/root/Desktop/chromedriver')
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/chrome/webdriver.py", line 75, in __init__
    desired_capabilities=desired_capabilities)
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/webdriver.py", line 156, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/webdriver.py", line 251, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/webdriver.py", line 318, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/remote_connection.py", line 375, in execute
    return self._request(command_info[0], url, body=data)
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/remote_connection.py", line 397, in _request
    resp = self._conn.request(method, url, body=body, headers=headers)
  File "/usr/lib/python3/dist-packages/urllib3/request.py", line 79, in request …

对于以下代码

theurl = "https://%s:%s@members.dyndns.org/nic/update?hostname=%s&myip=%s&wildcard=NOCHG&mx=NOCHG&backmx=NOCHG" % (username, password, hostname, theip)

conn = urlopen(theurl) # send the request to the url
print(conn.read())  # read the response
conn.close()   # close the connection

我收到以下错误

File "c:\Python31\lib\http\client.py", line 667, in _set_hostport
    raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])

有任何想法吗？？？

我试图从http://flight.qunar.com/使用python3和BeautifulSoup刮取机票信息(包括飞机信息和价格信息等).下面是我正在使用的python代码.在这段代码中,我试图在2012-07-25从北京(北京)到丽江(丽江)的航班信息.

import urllib.parse
import urllib.request
from bs4 import BeautifulSoup
url = 'http://flight.qunar.com/site/oneway_list.htm'
values = {'searchDepartureAirport':'??', 'searchArrivalAirport':'??', 'searchDepartureTime':'2012-07-25'}
encoded_param = urllib.parse.urlencode(values)
full_url = url + '?' + encoded_param
response = urllib.request.urlopen(full_url)
soup = BeautifulSoup(response)
print(soup.prettify())

我得到的是提交请求后的初始页面,页面仍在加载搜索结果.我想要的是完成加载搜索结果后的最后一页.那么如何使用python实现这一目标呢？

我遇到了请求包的奇怪行为……或因此，我相信：

（1）对于某个网站，requests.get在Windows上运行，但在Linux上冻结（如果将超时设置为几秒钟，则会引发超时错误）

（2）urllib.request.urlopen在Windows和Linux上均可使用

是否有人建议如何进一步调试/调查？

详细信息如下：

在Windows计算机上，我在Jupyter Notebook中运行：

import sys
import requests
print(sys.version)
print(requests.__version__)

3.6.3 | Anaconda自定义（32位）| （默认值，2017年10月15日，07：29：16）[MSC v.1900 32位（Intel）]

2.18.4

然后我尝试：

time1 = time.time()
response=requests.get('https://www.tagheuer.com/en/watches/tag-heuer-carrera-calibre-5-automatic-watch-39-mm-war211a-ba0782')
time2 = time.time()
print(response.status_code)
print(time2-time1)

结果：

200

0.6808576583862305

到目前为止，一切顺利……然后我移至Unix服务器：

import sys
import requests
print(sys.version)
print(requests.__version__)

3.5.2（默认值，2017年11月23日，16：37：01）\ n [GCC 5.4.0 20160609]

2.18.4

当我发出请求声明时，

response=requests.get('https://www.tagheuer.com/en/watches/tag-heuer-carrera-calibre-5-automatic-watch-39-mm-war211a-ba0782')

回来需要很长时间。有时返回的字节看起来根本不像请求的页面。有时我会忍耐不住，按CTRL-C。

但是在同一台服务器上：

 from urllib.request import urlopen
 import time
 url='https://www.tagheuer.com/en/watches/tag-heuer-carrera-calibre-5-automatic-watch-39-mm-war211a-ba0782'
 time1 = time.time()
 f = urlopen(url)
 myfile = f.read()
 time2 = time.time()
 print(f.status)
 print(time2-time1)

完成结果

200

0.22718000411987305

为什么请求库会让我失望的是Linux机器而不是Windows机器？

为什么urllib.request.urlopen可以工作，但请求会冻结？他们在做什么不同？

两种平台上的www.google.com都可以使用相同的代码。TAG_Heuer网站的哪种（非标准？）行为会导致请求失败？

期待您的反馈和想法！

我正在尝试使用devserver将数据从CSV上传到我的应用:

appcfg.py upload_data --config_file="DataLoader.py" --filename="data.csv" --kind=Foo --url=http://localhost:8083/remote_api "path/to/app"

结果:

Application: appname; version: 1.
Uploading data records.
[INFO    ] Logging to bulkloader-log-20100626.181045
[INFO    ] Throttling transfers:
[INFO    ] Bandwidth: 250000 bytes/second
[INFO    ] HTTP connections: 8/second
[INFO    ] Entities inserted/fetched/modified: 20/second
[INFO    ] Batch Size: 10
[INFO    ] Opening database: bulkloader-progress-20100626.181045.sql3
Please enter login credentials for localhost
Email: email@domain.com
Password for email@domain.com:
[INFO    ] Connecting to localhost:8083/remote_api
[INFO    ] Starting import; maximum 10 entities per post
Google\google_appengine\google\appengine\api\datastore_types.py:673: DeprecationWarning: object.__init__() takes no …

在我的应用程序中,我使用urllib2.urlopen()函数调用api并从该api获取结果.但这不能正常工作.有时它会显示结果但有时会出现以下错误:

Traceback (most recent call last):
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 700, in __call__
handler.get(*groups)
  File "/base/data/home/apps/s~malware-app/7.351334968546050391/main.py", line 505, in get
f = urllib2.urlopen('http://whoapi.com/api-v1/?domain=%s&rtype=alexarank&apikey=xyz'% domain)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 124, in urlopen
    return _opener.open(url, data)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 381, in open
response = self._open(req, data)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 399, in _open
  '_open', req)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 360, in _call_chain
result = func(*args)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 1114, in http_open
return self.do_open(httplib.HTTPConnection, req)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 1087, in do_open
r = h.getresponse()
  File "/base/python_runtime/python_dist/lib/python2.5/httplib.py", line …