我正在尝试打开并解析一个html页面.在python 2.7.8中我没有问题:
import urllib
url = "https://ipdb.at/ip/66.196.116.112"
html = urllib.urlopen(url).read()
一切都很好.但是我想转移到python 3.4并在那里得到HTTP错误403(禁止).我的代码:
import urllib.request
html = urllib.request.urlopen(url) # same URL as before
File "C:\Python34\lib\urllib\request.py", line 153, in urlopen
return opener.open(url, data, timeout)
File "C:\Python34\lib\urllib\request.py", line 461, in open
response = meth(req, response)
File "C:\Python34\lib\urllib\request.py", line 574, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Python34\lib\urllib\request.py", line 499, in error
return self._call_chain(*args)
File "C:\Python34\lib\urllib\request.py", line 433, in _call_chain
result = func(*args)
File "C:\Python34\lib\urllib\request.py", line 582, in http_error_default
raise HTTPError(req.full_url, code, …import requests
import webbrowser
from bs4 import BeautifulSoup
url = 'https://www.gamefaqs.com'
#headers={'User-Agent': 'Mozilla/5.0'}
headers ={'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36'}
response = requests.get(url, headers)
response.status_code 返回403。我可以使用firefox/chrome浏览网站，所以这似乎是一个编码错误。
我无法弄清楚我犯了什么错误。
谢谢你。
我是scrapy的新手,我让scrapy项目废弃了数据.
我正在尝试从网站上搜索数据,但我收到了错误日志
2016-08-29 14:07:57 [scrapy] INFO: Enabled item pipelines:
[]
2016-08-29 13:55:03 [scrapy] INFO: Spider opened
2016-08-29 13:55:03 [scrapy] INFO: Crawled 0 pages (at 0 pages/min),scraped 0 items (at 0 items/min)
2016-08-29 13:55:04 [scrapy] DEBUG: Crawled (403) <GET http://www.justdial.com/robots.txt> (referer: None)
2016-08-29 13:55:04 [scrapy] DEBUG: Crawled (403) <GET http://www.justdial.com/Mumbai/small-business> (referer: None)
2016-08-29 13:55:04 [scrapy] DEBUG: Ignoring response <403 http://www.justdial.com/Mumbai/small-business>: HTTP status code is not handled or not allowed
2016-08-29 13:55:04 [scrapy] INFO: Closing spider (finished)
我正在尝试按照命令然后在网站控制台然后我得到了响应,但当我在python脚本中使用相同的路径时,我得到了我上面描述的错误.
Web控制台上的命令:
$x('//div[@class="col-sm-5 col-xs-8 store-details …嗨，不是每次，但有时当我试图访问 LSE 代码时，我会被抛出每个烦人的 HTTP 错误 403：禁止消息。
任何人都知道我如何仅使用标准 python 模块来解决这个问题（遗憾的是没有漂亮的汤）。
import urllib.request
url = "http://www.londonstockexchange.com/exchange/prices-and-markets/stocks/indices/ftse-indices.html"
infile = urllib.request.urlopen(url) # Open the URL
data = infile.read().decode('ISO-8859-1') # Read the content as string decoded with ISO-8859-1
print(data) # Print the data to the screen
但是，时不时地这是我显示的错误：
Traceback (most recent call last):
File "/home/ubuntu/workspace/programming_practice/Assessment/Summative/removingThe403Error.py", line 5, in <module>
webpage = urlopen(req).read().decode('ISO-8859-1')
File "/usr/lib/python3.4/urllib/request.py", line 161, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.4/urllib/request.py", line 469, in open
response = meth(req, response)
File "/usr/lib/python3.4/urllib/request.py", line 579, …我试图自动下载出现在谷歌图像搜索中的第一张图像，但我无法读取网站源代码并且发生错误（“HTTP 错误 403：禁止”）。有任何想法吗？感谢您的帮助！
这是我的代码：
from urllib.request import urlopen
from bs4 import BeautifulSoup
word = 'house'
r = urlopen('https://www.google.pl/search?&dcr=0&tbm=isch&q='+word)
data = r.read()
我正在尝试阅读网站，但不幸的是出了点问题。
import bs4 as bs
import urllib.request
sauce = urllib.request.urlopen('https://csgoempire.com/withdraw').read()
soup = bs.BeautifulSoup(sauce,'lxml')
print(soup.find_all('p'))
错误：
Traceback (most recent call last):
File "F:/Informatika/Python3X/GamblinSitesBot/GamblingSitesBot.py", line 4, in <module>
sauce = urllib.request.urlopen('https://csgoempire.com/').read()
File "c:\users\edgaras\appdata\local\programs\python\python36\Lib\urllib\request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "c:\users\edgaras\appdata\local\programs\python\python36\Lib\urllib\request.py", line 532, in open
response = meth(req, response)
File "c:\users\edgaras\appdata\local\programs\python\python36\Lib\urllib\request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "c:\users\edgaras\appdata\local\programs\python\python36\Lib\urllib\request.py", line 570, in error
return self._call_chain(*args)
File "c:\users\edgaras\appdata\local\programs\python\python36\Lib\urllib\request.py", line 504, in _call_chain
result = func(*args)
File "c:\users\edgaras\appdata\local\programs\python\python36\Lib\urllib\request.py", …我在运行代码时遇到问题，并在 StackOverflow 上找到了完美的解决方案。但是，当我进行必要的更改并运行它时，我没有得到任何输出。
代码：
from bs4 import BeautifulSoup
import urllib.parse
import requests
r = requests.get('https://duckduckgo.com/html/?q=test')
soup = BeautifulSoup(r.text, 'html.parser')
results = soup.find_all('a', attrs={'class':'result__url'}, href=True)
for link in results:
url = link['href']
o = urllib.parse.urlparse(url)
d = urllib.parse.parse_qs(o.query)
print(d['uddg'][0])
urlparse() 用于路径组件“从中获取查询字符串并将其传递给parse_qs()进一步处理它。然后您可以使用名称提取链接uddg。” 这应该是前几个结果：
http://www.speedtest.net/
https://www.merriam-webster.com/dictionary/test
https://en.wikipedia.org/wiki/Test
https://www.thefreedictionary.com/test
https://www.dictionary.com/browse/test
我没有得到任何输出。输出：
In [14]: