标签: python-requests

我正在尝试向应该返回 json 的 api 端点发送 post 请求。但是我得到了奇怪的字符。我尝试用 utf-8 编码但没有成功。

import requests

url1= "https://cliff.mediacloud.org/process"

header ={"accept": "application/json, text/javascript, */*; q=0.01",
"accept-encoding": "gzip, deflate, br",
"accept-language": "en-US,en;q=0.9,fr-DZ;q=0.8,fr;q=0.7",
"content-length": "56",
"content-type": "application/x-www-form-urlencoded; charset=UTF-8",
"cookie": "__cfduid=d90785d2f72bf656c19e794239e26185d1585929126; _ga=GA1.2.1927368988.1585929126; _gid=GA1.2.61609766.1585929126 ; _gat_gtag_UA_60744513_13=1",
"origin": "https://cliff.mediacloud.org",
"referer": "https://cliff.mediacloud.org/",
"sec-fetch-dest": "empty",
"sec-fetch-mode": "cors",
"sec-fetch-site": "same-origin",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36",
"x-requested-with": "XMLHttpRequest"}

text1 = "i am traveling to france"

data1={"text": text1,
"demonyms": "false",
"language": "EN"}

r = requests.post(url=url1, headers=header, data=data1 ) …

我是python的新用户。我不知道为什么，但请求总是抛出 InvalidURL 异常：

>>> import requests
>>> r = requests.get('https://www.google.es/')

输出：

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/requests/models.py", line 380, in prepare_url
    scheme, auth, host, port, path, query, fragment = parse_url(url)
  File "/usr/lib/python3/dist-packages/urllib3/util/url.py", line 392, in parse_url
    return six.raise_from(LocationParseError(source_url), None)
  File "<string>", line 3, in raise_from
urllib3.exceptions.LocationParseError: Failed to parse: https://www.google.es/

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/dist-packages/requests/api.py", line 76, in get
    return request('get', url, …

我正在尝试获取 id 的值，即“id”：59，它以 json 的形式出现在 curl 输出中。下面是 json 中的 curl 输出：

[{"id":59,"description":"This is a demo project","name":"Demo_Project","name_with_namespace":"sam / Demo_Project","path":"demo_project","path_with_namespace":"sam/demo_project","created_at":"2020-03-02T08:43:13.664Z","default_branch":"master","tag_list":[],"ssh_url_to_repo":"ssh://git@od-test.od.com:2222/sam/demo_project.git","http_url_to_repo":"https://od-test.od.com/gitlab/sam/demo_project.git","web_url":"https://od-test.od.com/gitlab/sam/demo_project","readme_url":"https://od-test.od.com/gitlab/sam/demo_project/blob/master/README.md","avatar_url":null,"star_count":0,"forks_count":0,"last_activity_at":"2020-04-09T09:28:09.860Z","namespace":{"id":2259,"name":"sam","path":"sam","kind":"user","full_path":"sam","parent_id":null,"avatar_url":"https://secure.gravatar.com/avatar/755db8ssqaq50dcc9d189c53523b?s=80\u0026d=identicon","web_url":"https://od-test.od.com/gitlab/sam"}}]

我正在使用python来解析json并获取id的值。我已尝试使用以下命令执行相同操作，但出现错误。

curl --header "PRIVATE-TOKEN: 9999ayayayar66" "https://od-test.od.com/gitlab/api/v4/search?scope=projects&search=demo_project" | python -c 'import sys, json; print(json.load(sys.stdin)["id"])'

错误：

谁能帮我正确的python命令来获取id的值。提前致谢。

我目前正在尝试使用 Python Requests 下载视频，我想先找出它的大小。

import requests

print("STARTING PROGRAM...")

req = requests.get("https://www.source.com/source.mp4")

我想到的唯一方法如下：

for chunk in req.iter_content():
  count+=1
print("FOUND %d CHUNKS" %(count))

但是自从我下载 24 分钟的 mp4 以来，这花了很长时间。有一个更好的方法吗？

你好，我正在尝试用 python 请求做一个项目。但是当我尝试登录 instagram 时，我变成了这样：

** 绑定方法Response.json of Response [400] **

我的代码：

import os
import requests
import getpass
import json
import io
import time

X_SECOND = 2
BASE_URL = "https://www.instagram.com/"
LOGIN_URL = BASE_URL + "accounts/login/ajax/"
USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; ) Gecko/20100101 Firefox/65.0"
CHANGE_URL = "https://www.instagram.com/accounts/web_change_profile_picture/"
CHNAGE_DATA = {"Content-Disposition": "form-data", "name": "profile_pic",
               "filename": "profilepic.jpg", "Content-Type": "image/jpeg"}
headers = {
    "Host": "www.instagram.com",
    "Accept": "*/*",
    "Accept-Language": "en-US,en;q=0.5",
    "Accept-Encoding": "gzip, deflate, br",
    "Referer": "https://www.instagram.com/accounts/edit/",
    "X-IG-App-ID": "936619743392459",
    "X-Requested-With": "XMLHttpRequest",
    "DNT": "1",
    "Connection": "keep-alive", …

使用 Pandas 读取 html 文件的数据时......这是我的代码：

import pandas as pd 
import geopandas as gpd
import requests

url=requests.get("https://www.worldometers.info/coronavirus/")
dataframe=pd.read_html(url.text)
print(dataframe)    

我得到一个 ValueError 显示没有找到匹配模式 '.+' 的表，这是错误：

 C:/Users/mayank/AppData/Local/Programs/Python/Python38-32/python.exe e:/skills/mayankvscod
e/projects/coronavirus_worldometer/corona_meter.py
Traceback (most recent call last):
  File "e:/skills/mayankvscode/projects/coronavirus_worldometer/corona_meter.py", line 6, in
<module>
    dataframe=pd.read_html(url.text)
  File "C:\Users\mayank\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pandas\io\html.py", line 1085, in read_html
    return _parse(
  File "C:\Users\mayank\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pandas\io\html.py", line 915, in _parse
    raise retained
  File "C:\Users\mayank\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pandas\io\html.py", line 895, in _parse
    tables = p.parse_tables()
  File "C:\Users\mayank\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pandas\io\html.py", line 213, in parse_tables
    tables = self._parse_tables(self._build_doc(), self.match, self.attrs)
  File "C:\Users\mayank\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pandas\io\html.py", line 561, in _parse_tables …

目前，我使用 Selenium 从网站上的表格中提取文本。以下是代码：

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager


# Using Chrome to access web
browser = webdriver.Chrome(ChromeDriverManager().install())

# Open the website
browser.get('https://launchstudio.bluetooth.com/Listings/Search')
element = browser.find_element_by_id('searchButton').click()

table_text = browser.find_element_by_class_name('table').text

while len(table_text) < 80:
    table_text = browser.find_element_by_class_name('table').text

print(table_text)

browser.close()

但是，我正在尝试找到一种方法对请求/美丽汤或任何其他库执行相同的操作，我可以将其安排为 Windows 中的任务，并每隔 x 间隔将结果存储在表中。显然，因为我希望所有这些都在后台发生，然后触发通知等。

我想要的是 - 打开这个网站，点击搜索按钮（或触发相应的 javascript），然后将表格导出为 Dataframe 或其他任何内容。

你能在这里指导我吗？

提前致谢！！

大家好，我正在抓取亚马逊网站，我正在获取所有 16 个链接，但想从新生成的列表中删除 {} 部分。提供的输出

from requests_html import HTMLSession

import time

import pandas as pd


s = HTMLSession()


r = s.get("https://www.amazon.in/s?k=oneplus&page=1")

r.html.render(sleep=1)

t= []


Everything = r.html.find("div.s-include-content-margin.s-border-bottom.s-latency-cf-section")


for e in Everything:

links = e.find("a.a-link-normal.a-text-normal")[0].absolute_links

t.append(links)

print("\n",t)

我正在尝试使用 FastAPI 为神经网络提供服务。

from fastapi import Depends, FastAPI
from pydantic import BaseModel
from typing import Dict

class iRequest(BaseModel):
    arg1: str
    arg2: str

class iResponse(BaseModel):
    pred: str
    probs: Dict[str, float]

@app.post("/predict", response_model=iResponse)
def predict(request: iRequest, model: Model = Depends(get_model)):
    pred, probs = model.predict(request.arg1, request.arg2)
    return iResponse(pred = pred, probs = probs)

手动站点 http://localhost:8000/docs#/default/predict_predict_post 工作正常并转换为以下 curl 命令：

curl -X POST "http://localhost:8000/predict" -H  "accept: application/json" -H  "Content-Type: application/json" -d "{\"arg1\":\"I am the King\",\"arg2\":\"You are not my King\"}"

这也有效。当我尝试使用 python 请求查询 API 时：

import …

在这里考虑这个网站：https : //dlnr.hawaii.gov/dsp/parks/oahu/ahupuaa-o-kahana-state-park/

我正在寻找右侧标题下的内容。这是我的示例代码，它应该返回内容列表但返回空字符串：

import requests as req
from bs4 import BeautifulSoup as bs

r = req.get('https://dlnr.hawaii.gov/dsp/parks/oahu/ahupuaa-o-kahana-state-park/').text
soup = bs(r)

par = soup.find('h3', text= 'Facilities')

for sib in par.next_siblings:
    print(sib)

这将返回：

<ul class="park_icon">
<div class="clearfix"></div>
</ul>

该网站不显示该类的任何 div 元素。此外，未捕获列表项。