我想通过 9gag.com Api 下载帖子,该 API 显然存在,但没有在任何地方记录。如果您在浏览器中检查此链接,您将收到 json 响应:https://9gag.com/v1/tag-posts/tag/wtf/type/fresh
所以使用httr2这个应该很容易。然而:
library(httr2)
req <- request("https://9gag.com/v1/tag-posts/tag/samantha/type/fresh")
req_dry_run(req)
#> GET /v1/tag-posts/tag/wtf/type/fresh HTTP/1.1
#> Host: 9gag.com
#> User-Agent: httr2/0.1.1 r-curl/4.3.2 libcurl/7.82.0
#> Accept: */*
#> Accept-Encoding: deflate, gzip, br, zstd
result <- req_perform(req)
#> Error: HTTP 403 Forbidden.
Run Code Online (Sandbox Code Playgroud)
我检查了 Firefox Inspect 中的网络选项卡,并复制了输入 URL 时发送的 GET 请求:
curl 'https://9gag.com/v1/tag-posts/tag/wtf/type/fresh' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:98.0) Gecko/20100101 Firefox/98.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate, br' -H 'Connection: keep-alive' -H …Run Code Online (Sandbox Code Playgroud)