通过在 URL 末尾添加 (?__a=1) 抓取 Instagram 媒体时出错

Question

有时，当尝试抓取 Instagram 媒体时，可以在 URL 末尾添加 (?__a=1)

例如：https: //www.instagram.com/p/CP-Kws6FoRS/ ?__a=1

返回的响应

{
    "__ar": 1,
    "error": 1357004,
    "errorSummary": "Sorry, something went wrong",
    "errorDescription": "Please try closing and re-opening your browser window.",
    "payload": null,
    "hsrp": {
        "hblp": {
            "consistency": {
                "rev": 1005622141
            }
        }
    },
    "lid": "7104767527440109183"
}

为什么会返回此响应？我应该如何解决此问题？另外，我们还有其他方法来获取视频和照片 URL 吗？

Answer 1

&__d=dis我通过在 URL 末尾添加查询字符串解决了这个问题，如下所示：https://www.instagram.com/p/CFr6G-whXxp/?__a=1&__d=dis

Answer 2

我相信我可以使用以下方法找到解决方法：

• https://i.instagram.com/api/v1/users/web_profile_info/?username={username}获取用户的信息和最近的帖子。from 的响应与fromdata.user相同。graphql.userhttps://i.instagram.com/{username}/?__a=1
• <meta property="al:ios:url" content="instagram://media?id={media_id}">从的 HTML 响应中提取媒体 ID https://instagram.com/p/{post_shortcode}。
• https://i.instagram.com/api/v1/media/{media_id}/info使用提取的媒体 ID 获得与 相同的响应https://instagram.com/p/{post_shortcode}/?__a=1。

有几点很重要：

• user-agent脚本中使用的内容很重要。我发现在开发工具中重新发送请求时生成的一个 Firefox 返回了错误"Sorry, something went wrong"。
• 此解决方案使用您的 Firefox 配置文件中的 cookie。运行此脚本之前，您需要在 Firefox 中登录 Instagram。如果您愿意，可以将 Firefox 切换到 Chrome。

cookiejar = browser_cookie3.chrome(domain_name='instagram.com')

这是完整的脚本。让我知道这是否有帮助！

import os
import pathlib
import string
from datetime import datetime, timedelta
from urllib.parse import urlparse
import bs4 as bs
import browser_cookie3
from google.auth.transport import requests
import requests

# setup.
username = "<username>"
output_path = "C:\\some\\path"
headers = {
    "User-Agent": "Mozilla/5.0 (Linux; Android 9; GM1903 Build/PKQ1.190110.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/75.0.3770.143 Mobile Safari/537.36 Instagram 103.1.0.15.119 Android (28/9; 420dpi; 1080x2260; OnePlus; GM1903; OnePlus7; qcom; sv_SE; 164094539)"
}


def download_post_media(post: dict, media_list: list, number: int):
    output_filename = f"{output_path}/{username}"
    if not os.path.isdir(output_filename):
        os.mkdir(output_filename)
    post_time = datetime.fromtimestamp(int(post["taken_at_timestamp"])) + timedelta(hours=5)
    output_filename += f"/{username}_{post_time.strftime('%Y%m%d%H%M%S')}_{post['shortcode']}_{number}"
    current_media_json = media_list[number - 1]
    if current_media_json['media_type'] == 1:
        media_type = "image"
        media_ext = ".jpg"
        media_url = current_media_json["image_versions2"]['candidates'][0]['url']
    elif current_media_json['media_type'] == 2:
        media_type = "video"
        media_ext = ".mp4"
        media_url = current_media_json["video_versions"][0]['url']
    output_filename += media_ext
    response = send_request_get_response(media_url)
    with open(output_filename, 'wb') as f:
        f.write(response.content)


def send_request_get_response(url):
    cookiejar = browser_cookie3.firefox(domain_name='instagram.com')
    return requests.get(url, cookies=cookiejar, headers=headers)


# use the /api/v1/users/web_profile_info/ api to get the user's information and its most recent posts.
profile_api_url = f"https://i.instagram.com/api/v1/users/web_profile_info/?username={username}"
profile_api_response = send_request_get_response(profile_api_url)
# data.user is the same as graphql.user from ?__a=1.
timeline_json = profile_api_response.json()["data"]["user"]["edge_owner_to_timeline_media"]
for post in timeline_json["edges"]:
    # get the HTML page of the post.
    post_response = send_request_get_response(f"https://instagram.com/p/{post['node']['shortcode']}")
    html = bs.BeautifulSoup(post_response.text, 'html.parser')
    # find the meta tag containing the link to the post's media.
    meta = html.find(attrs={"property": "al:ios:url"})
    media_id = meta.attrs['content'].replace("instagram://media?id=", "")
    # use the media id to get the same response as ?__a=1 for the post.
    media_api_url = f"https://i.instagram.com/api/v1/media/{media_id}/info"
    media_api_response = send_request_get_response(media_api_url)
    media_json = media_api_response.json()["items"][0]
    media = list()
    if 'carousel_media_count' in media_json:
        # multiple media post.
        for m in media_json['carousel_media']:
            media.append(m)
    else:
        # single media post.
        media.append(media_json)
    media_number = 0
    for m in media:
        media_number += 1
        download_post_media(post['node'], media, media_number)

Answer 3

ig修改了方法，使用了新方法：

GET https://i.instagram.com/api/v1/tags/web_info/?tag_name=${tags}

下一页：

POST https://i.instagram.com/api/v1/tags/${tags}/sections/
body: 
{
include_persistent: 0
max_id: ${The last request contained this field}
next_media_ids[]: ${The last request contained this field}
next_media_ids[]: ${The last request contained this field}
page: ${The last request contained this field}
surface: grid
tab: recent
}