jlo*_*jlo 8 javascript json web-scraping
我想从这个站点输出一些值。使用浏览器检查器,我找到了包含我想要的信息的 JSON的URL。使用 JSON 后,我可以很好地提取值。
问题是 JSON 的 URL 仅在有限的时间内有效。如果我稍后尝试访问它(通过浏览器或通过脚本的 UrlFetch),我会得到:
{
"status": 401,
"response": "unauthorized"
}
Run Code Online (Sandbox Code Playgroud)
由于我想定期检查目标 JSON,我想找到一种方法来授权请求。有任何想法吗?
谢谢!
PS:这是我迄今为止尝试过的(谷歌应用程序脚本):
function getUserAndJSON(){
var url = 'https://asunnot.oikotie.fi/api/cards?cardType=100&conditionType%5B%5D=1&conditionType%5B%5D=2&limit=24&locations=%5B%5B1669,4,%22Lauttasaari,%20Helsinki%22%5D,%5B14714,5,%2200340,%20Helsinki%22%5D%5D&lotOwnershipType%5B%5D=1&offset=0&price%5Bmax%5D=600000&price%5Bmin%5D=150000&roomCount%5B%5D=3&size%5Bmin%5D=35&sortBy=published_sort_desc';
var opt = {
"method": "GET",
"muteHttpExceptions": true,
"escaping":false,
"headers":JSON.parse(user)
}
var str = UrlFetchApp.fetch(url,opt).getContentText();
Logger.log(str); //error 401
}
Run Code Online (Sandbox Code Playgroud)
解决此问题的一种方法是在浏览器检查器(在我的情况下为 Opera)中简单地使用选项“复制为...> 获取”,它给出以下内容(与浏览器发送的标题略有不同):
fetch("https://asunnot.oikotie.fi/api/cards?cardType=100&conditionType%5B%5D=1&conditionType%5B%5D=2&limit=24&locations=%5B%5B1669,4,%22Lauttasaari,+Helsinki%22%5D,%5B14714,5,%2200340,+Helsinki%22%5D%5D&lotOwnershipType%5B%5D=1&offset=0&price%5Bmax%5D=600000&price%5Bmin%5D=150000&roomCount%5B%5D=3&size%5Bmin%5D=35&sortBy=published_sort_desc", {"credentials":"omit","headers":{"accept":"application/json","ota-cuid":"fd2a3a03d52a2721f9a9aa844ddf7eef2ac66ed6","ota-loaded":"1586685082","ota-token":"ab7e9f830a7dff3a9b01fbdcbc899ed7bfa659a4793103f1943e83ef5f938b16","sec-fetch-dest":"empty"},"referrer":"https://asunnot.oikotie.fi/myytavat-asunnot?conditionType%5B%5D=1&conditionType%5B%5D=2&locations=%5B%5B1669,4,%22Lauttasaari,%20Helsinki%22%5D,%5B14714,5,%2200340,%20Helsinki%22%5D%5D&lotOwnershipType%5B%5D=1&price%5Bmax%5D=600000&price%5Bmin%5D=150000&size%5Bmin%5D=35&roomCount%5B%5D=3&cardType=100","referrerPolicy":"no-referrer-when-downgrade","body":null,"method":"GET","mode":"cors"});
Run Code Online (Sandbox Code Playgroud)
我在 GAS 中使用的方式如下:
function testGetJSON(){
var str = UrlFetchApp.fetch("https://asunnot.oikotie.fi/api/cards?cardType=100&conditionType%5B%5D=1&conditionType%5B%5D=2&limit=24&locations=%5B%5B1669,4,%22Lauttasaari,+Helsinki%22%5D,%5B14714,5,%2200340,+Helsinki%22%5D%5D&lotOwnershipType%5B%5D=1&offset=0&price%5Bmax%5D=600000&price%5Bmin%5D=150000&roomCount%5B%5D=3&size%5Bmin%5D=35&sortBy=published_sort_desc",
{"credentials":"omit",
"headers":{"accept":"application/json","ota-cuid":"fd2a3a03d52a2721f9a9aa844ddf7eef2ac66ed6","ota-loaded":"1586685082","ota-token":"ab7e9f830a7dff3a9b01fbdcbc899ed7bfa659a4793103f1943e83ef5f938b16","sec-fetch-dest":"empty"},"referrer":"https://asunnot.oikotie.fi/myytavat-asunnot?conditionType%5B%5D=1&conditionType%5B%5D=2&locations=%5B%5B1669,4,%22Lauttasaari,%20Helsinki%22%5D,%5B14714,5,%2200340,%20Helsinki%22%5D%5D&lotOwnershipType%5B%5D=1&price%5Bmax%5D=600000&price%5Bmin%5D=150000&size%5Bmin%5D=35&roomCount%5B%5D=3&cardType=100","referrerPolicy":"no-referrer-when-downgrade","body":null,"method":"GET","mode":"cors"});
Logger.log(str.getContentText());
}
Run Code Online (Sandbox Code Playgroud)
哪个工作得很好。
谢谢大家的提示和建议!
| 归档时间: |
|
| 查看次数: |
2045 次 |
| 最近记录: |