use*_*529 0 javascript node.js web-scraping puppeteer
我在 Try Puppeteer 中使用此代码:
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://www.barchart.com/futures/quotes/ESM19/interactive-chart/fullscreen');
const linkHandlers = await page.$x("//li[contains(text(), '1D')]");
if (linkHandlers.length > 0) {
await linkHandlers[0].click();
} else {
throw new Error("Link not found");
}
await page.$eval('input[name="fieldInput"]', el => el.value = '1');
console.log(await page.content())
// const text = page.evaluate(() => document.querySelector('rect'))
// text.then((r) => {console.log(r[0])})
await page.screenshot({path: 'screenshot.png'});
await browser.close();
Run Code Online (Sandbox Code Playgroud)
Chrome 浏览器中加载的同一页面显示了指示价格变动的条形图,但在 Puppeteer 中获得的屏幕截图中,图表是空的。
还page.content()给出了一个与我在 Chrome 中检查元素时看到的完全不同的 html。
You are not waiting for the request to resolve when the input is changed. As a change will trigger a request, you should use page.waitForResponse to wait until the data is loaded.
In addition, this is an Angular application, which does not seem to like it if you simply change the value of the field via el.value = '1'. Instead you need to try to behave more like a human (and hit backspace and type the input value).
First, you get the element handle (input[name="fieldInput") from the document. Then, you focus the element, remove the value inside by pressing backspace. After that you type the desired input value.
The input field now has the correct value, now we need to trigger the blur event by calling blur() on the element. In parallel, we wait for the request to the server to finish. After the request finishes, we should give the page a few milliseconds to render the data.
All together, the resulting code looks like this:
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://www.barchart.com/futures/quotes/ESM19/interactive-chart/fullscreen');
// wait until the element appears
const linkHandler = await page.waitForXPath("//li[contains(text(), '1D')]");
await linkHandler.click();
// get the input field, focus it, remove what's inside, then type the value
const elementHandle = await page.$('input[name="fieldInput"]');
await elementHandle.focus();
await elementHandle.press('Backspace');
await elementHandle.type('1');
// trigger the blur event and wait for the response from the server
await Promise.all([
page.waitForResponse(response => response.url().includes('https://www.barchart.com/proxies/timeseries/queryminutes.ashx')),
page.evaluate(el => el.blur(), elementHandle)
]);
// give the page a few milliseconds to render the diagram
await page.waitFor(100);
await page.screenshot({path: 'screenshot.png'});
await browser.close();
Run Code Online (Sandbox Code Playgroud)
我还删除了该page.$x功能并用该page.waitForXPath功能替换了它。这可确保您的脚本一直等到页面加载完毕并且您要单击的元素可用后才能继续执行脚本。
| 归档时间: |
|
| 查看次数: |
1683 次 |
| 最近记录: |