小编Mat*_*eus的帖子

How to crawl a whole website with Headless Chrome Crawler?

i've been studying chrome puppeteer to develop a crawler for learning purposes. So i discovered HeadLess Chrome Crawler, a good node package. However, i found some troubles tryng crawl a entire website using this awesome package. I not found in docs where i can do this. I want to get all links from a page and pass them into an array list to crawl them. This is my code now:

const HCCrawler = require('headless-chrome-crawler');

(async() => {
  var urlsToVisit = …
Run Code Online (Sandbox Code Playgroud)

web-crawler node.js async-await google-chrome-headless puppeteer

1
推荐指数
1
解决办法
4038
查看次数