我想对通过在网页中向下滚动生成的内容进行逆向工程.问题出在网址上https://www.crowdfunder.com/user/following_page/80159?user_id=80159&limit=0&per_page=20&screwrand=933.screwrand似乎没有遵循任何模式,因此撤销网址不起作用.我正在考虑使用Splash进行自动渲染.如何使用Splash滚动浏览器？非常感谢!以下是两个请求的代码:
request1 = scrapy_splash.SplashRequest('https://www.crowdfunder.com/user/following/{}'.format(user_id),
self.parse_follow_relationship,
args={'wait':2},
meta={'user_id':user_id, 'action':'following'},
endpoint='http://192.168.99.100:8050/render.html')
yield request1
request2 = scrapy_splash.SplashRequest('https://www.crowdfunder.com/user/following_user/80159?user_id=80159&limit=0&per_page=20&screwrand=76',
self.parse_tmp,
meta={'user_id':user_id, 'action':'following'},
endpoint='http://192.168.99.100:8050/render.html')
yield request2
