在python中使用scrapy执行Javascript函数

Question

我对“ scrapy ” 非常陌生，我正在剪贴一个网站，因为我有一些锚标记，其中包含带有Java脚本SubmitForm函数的href属性。当我单击该javascript函数时，正在打开一个页面，我需要从中获取数据。我使用Xpath并找到了特定锚标记的href，但无法执行包含javascript函数的href属性。谁能告诉我如何在scrapy python中执行javascript锚定标记的Submit函数。我的HTML代码是

   <table class="Tbl" cellspacing="2" cellpadding="0" border="0">
     <tbody>
        <tr>
           <td class="TblOddRow">
             <table cellspacing="0" cellpadding="0" border="0">
               <tbody>
                 <tr>
                   <td valign="middle" nowrap="">
                        <a class="Page" alt="Click to view job description" title="Click to view job description" href="javascript:sysSubmitForm('frmSR1');">Accountant&nbsp;</a>
                   </td>
                 </tr>
               </tbody>
             </table>
           </td>
        </tr>
      </tbody>
  </table>                      

蜘蛛代码是

class MountSinaiSpider(BaseSpider):
   name = "mountsinai"
   allowed_domains = ["mountsinaicss.igreentree.com"]
   start_urls = [
       "https://mountsinaicss.igreentree.com/css_external/CSSPage_SearchAndBrowseJobs.ASP?T=20120517011617&",
   ]
   def parse(self, response):
       return [FormRequest.from_response(response,
                                        formdata={ "Type":"CSS","SRCH":"Search Jobs","InitURL":"CSSPage_SearchAndBrowseJobs.ASP","RetColsQS":"Requisition.Key¤Requisition.JobTitle¤Requisition.fk_Code_Full_Part¤[Requisition.fk_Code_Full_Part]OLD.Description(sysfk_Code_Full_PartDesc)¤Requisition.fk_Code_Location¤[Requisition.fk_Code_Location]OLD.Description(sysfk_Code_LocationDesc)¤Requisition.fk_Code_Dept¤[Requisition.fk_Code_Dept]OLD.Description(sysfk_Code_DeptDesc)¤Requisition.Req¤","RetColsGR":"Requisition.Key¤Requisition.JobTitle¤Requisition.fk_Code_Full_Part¤[Requisition.fk_Code_Full_Part]OLD.Description(sysfk_Code_Full_PartDesc)¤Requisition.fk_Code_Location¤[Requisition.fk_Code_Location]OLD.Description(sysfk_Code_LocationDesc)¤Requisition.fk_Code_Dept¤[Requisition.fk_Code_Dept]OLD.Description(sysfk_Code_DeptDesc)¤Requisition.Req¤","ResultSort":"" },
                                        callback=self.parse_main_list)]
   def parse_main_list(self, response):
       hxs = HtmlXPathSelector(response)
       firstpage_urls = hxs.select("//table[@class='Tbl']/tr/td/table/tr/td")

   for link in firstpage_urls:
       hrefs = link.select('a/@href').extract()

Answer 1

使用内置 FormRequest 或 FormRequest.from_response 函数 - 在它们内部指定要提交到表单的数据。