如何使用 Selenium 和 Python 抓取传单地图多边形

Question

这篇文章与这篇文章非常相似：使用 selenium 和 python 在鼠标悬停后弹出时提取数据

但我无法找到我想要的答案。

我正在尝试抓取与此非常相似的传单地图：https://leafletjs.com/examples/choropleth/，理想情况下，我想下载将鼠标移到多边形上后出现的所有信息：

原始帖子循环遍历每个圆元素，我想对每个多边形执行相同的操作。

代码试验：

from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome
driver.get("https://leafletjs.com/examples/choropleth/")
timeout = 1000

explicit_wait30 = WebDriverWait(driver, 30)
try:
    # Wait for all circles to load
    poli = explicit_wait30.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, '.leaflet-interactive')))
except TimeoutException:
    driver.refresh()


data = []
i=1
for circle in poli:
    i+=1
    # Execute mouseover on the element
    driver.execute_script("const mouseoverEvent = new Event('mouseover');arguments[0].dispatchEvent(mouseoverEvent)", poli)
    # Wait for the data to appear
    listing = explicit_wait30.until(EC.visibility_of_element_located((By.CSS_SELECTOR, '#listingHover')))
    data.append(listing.text)
    # Close the listing
    driver.execute_script("arguments[0].click()", listing.find_element_by_tag_name('button'))
    print(i)
    if i>15 : 
        break

我得到错误：

JavascriptException: Message: javascript error: arguments[0].dispatchEvent is not a function
  (Session info: chrome=85.0.4183.102)

似乎“传单交互”元素没有鼠标悬停的事件类型，如何重现在多边形上移动鼠标的人类动作？

Answer 1

要抓取传单地图并提取将鼠标移动到多边形上后出现的所有信息，因为所需的元素位于 an 内，<iframe>因此您必须：

• 诱导WebDriverWait等待所需的框架可用并切换到它。

• 引发WebDriverWait以获得所需的visibility_of_element_ located。

• 您可以使用以下定位策略：

  driver.get('https://leafletjs.com/examples/choropleth/')
  driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//h2[text()='Interactive Choropleth Map']"))))
  WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[src='example.html']")))
  elements = driver.find_elements_by_css_selector("svg.leaflet-zoom-animated>g path")
  for element in elements:
      ActionChains(driver).move_to_element(element).perform()
      print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='info leaflet-control']"))).text)
  

• 注意：您必须添加以下导入：

  from selenium.webdriver.common.action_chains import ActionChains
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

• 控制台输出：

  US Population Density
  Alabama
  94.65 people / mi2
  US Population Density
  Hover over a state
  US Population Density
  Arizona
  57.05 people / mi2
  US Population Density
  Arkansas
  56.43 people / mi2
  US Population Density
  California
  241.7 people / mi2
  US Population Density
  Colorado
  49.33 people / mi2
  US Population Density
  Connecticut
  739.1 people / mi2
  US Population Density
  Delaware
  464.3 people / mi2
  US Population Density
  Maryland
  596.3 people / mi2
  US Population Density
  Hover over a state
  US Population Density
  Georgia
  169.5 people / mi2
  US Population Density
  Hover over a state
  US Population Density
  Montana
  6.858 people / mi2
  US Population Density
  Illinois
  231.5 people / mi2
  US Population Density
  Indiana
  181.7 people / mi2
  US Population Density
  Iowa
  54.81 people / mi2
  US Population Density
  Kansas
  35.09 people / mi2
  US Population Density
  Kentucky
  110 people / mi2
  US Population Density
  Mississippi
  63.5 people / mi2
  US Population Density
  Maine
  43.04 people / mi2
  US Population Density
  Virginia
  204.5 people / mi2
  US Population Density
  Massachusetts
  840.2 people / mi2
  US Population Density
  

参考

您可以在以下位置找到一些相关讨论：

• iframe下处理#document的方法
• 通过 Selenium 和 python 切换到 iframe