我正在尝试从以下网址抓取数据:\n https://university.careers360.com/colleges/list-of- Degree-colleges-in-India \n我想单击每个大学名称并获取每个学院的具体数据。
\n\n首先,我所做的是将所有大学网址收集到一个向量中:
\n\n#loading the package:\nlibrary(xml2)\nlibrary(rvest)\nlibrary(stringr)\nlibrary(dplyr)\n\n#Specifying the url for desired website to be scrapped\nbaseurl <- "https://university.careers360.com/colleges/list-of-degree-colleges-in-India"\n\n#Reading the html content from Amazon\nbasewebpage <- read_html(baseurl)\n\n#Extracting college name and its url\nscraplinks <- function(url){\n #Create an html document from the url\n webpage <- xml2::read_html(url)\n #Extract the URLs\n url_ <- webpage %>%\n rvest::html_nodes(".title a") %>%\n rvest::html_attr("href") \n #Extract the link text\n link_ <- webpage %>%\n rvest::html_nodes(".title a") %>%\n rvest::html_text()\n return(data_frame(link = link_, url = url_))\n}\n\n#College names and Urls\nallcollegeurls<-scraplinks(baseurl)\nRun Code Online (Sandbox Code Playgroud)\n\n到目前为止工作正常,但是当我对每个 url 使用 read_html 时,它显示错误。
\n\n#Reading the each url\nfor (i in allcollegeurls$url) {\n clgwebpage <- read_html(allcollegeurls$url[i])\n}\nRun Code Online (Sandbox Code Playgroud)\n\n\n\n\n错误:当前工作目录 (\'C:/Users/User/Documents\') 中不存在 \'NA\'。
\n
我什至使用了“break”命令,但仍然出现同样的错误:
\n\n#Reading the each url\nfor (i in allcollegeurls$url) {\n clgwebpage <- read_html(allcollegeurls$url[i])\n if(is.na(allcollegeurls$url[i]))break\n}\nRun Code Online (Sandbox Code Playgroud)\n\n请帮忙。
\n\n按要求发布所有collegeurls的str -:
\n\n> str(allcollegeurls)\nClasses \xe2\x80\x98tbl_df\xe2\x80\x99, \xe2\x80\x98tbl\xe2\x80\x99 and \'data.frame\': 30 obs. of 2 variables:\n $ link: chr "Netaji Subhas Institute of Technology, Delhi" "Hansraj \nCollege, Delhi" "School of Business, University of Petroleum and Energy \nStudies, D.." "Hindu College, Delhi" ...\n $ url : chr "https://www.careers360.com/university/netaji-subhas- \n university-of-technology-new-delhi" \n"https://www.careers360.com/colleges/hansraj-college-delhi" \n"https://www.careers360.com/colleges/school-of-business-university-of- \n petroleum-and-energy-studies-dehradun" \n"https://www.careers360.com/colleges/hindu-college-delhi" ...\nRun Code Online (Sandbox Code Playgroud)\n
这个作品,
purrr::map(allcollegeurls$url, read_html)
Run Code Online (Sandbox Code Playgroud)
映射函数:映射函数通过将函数应用于每个元素并返回与输入长度相同的向量来转换其输入。我喜欢避免for在 R 中使用。