我正在尝试加载我下载的本地 html 文件。有人知道怎么做这个吗？我目前正在获取状态代码 [404]。
这就是我的做法
HtmlPage htmlPage=webClient.getPage("file:\\\\Users\\njd202\\Desktop\\zipmeupbich\\springbootmongodb\\src\\main\\resources\\downloadedfiles\\Cuban2.html") ;
我在Java中使用HTMLUnit连接到远程URL并从获得的网页中获取一些信息.
我使用以下代码:
final WebClient webClient = new WebClient(BrowserVersion.INTERNET_EXPLORER_6_0, "companyproxy.server", 8080);
final DefaultCredentialsProvider scp = new DefaultCredentialsProvider();
scp.addProxyCredentials("username", "password","companyproxy.server",8080);
webClient.setCredentialsProvider(scp);
final URL url = new URL("http://htmlunit.sourceforge.net");
final HtmlPage page = (HtmlPage)webClient.getPage(url);
System.out.println(page.asXml());
提供代理服务器的详细信息后,我收到此错误消息:
SEVERE: Credentials cannot be used for NTLM authentication:
org.apache.commons.httpclient.UsernamePasswordCredentials
org.apache.commons.httpclient.auth.InvalidCredentialsException: Credentials cannot be used for NTLM authentication: org.apache.commons.httpclient.UsernamePasswordCredentials
at org.apache.commons.httpclient.auth.NTLMScheme.authenticate(NTLMScheme.java:332)
at org.apache.commons.httpclient.HttpMethodDirector.authenticateProxy(HttpMethodDirector.java:320)
at org.apache.commons.httpclient.HttpMethodDirector.authenticate(HttpMethodDirector.java:232)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:97)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1477)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1435)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:327)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:388)
at com.test.Test.main(Test.java:25)
Jun 5, 2009 9:28:35 AM org.apache.commons.httpclient.HttpMethodDirector …我目前正在使用HtmlUnit尝试从页面中抓取一个href并且遇到了一些麻烦.
XPath是:
/html/body/div[2]/div/div/table/tbody/tr/td[2]/div/div[5]/div/div[2]/span/a
在网页上看起来像:
<a class="t" title="This Brush" href=http://domain.com/this/that">Brush Set</a>
在我的代码中我正在做:
hrefs = page.getByXPath("//html/body/div[2]/div/div/table/tbody/tr/td[2]/div/div[5]/div/div[2]/span/a[@class='t']")
但是,这会返回那里的所有内容,而不仅仅是我想要的网址.
有人可以解释我必须添加什么来获得href？(也不以.html结尾)
我有一个HTML文件,其中postIt()在<script>标记内定义了JavaScript方法(比如说).它的标记前面有两个不再存在的外部JavaScript文件的引用.所有这些<script>标签都在里面<body>.
我必须设置WebClient.setThrowExceptionOnFailingStatusCode为false,以便我可以继续使用不存在的JavaScript文件.
但是当我尝试调用时page.executeJavascript("postIt()"),我得到Reference not found错误!
我正在尝试在标签之间获取文本<dev>Text Here</dev>:
<div id="tt" class="info">
Text Here
</div>
Output: Text Here
如何在java中使用正则表达式来实现这一点？谢谢.
编辑:
我正在使用HtmlUnit:
currentPage.getElementById("tt").asXml();
currentPage.getElementById("tt").asText(); // returns ""
我是测试的新手.
我在Linux上工作.我正在阅读有关无头模式的测试,并遇到了两件事.一个是X虚拟帧缓冲区,它在内存中进行图形操作.因此,不显示任何输出.我在此链接http://www.seleniumtests.com/2012/04/headless-tests-with-firefox-webdriver.html中找到的实施细节.
我遇到的另一个是HtmlUnitDriver.这也不会在运行测试时打开任何浏览器.我使用HtmlUnitDriver编写了一个基本的示例代码,断言似乎工作正常.
我知道HtmlUnitDriver与javascript的效果不太好.但除此之外,选择其中一个有什么重大差异吗？
我将测试一个确实有一些javascript的web应用程序.
我是这个领域的新手.所以,任何答案,建议等将不胜感激.先感谢您
我有一个网站,当我向下滚动时会生成更多产品.与其他网站不同,firebug控制台中没有任何内容.所以,我使用selenium来模拟浏览器.我已经使它工作,但与Firefox驱动程序.但是,由于我正在托管在命令行上运行的Web服务器,因此我使用的是HTMLUNIT.有人能告诉我如何使用HTMLUNIT滚动页面吗？这是现在的代码:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
browser = webdriver.Remote("http://127.0.0.1:4444/wd/hub",desired_capabilities=webdriver.DesiredCapabilities.HTMLUNITWITHJS)
browser.get("http://www.somewebsite.com/")
x = browser.find_elements_by_xpath("//div[@id='containeriso3']/div/a[1]")
hrefs = [i.get_attribute('href') for i in x]
print len(hrefs)
time.sleep(2)
browser.execute_script("scroll(0, 2500);")
time.sleep(2)
x = browser.find_elements_by_xpath("//div[@id='containeriso3']/div/a[1]")
hrefs = [i.get_attribute('href') for i in x]
print len(hrefs)
谢谢.
我用firefoxdriver和firefox 21对htmlunit进行了对硒
的性能测试.性能测试是在我的windows7机器上通过Eclipse进行的.
当两者都禁用javascript时,性能是相同的.
当两个都打开javascript htmlunit 2.12 比firefox慢150%.
我想这是因为spidermonkey引擎对犀牛的优越性.
有没有办法配置rhino会更快？
有没有更好的方式我们可以加速htmlunit？
package utils;
import java.io.IOException;
import java.net.MalformedURLException;
import java.text.DateFormat;
import java.util.Date;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxProfile;
import com.gargoylesoftware.htmlunit.BrowserVersion;
import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
public class PerformanceTest {
public static void main(String[] args) {
String[] urls = new String[] {
...
};
Date beforeSelenium = new Date();
System.out.println("Going to run selenium");
testSelenium(urls);
Date afterSelenium = new Date();
Date beforehtmlUnit= new Date();
System.out.println("Going to run htmlunit");
testHtmlUnit(urls);
Date …我正试图在这个网站上做一点点抓取以编程方式查找轮询信息.我最初尝试使用Python,它非常适合加载网站并在aspx表单周围导航,但无法提取嵌入的地图数据(因为没有包(至今)处理javascript).所以我选择了除掉我的Java技能并打破HtmlUnit.但是,我几乎立即遇到了障碍.
看起来好像网站上存在一些不存在的javascript文件的死链接.当HtmlUnit尝试加载它们时,它会获得404并自我毁灭.
Jul 21, 2013 9:51:22 PM com.gargoylesoftware.htmlunit.html.HtmlPage loadExternalJavaScriptFile
SEVERE: Error loading JavaScript from [http://www.eci-polldaymonitoring.nic.in/psl/GoogleMapForASPNet.ascx/jsdebug].
com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException: 404 Not Found for http://www.eci-polldaymonitoring.nic.in/psl/GoogleMapForASPNet.ascx/jsdebug
at com.gargoylesoftware.htmlunit.WebClient.throwFailingHttpStatusCodeExceptionIfNecessary(WebClient.java:544)
at com.gargoylesoftware.htmlunit.html.HtmlPage.loadJavaScriptFromUrl(HtmlPage.java:1119)
at com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:1059)
at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:399)
at com.gargoylesoftware.htmlunit.html.HtmlScript$3.execute(HtmlScript.java:260)
at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:276)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:676)
at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:635)
at org.cyberneko.html.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1170)
at org.cyberneko.html.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1072)
at org.cyberneko.html.filters.DefaultFilter.endElement(DefaultFilter.java:206)
at org.cyberneko.html.filters.NamespaceBinder.endElement(NamespaceBinder.java:330)
at org.cyberneko.html.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3074)
at org.cyberneko.html.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2041)
at org.cyberneko.html.HTMLScanner.scanDocument(HTMLScanner.java:918)
at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:499)
at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:452)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.parse(HTMLParser.java:892)
at com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:241)
at com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:187)
at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:268)
at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:156)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:434)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:309) …我编写了一个解析程序来评估信息列表,并在条目末尾附加价格.我正在使用HTMLUnit来获取我想要使用asText()查看的页面的字符串.我的代码段如下:
HtmlPage page = wc.getPage(name.substring(0, name.length() - 1));
wc.waitForBackgroundJavaScriptStartingBefore(100);
String test = page.asText();
在我的一个案例中,字符串的索引计数为111,471(根据Eclipse调试器),它以"..."结尾,我假设这意味着此特定变量在内存中没有更多空间来存储额外的信息.字符串不包含整个网页,我需要提取的位位于页面的最底部.有没有办法解析这么大的字符串？