sub*_*754 6 java ssl web-scraping jsoup
我正在尝试使用Jsoup从站点获取数据.链接到该网站是点击这里!
这是我获取数据的代码.`
// WARNING: do it only if security isn't important, otherwise you have
// to follow this advices: http://stackoverflow.com/a/7745706/1363265
// Create a trust manager that does not validate certificate chains
TrustManager[] trustAllCerts = new TrustManager[]{new X509TrustManager(){
public X509Certificate[] getAcceptedIssuers(){return null;}
public void checkClientTrusted(X509Certificate[] certs, String authType){}
public void checkServerTrusted(X509Certificate[] certs, String authType){}
}};
// Install the all-trusting trust manager
try {
SSLContext sc = SSLContext.getInstance("TLS");
sc.init(null, trustAllCerts, new SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
} catch (Exception e) {
;
}`
Run Code Online (Sandbox Code Playgroud)
String url = "https://www.sos.nh.gov/corporate/soskb/SearchResults.asp?FormName=CorpNameSearch&Words=Starting&SearchStr="+query+"&SearchType=Search";
Connection.Response response = Jsoup.connect(url).timeout(30000)
.method(Connection.Method.GET)
.userAgent("Mozilla/5.0 (Windows NT 10.0; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0")
.execute();
Document document = response.parse();
请告诉我这里的错误是什么.
您希望在此使用Java 8,因为它默认支持TLSv1.2以及其他必需的密码套件.
我在Java 7(1.7.0_45)的盒子上测试并得到了同样的错误.
我激活了调试消息并强制使用TLSv1.2.
System.setProperty("javax.net.debug", "all");
System.setProperty("https.protocols", "TLSv1.2");
Run Code Online (Sandbox Code Playgroud)
然后我点击了这个新错误:
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
Run Code Online (Sandbox Code Playgroud)
最后,我去了Comodoca的SSL分析仪,看到了一些有趣的东西.根据SSL分析器,您定位的网站仅启用了以下密码套件:
Cipher Suites Enabled Name (ID) Key Size (in bits) TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xC02F) 128 ECDH 256-bit (P-256) TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xC030) 256 ECDH 256-bit (P-256) TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x9E) 128 DH 2048-bit TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x9F) 256 DH 2048-bit
(见全部细节)
就我而言,我没有上述套房.检查你是否有它们:
SSLContext sc = SSLContext.getInstance("TLS");
sc.init(null, null, new java.security.SecureRandom());
String[] scs = sc.getSocketFactory().getSupportedCipherSuites();
Arrays.sort(scs);
for(String s : scs) {
System.out.println(s);
}
Run Code Online (Sandbox Code Playgroud)
请参阅SSLSocketFactoryEx以启用所需的密码套件.
另一方面,我成功运行代码,从Java 7迁移到Java 8(1.8.0_20),默认情况下支持TLS v1.2,并提供所需的密码套件.
以下是Windows 7上Java 8(1.8.0_20)支持的密码套件(总共71套件)的修整列表.
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
...
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
Run Code Online (Sandbox Code Playgroud)
片段
try {
// Create a trust manager that does not validate certificate chains
TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(java.security.cert.X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(java.security.cert.X509Certificate[] certs, String authType) {
}
} };
// Install the all-trusting trust manager
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new java.security.SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
// Fetch url
String url = "https://www.sos.nh.gov/corporate/soskb/SearchResults.asp?FormName=CorpNameSearch&Words=All&SearchStr=facebook&SearchType=Search";
Connection.Response response = Jsoup //
.connect(url) //
.timeout(60000) //
.method(Connection.Method.GET) //
.userAgent("Mozilla/5.0 (Windows NT 10.0; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0") //
.execute();
Document document = response.parse();
System.out.println(document);
} catch (Exception e) {
e.printStackTrace();
}
Run Code Online (Sandbox Code Playgroud)
在安全方面,始终使用最新的更新版本.
| 归档时间: |
|
| 查看次数: |
18375 次 |
| 最近记录: |