直接使用 HttpClient 进行 https 请求，会由于证书问题导致请求失败，既然我们想利用程序访问某个网站(比如做爬虫)，其实我们对证书并不关注，可以采用忽略证书校验的方式来实现对 https 请求的访问。

具体实现流程

依赖的 HttpClient jar 包版本

org.apache.httpcomponents

httpclient

4.5.3

封装获取 HttpClient 的方法

public static CloseableHttpClient getHttpClient() {

try {

SSLContextBuilder builder = new SSLContextBuilder();

builder.loadTrustMaterial(null, new TrustSelfSignedStrategy());

//不进行主机名验证

SSLConnectionSocketFactory sslConnectionSocketFactory = new SSLConnectionSocketFactory(builder.build(),

NoopHostnameVerifier.INSTANCE);

Registry registry = RegistryBuilder. create()

.register("http", new PlainConnectionSocketFactory())

.register("https", sslConnectionSocketFactory)

.build();

PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(registry);

cm.setMaxTotal(100);

CloseableHttpClient httpclient = HttpClients.custom()

.setSSLSocketFactory(sslConnectionSocketFactory)

.setDefaultCookieStore(new BasicCookieStore())

.setConnectionManager(cm).build();

return httpclient;

} catch (KeyManagementException e) {

e.printStackTrace();

} catch (NoSuchAlgorithmException e) {

e.printStackTrace();

} catch (KeyStoreException e) {

e.printStackTrace();

}

return HttpClients.createDefault();

}

创建一个利用 HttpClient 发送 get 请求的工具方法

public static String get(String uri, Map params, Map headers) throws IOException {

HttpClient client = getHttpClient();

return get(client, uri, params, headers);

}

public static String get(HttpClient client, String uri, Map params, Map headers) throws IOException {

String result = StringUtils.EMPTY;

String fullUrl = buildUrlWithParams(uri, params);

HttpGet httpGet = new HttpGet(fullUrl);

if (headers != null) {

for (Entry entry : headers.entrySet()) {

httpGet.addHeader(entry.getKey(), entry.getValue());

}

}

HttpResponse httpResponse = client.execute(httpGet);

InputStream input = httpResponse.getEntity().getContent();

if (null != input) {

try {

result = IOUtils.toString(input, "UTF-8");

} catch (IOException e) {

throw e;

} finally {

IOUtils.closeQuietly(input);

}

}

return result;

}

private static String buildUrlWithParams(String uri, Map params) throws UnsupportedEncodingException {

StringBuilder urlBuilder = new StringBuilder(uri);

if (null != params && !params.isEmpty()) {

if (!uri.contains("?")) {

urlBuilder.append("?");

}

for (Map.Entry entry : params.entrySet()) {

String key = entry.getKey();

Object value = entry.getValue();

String valueStr = null == value ? "" : value.toString();

if (!urlBuilder.toString().endsWith("?")) {

urlBuilder.append("&");

}

urlBuilder.append(key).append("=").append(URLEncoder.encode(valueStr, "utf-8"));

}

}

String fullUrl = urlBuilder.toString();

return fullUrl;

}

测试代码

public class HttpsReqTest extends TestCase {

public void reqHttpsUrlTest() throws IOException {

String url = "https://www.xxx.com/";

String result = get(url, null, null);

System.out.println(result);

}

}

问题总结

上面这段代码其实已经可以满足大部分 https 地址的请求，本来我也以为高枕无忧了，但是昨天我请求一个 https 地址的时候(地址就不暴露了)，又发现了新的问题，刚好补充下，后续有其他问题也会继续补充。

问题一

问题描述

Exception in thread "main" javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: Certificates does not conform to algorithm constraints

at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)

at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904)

at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:279)

at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:273)

at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1446)

at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:209)

at sun.security.ssl.Handshaker.processLoop(Handshaker.java:901)

at sun.security.ssl.Handshaker.process_record(Handshaker.java:837)

at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1023)

at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1332)

at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1359)

at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1343)

at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396)

at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)

at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)

at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)

at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)

at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)

at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)

at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)

at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)

at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)

at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)

at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)

at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)

Caused by: java.security.cert.CertificateException: Certificates does not conform to algorithm constraints

at sun.security.ssl.AbstractTrustManagerWrapper.checkAlgorithmConstraints(SSLContextImpl.java:1018)

at sun.security.ssl.AbstractTrustManagerWrapper.checkAdditionalTrust(SSLContextImpl.java:944)

at sun.security.ssl.AbstractTrustManagerWrapper.checkServerTrusted(SSLContextImpl.java:886)

at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1428)

... 26 more

问题原因

原因是 Java 7/8 版本增加了安全性

解决方案

把 %JAVA_HOME%/jre/lib/security/java.security 文件里的

jdk.certpath.disabledAlgorithms=MD2, DSA, RSA keySize < 1024

改成

jdk.certpath.disabledAlgorithms=

就是去掉 "MD2, DSA, RSA keySize < 1024"。

用 notepad++ 修改的时候会提示文件被占用，可以用管理员运行 notepad++ 在进行修改即可。