聊聊HttpClient的重试机制

序

本文主要研究一下HttpClient的重试机制

HttpRequestRetryHandler

org/apache/http/client/HttpRequestRetryHandler.java

public interface HttpRequestRetryHandler {/*** Determines if a method should be retried after an IOException* occurs during execution.** @param exception the exception that occurred* @param executionCount the number of times this method has been* unsuccessfully executed* @param context the context for the request execution** @return {@code true} if the method should be retried, {@code false}* otherwise*/boolean retryRequest(IOException exception, int executionCount, HttpContext context);}

HttpRequestRetryHandler接口定义了retryRequest方法，它接收IOException、executionCount及context，然后判断是否可以重试

DefaultHttpRequestRetryHandler

org/apache/http/impl/client/DefaultHttpRequestRetryHandler.java

@Contract(threading = ThreadingBehavior.IMMUTABLE)
public class DefaultHttpRequestRetryHandler implements HttpRequestRetryHandler {public static final DefaultHttpRequestRetryHandler INSTANCE = new DefaultHttpRequestRetryHandler();/** the number of times a method will be retried */private final int retryCount;/** Whether or not methods that have successfully sent their request will be retried */private final boolean requestSentRetryEnabled;private final Set<Class<? extends IOException>> nonRetriableClasses;/*** Create the request retry handler using the specified IOException classes** @param retryCount how many times to retry; 0 means no retries* @param requestSentRetryEnabled true if it's OK to retry requests that have been sent* @param clazzes the IOException types that should not be retried* @since 4.3*/protected DefaultHttpRequestRetryHandler(final int retryCount,final boolean requestSentRetryEnabled,final Collection<Class<? extends IOException>> clazzes) {super();this.retryCount = retryCount;this.requestSentRetryEnabled = requestSentRetryEnabled;this.nonRetriableClasses = new HashSet<Class<? extends IOException>>();for (final Class<? extends IOException> clazz: clazzes) {this.nonRetriableClasses.add(clazz);}}/*** Create the request retry handler using the following list of* non-retriable IOException classes: <br>* <ul>* <li>InterruptedIOException</li>* <li>UnknownHostException</li>* <li>ConnectException</li>* <li>SSLException</li>* </ul>* @param retryCount how many times to retry; 0 means no retries* @param requestSentRetryEnabled true if it's OK to retry non-idempotent requests that have been sent*/@SuppressWarnings("unchecked")public DefaultHttpRequestRetryHandler(final int retryCount, final boolean requestSentRetryEnabled) {this(retryCount, requestSentRetryEnabled, Arrays.asList(InterruptedIOException.class,UnknownHostException.class,ConnectException.class,SSLException.class));}/*** Create the request retry handler with a retry count of 3, requestSentRetryEnabled false* and using the following list of non-retriable IOException classes: <br>* <ul>* <li>InterruptedIOException</li>* <li>UnknownHostException</li>* <li>ConnectException</li>* <li>SSLException</li>* </ul>*/public DefaultHttpRequestRetryHandler() {this(3, false);}/*** Used {@code retryCount} and {@code requestSentRetryEnabled} to determine* if the given method should be retried.*/@Overridepublic boolean retryRequest(final IOException exception,final int executionCount,final HttpContext context) {Args.notNull(exception, "Exception parameter");Args.notNull(context, "HTTP context");if (executionCount > this.retryCount) {// Do not retry if over max retry countreturn false;}if (this.nonRetriableClasses.contains(exception.getClass())) {return false;}for (final Class<? extends IOException> rejectException : this.nonRetriableClasses) {if (rejectException.isInstance(exception)) {return false;}}final HttpClientContext clientContext = HttpClientContext.adapt(context);final HttpRequest request = clientContext.getRequest();if(requestIsAborted(request)){return false;}if (handleAsIdempotent(request)) {// Retry if the request is considered idempotentreturn true;}if (!clientContext.isRequestSent() || this.requestSentRetryEnabled) {// Retry if the request has not been sent fully or// if it's OK to retry methods that have been sentreturn true;}// otherwise do not retryreturn false;}/*** @return {@code true} if this handler will retry methods that have* successfully sent their request, {@code false} otherwise*/public boolean isRequestSentRetryEnabled() {return requestSentRetryEnabled;}/*** @return the maximum number of times a method will be retried*/public int getRetryCount() {return retryCount;}/*** @since 4.2*/protected boolean handleAsIdempotent(final HttpRequest request) {return !(request instanceof HttpEntityEnclosingRequest);}/*** @since 4.2** @deprecated (4.3)*/@Deprecatedprotected boolean requestIsAborted(final HttpRequest request) {HttpRequest req = request;if (request instanceof RequestWrapper) { // does not forward request to originalreq = ((RequestWrapper) request).getOriginal();}return (req instanceof HttpUriRequest && ((HttpUriRequest)req).isAborted());}}

DefaultHttpRequestRetryHandler实现了HttpRequestRetryHandler接口，其无参构造器默认将InterruptedIOException、UnknownHostException、ConnectException、SSLException设定为不重试的异常，默认retryCount为3，requestSentRetryEnabled为false；其retryRequest方法先判断executionCount是否超出retryCount，接着判断异常类型是否是不重试的异常类型，若request为aborted则返回false，若request非HttpEntityEnclosingRequest则表示幂等请求，返回true，若请求未完全发送则返回true，其余的默认返回false。

RetryExec

org/apache/http/impl/execchain/RetryExec.java

@Contract(threading = ThreadingBehavior.IMMUTABLE_CONDITIONAL)
public class RetryExec implements ClientExecChain {private final Log log = LogFactory.getLog(getClass());private final ClientExecChain requestExecutor;private final HttpRequestRetryHandler retryHandler;public RetryExec(final ClientExecChain requestExecutor,final HttpRequestRetryHandler retryHandler) {Args.notNull(requestExecutor, "HTTP request executor");Args.notNull(retryHandler, "HTTP request retry handler");this.requestExecutor = requestExecutor;this.retryHandler = retryHandler;}@Overridepublic CloseableHttpResponse execute(final HttpRoute route,final HttpRequestWrapper request,final HttpClientContext context,final HttpExecutionAware execAware) throws IOException, HttpException {Args.notNull(route, "HTTP route");Args.notNull(request, "HTTP request");Args.notNull(context, "HTTP context");final Header[] origheaders = request.getAllHeaders();for (int execCount = 1;; execCount++) {try {return this.requestExecutor.execute(route, request, context, execAware);} catch (final IOException ex) {if (execAware != null && execAware.isAborted()) {this.log.debug("Request has been aborted");throw ex;}if (retryHandler.retryRequest(ex, execCount, context)) {if (this.log.isInfoEnabled()) {this.log.info("I/O exception ("+ ex.getClass().getName() +") caught when processing request to "+ route +": "+ ex.getMessage());}if (this.log.isDebugEnabled()) {this.log.debug(ex.getMessage(), ex);}if (!RequestEntityProxy.isRepeatable(request)) {this.log.debug("Cannot retry non-repeatable request");throw new NonRepeatableRequestException("Cannot retry request " +"with a non-repeatable request entity", ex);}request.setHeaders(origheaders);if (this.log.isInfoEnabled()) {this.log.info("Retrying request to " + route);}} else {if (ex instanceof NoHttpResponseException) {final NoHttpResponseException updatedex = new NoHttpResponseException(route.getTargetHost().toHostString() + " failed to respond");updatedex.setStackTrace(ex.getStackTrace());throw updatedex;}throw ex;}}}}}

RetryExec实现了ClientExecChain接口，其execute方法会循环执行requestExecutor.execute，它catch了IOException，对于retryHandler.retryRequest(ex, execCount, context)返回true的，会在通过RequestEntityProxy.isRepeatable(request)判断一下是否是可重复读取的request，不是则抛出NonRepeatableRequestException；对于retryHandler.retryRequest返回false的则针对NoHttpResponseException重新包装一下，将targetHost体现在message里头然后重新抛出

HttpEntityEnclosingRequest

org/apache/http/HttpEntityEnclosingRequest.java

public interface HttpEntityEnclosingRequest extends HttpRequest {/*** Tells if this request should use the expect-continue handshake.* The expect continue handshake gives the server a chance to decide* whether to accept the entity enclosing request before the possibly* lengthy entity is sent across the wire.* @return true if the expect continue handshake should be used, false if* not.*/boolean expectContinue();/*** Associates the entity with this request.** @param entity the entity to send.*/void setEntity(HttpEntity entity);/*** Returns the entity associated with this request.** @return entity*/HttpEntity getEntity();}

HttpEntityEnclosingRequest定义了getEntity、setEntity、expectContinue方法，它的子类有HttpPut、HttpPost、HttpPatch、HttpDelete等

RequestEntityProxy.isRepeatable

org/apache/http/impl/execchain/RequestEntityProxy.java

class RequestEntityProxy implements HttpEntity  {private final HttpEntity original;private boolean consumed = false;public boolean isConsumed() {return consumed;}static boolean isRepeatable(final HttpRequest request) {if (request instanceof HttpEntityEnclosingRequest) {final HttpEntity entity = ((HttpEntityEnclosingRequest) request).getEntity();if (entity != null) {if (isEnhanced(entity)) {final RequestEntityProxy proxy = (RequestEntityProxy) entity;if (!proxy.isConsumed()) {return true;}}return entity.isRepeatable();}}return true;}
}

RequestEntityProxy提供了静态方法isRepeatable用于判断该request的entity是否可以重复读取，对于非HttpEntityEnclosingRequest的返回true，是HttpEntityEnclosingRequest类型的话则判断entity.isRepeatable()，若entity是RequestEntityProxy类型的，则通过RequestEntityProxy.isConsumed来判断

entity.isRepeatable()

public interface HttpEntity {/*** Tells if the entity is capable of producing its data more than once.* A repeatable entity's getContent() and writeTo(OutputStream) methods* can be called more than once whereas a non-repeatable entity's can not.* @return true if the entity is repeatable, false otherwise.*/boolean isRepeatable();//......
}    

HttpEntity接口定义了isRepeatable方法，用于表示entity的content及OutputStream是否可以读写多次。其实现类里头，BufferedHttpEntity、ByteArrayEntity、EntityTemplate、FileEntity、SerializableEntity、StringEntity为true，BasicHttpEntity、InputStreamEntity、StreamingHttpEntity为false

小结

HttpRequestRetryHandler接口定义了retryRequest方法，它接收IOException、executionCount及context，然后判断是否可以重试

DefaultHttpRequestRetryHandler实现了HttpRequestRetryHandler接口，其无参构造器默认将InterruptedIOException、UnknownHostException、ConnectException、SSLException设定为不重试的异常，默认retryCount为3，requestSentRetryEnabled为false；其retryRequest方法先判断executionCount是否超出retryCount，接着判断异常类型是否是不重试的异常类型，若request为aborted则返回false，若request非HttpEntityEnclosingRequest则表示幂等请求，返回true，若请求未完全发送则返回true，其余的默认返回false。

RetryExec实现了ClientExecChain接口，其execute方法会循环执行requestExecutor.execute，它catch了IOException，对于retryHandler.retryRequest(ex, execCount, context)返回true的，会在通过RequestEntityProxy.isRepeatable(request)判断一下是否是可重复读取的request，不是则抛出NonRepeatableRequestException

DefaultHttpRequestRetryHandler针对不是幂等请求的HttpEntityEnclosingRequest类型(HttpPut、HttpPost、HttpPatch、HttpDelete)，不会重试；若retryHandler.retryRequest返回可以重试，RetryExec还有一个repeatable的判断，BufferedHttpEntity、ByteArrayEntity、EntityTemplate、FileEntity、SerializableEntity、StringEntity为true，BasicHttpEntity、InputStreamEntity、StreamingHttpEntity为false