Elasticsearch Scroll 报错entity content is too long

2024-06-24 15:22:01:568 ERROR [task-31] (ScrollFetcherProduceAction.java:129) 访问ES出错org.apache.http.ContentTooLongException: entity content is too long [112750110] for the configured buffer limit [104857600]at org.elasticsearch.client.HeapBufferedAsyncResponseConsumer.onEntityEnclosed(HeapBufferedAsyncResponseConsumer.java:76) ~[elasticsearch-rest-client-6.1.1.jar:6.1.1]at org.apache.http.nio.protocol.AbstractAsyncResponseConsumer.responseReceived(AbstractAsyncResponseConsumer.java:137) ~[httpcore-nio-4.4.13.jar:4.4.13]at org.apache.http.impl.nio.client.MainClientExec.responseReceived(MainClientExec.java:315) ~[httpasyncclient-4.1.4.jar:4.1.4]at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseReceived(DefaultClientExchangeHandlerImpl.java:151) ~[httpasyncclient-4.1.4.jar:4.1.4]at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.responseReceived(HttpAsyncRequestExecutor.java:315) ~[httpcore-nio-4.4.13.jar:4.4.13]at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:255) ~[httpcore-nio-4.4.13.jar:4.4.13]

high-level-rest api 在获取大量数据的时候报了这个错。 之前的解决办法是修改Option 设置heap 大小，但是这次集群是使用的6.1.1 版本，没有这个参数

办法：

在代码中修改Scroll Request的 size() 函数

```

searchRequest.scroll(keepAlive).source().size(2000);

```

在我的程序里用一个参数控制

elastic.sql.fetch.scroll.size: 2000

我默认是5000，把这个改成小一些，一次http请求就不会返回那么多数据就可以解决。