Elasticsearch 之 ElasticsearchRestTemplate 聚合查询

前言：

上一篇我们分享了 ElasticsearchRestTemplate 的常用普通查询，本篇我们使用 ElasticsearchRestTemplate 来完成 Elasticsearch 更为复杂的聚合查询。

Elasticsearch 系列文章传送门

Elasticsearch 基础篇【ES】

Elasticsearch Windows 环境安装

Elasticsearch 之 ElasticsearchRestTemplate 普通查询

聚合查询

聚合查询是指从数据中提取统计信息，聚合又可以分为以下三类：

• 度量聚合：度量聚合其实就是计算平均值、求和、计数、最大值、最小值等的聚合查询。
• 桶聚合：桶聚合其实就是按指定的字段进行分组后聚合，例如先按汽车类型分类后，求汽车平均价格。
• 嵌套聚合：嵌套聚合其实就在嵌套二字中，也就是文档的嵌套，比如汽车除了一些主要信息之外，还有一些次要零部件的信息，在汽车主信息中嵌套一个次要的车载手机支架信息文档对象，当我们需要统计某个汽车类型的车载手机支架信息的时候，就需要使用到嵌套聚合了。

度量聚合

Max（最大值查询）

查询最大值是一个非常常见的业务场景，而查询最大值就需要使用到聚合查询了，聚合查询需要使用到 AggregationBuilders，使用 ElasticsearchRestTemplate 完成最大值查询代码如下：

public double carMaxPrice() {MaxAggregationBuilder maxAggregationBuilder = AggregationBuilders.max("maxPrice").field("price");NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();NativeSearchQuery build = queryBuilder.withQuery(QueryBuilders.matchAllQuery()).addAggregation(maxAggregationBuilder).build();SearchHits<CarDO> search = elasticsearchRestTemplate.search(build, CarDO.class);Aggregations aggregations = search.getAggregations();double maxPrice = 0;if (aggregations != null) {Max max = aggregations.get("maxPrice");maxPrice = max.getValue();}return maxPrice;
}

Min（最小值查询）

查询最小值是一个非常常见的业务场景，而查询最小值就需要使用到聚合查询了，聚合查询需要使用到 AggregationBuilders，使用 ElasticsearchRestTemplate 完成最小值查询代码如下：

public double carMinPrice() {MinAggregationBuilder minAggregationBuilder = AggregationBuilders.min("minPrice").field("price");NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();NativeSearchQuery build = queryBuilder.withQuery(QueryBuilders.matchAllQuery()).addAggregation(minAggregationBuilder).build();SearchHits<CarDO> search = elasticsearchRestTemplate.search(build, CarDO.class);Aggregations aggregations = search.getAggregations();double minPrice = 0;if (aggregations != null) {Min min = aggregations.get("minPrice");minPrice = min.getValue();}return minPrice;
}

Avg（平均值查询）

计算平均值是一个非常常见的业务场景，如果没有聚合查询我们就需要把目标数据全部查询出来之后，在进行求平均操作，而 Elasticsearch 给我们提供了平均值的聚合查询方法，查询平均值的代码如下：

public double carAvgPrice() {AvgAggregationBuilder avgAggregationBuilder = AggregationBuilders.avg("avgPrice").field("price");NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();NativeSearchQuery build = queryBuilder.withQuery(QueryBuilders.matchAllQuery()).addAggregation(avgAggregationBuilder).build();SearchHits<CarDO> search = elasticsearchRestTemplate.search(build, CarDO.class);Aggregations aggregations = search.getAggregations();double avgPrice = 0;if (aggregations != null) {Avg avg = aggregations.get("avgPrice");avgPrice = avg.getValue();}return avgPrice;
}

Sum（求和查询）

计算求和是一个非常常见的业务场景，如果没有聚合查询我们就需要把目标数据全部查询出来之后，在进行求和操作，而 Elasticsearch 给我们提供了求和的聚合查询方法，代码如下：

public double carSumPrice() {SumAggregationBuilder sumAggregationBuilder = AggregationBuilders.sum("sumPrice").field("price");NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();NativeSearchQuery build = queryBuilder.withQuery(QueryBuilders.matchAllQuery()).addAggregation(sumAggregationBuilder).build();SearchHits<CarDO> search = elasticsearchRestTemplate.search(build, CarDO.class);Aggregations aggregations = search.getAggregations();double sumPrice = 0;if (aggregations != null) {Sum sum = aggregations.get("sumPrice");sumPrice = sum.getValue();}return sumPrice;
}

Max、Min（同时查询最大、最小值）

前面我们分别演示了最大值、最小值、平均值、求和的场景，其实如果有同时要求最大值、最小值、平均值、求和的场景的时候，我们只可以在同一次查询得到结果的，代码如下：

public void aggregationCarPrice() {MaxAggregationBuilder maxAggregationBuilder = AggregationBuilders.max("maxPrice").field("price");MinAggregationBuilder minAggregationBuilder = AggregationBuilders.min("minPrice").field("price");AvgAggregationBuilder avgAggregationBuilder = AggregationBuilders.avg("avgPrice").field("price");SumAggregationBuilder sumAggregationBuilder = AggregationBuilders.sum("sumPrice").field("price");NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();NativeSearchQuery build = queryBuilder.withQuery(QueryBuilders.matchAllQuery()).addAggregation(maxAggregationBuilder).addAggregation(minAggregationBuilder).addAggregation(avgAggregationBuilder).addAggregation(sumAggregationBuilder).build();SearchHits<CarDO> search = elasticsearchRestTemplate.search(build, CarDO.class);Aggregations aggregations = search.getAggregations();double maxPrice = 0;if (aggregations != null) {Max max = aggregations.get("maxPrice");maxPrice = max.getValue();Min min = aggregations.get("minPrice");double minPrice = min.getValue();Avg avg = aggregations.get("avgPrice");double avgPrice = avg.getValue();Sum sum = aggregations.get("sumPrice");double sumPrice = sum.getValue();log.info("汽车最贵价格:{},汽车最便宜价格:{},汽车平均价格:{},汽车价格之和:{}", maxPrice, minPrice, avgPrice, sumPrice);}
}

可以看到我们使用一次查询就可以得到最大值、最小值、平均值、求和。

桶聚合

数值类字段桶聚合

我们统计一下不同价格的汽车各有多少个，实现这个业务也需要使用聚合查询，代码如下：

public void categoryPriceCountCar() {TermsAggregationBuilder termsAggregationBuilder = AggregationBuilders.terms("priceCount").field("price");NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();NativeSearchQuery build = queryBuilder.withQuery(QueryBuilders.matchAllQuery()).addAggregation(termsAggregationBuilder).build();SearchHits<CarDO> search = elasticsearchRestTemplate.search(build, CarDO.class);Aggregations aggregations = search.getAggregations();if (aggregations != null) {Terms terms = (Terms) aggregations.asMap().get("priceCount");for (Terms.Bucket bucket : terms.getBuckets()) {String price = bucket.getKeyAsString();long priceCount = bucket.getDocCount();log.info("汽车价格:{},有:{} 辆", price, priceCount);}}
}

字符串类型字段桶聚合

我们统计一下不同颜色的汽车各有多少辆，实现这个业务也需要使用聚合查询，使用按价格统计的方式来实现，代码如下：

public void categoryColorCountCar() {TermsAggregationBuilder termsAggregationBuilder = AggregationBuilders.terms("colorCount").field("color");NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();NativeSearchQuery build = queryBuilder.withQuery(QueryBuilders.matchAllQuery()).addAggregation(termsAggregationBuilder).build();SearchHits<CarDO> search = elasticsearchRestTemplate.search(build, CarDO.class);Aggregations aggregations = search.getAggregations();if (aggregations != null) {Terms terms = (Terms) aggregations.asMap().get("colorCount");for (Terms.Bucket bucket : terms.getBuckets()) {String color = bucket.getKeyAsString();long colorCount = bucket.getDocCount();log.info("汽车颜色:{},有:{} 辆", color, colorCount);}}
}

执行上述代码我们发现控制台报错如下：

org.elasticsearch.ElasticsearchException: Elasticsearch exception [type=illegal_argument_exception, reason=Fielddata is disabled on text fields by default. Set fielddata=true on [color] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.]at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:496) ~[elasticsearch-7.9.3.jar:7.9.3]at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:407) ~[elasticsearch-7.9.3.jar:7.9.3]at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:437) ~[elasticsearch-7.9.3.jar:7.9.3]at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:407) ~[elasticsearch-7.9.3.jar:7.9.3]at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:437) ~[elasticsearch-7.9.3.jar:7.9.3]at org.elasticsearch.ElasticsearchException.failureFromXContent(ElasticsearchException.java:603) ~[elasticsearch-7.9.3.jar:7.9.3]at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:179) ~[elasticsearch-7.9.3.jar:7.9.3]at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1892) ~[elasticsearch-rest-high-level-client-7.9.3.jar:7.9.3]at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1869) ~[elasticsearch-rest-high-level-client-7.9.3.jar:7.9.3]at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1626) ~[elasticsearch-rest-high-level-client-7.9.3.jar:7.9.3]at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1583) ~[elasticsearch-rest-high-level-client-7.9.3.jar:7.9.3]at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1553) ~[elasticsearch-rest-high-level-client-7.9.3.jar:7.9.3]at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:1069) ~[elasticsearch-rest-high-level-client-7.9.3.jar:7.9.3]at org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate.lambda$search$10(ElasticsearchRestTemplate.java:256) ~[spring-data-elasticsearch-4.1.8.jar:4.1.8]at org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate.execute(ElasticsearchRestTemplate.java:343) ~[spring-data-elasticsearch-4.1.8.jar:4.1.8]at org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate.search(ElasticsearchRestTemplate.java:256) ~[spring-data-elasticsearch-4.1.8.jar:4.1.8]at org.springframework.data.elasticsearch.core.AbstractElasticsearchTemplate.search(AbstractElasticsearchTemplate.java:446) ~[spring-data-elasticsearch-4.1.8.jar:4.1.8]at com.order.service.service.impl.CarEsServiceImpl.categoryColorCountCar(CarEsServiceImpl.java:288) ~[classes/:na]at com.order.service.controller.ElasticsearchController.categoryColorCountCar(ElasticsearchController.java:164) ~[classes/:na]at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_121]at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_121]at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_121]at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_121]at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:197) ~[spring-web-5.3.6.jar:5.3.6]at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:141) ~[spring-web-5.3.6.jar:5.3.6]at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106) ~[spring-webmvc-5.3.6.jar:5.3.6]at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:894) ~[spring-webmvc-5.3.6.jar:5.3.6]at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808) ~[spring-webmvc-5.3.6.jar:5.3.6]at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.3.6.jar:5.3.6]at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1060) ~[spring-webmvc-5.3.6.jar:5.3.6]at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:962) ~[spring-webmvc-5.3.6.jar:5.3.6]at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) ~[spring-webmvc-5.3.6.jar:5.3.6]at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) ~[spring-webmvc-5.3.6.jar:5.3.6]at javax.servlet.http.HttpServlet.service(HttpServlet.java:626) ~[tomcat-embed-core-9.0.45.jar:4.0.FR]at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) ~[spring-webmvc-5.3.6.jar:5.3.6]at javax.servlet.http.HttpServlet.service(HttpServlet.java:733) ~[tomcat-embed-core-9.0.45.jar:4.0.FR]at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) ~[tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) ~[tomcat-embed-websocket-9.0.45.jar:9.0.45]at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.45.jar:9.0.45]at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-5.3.6.jar:5.3.6]at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.6.jar:5.3.6]at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.45.jar:9.0.45]at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-5.3.6.jar:5.3.6]at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.6.jar:5.3.6]at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.45.jar:9.0.45]at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:93) ~[spring-boot-actuator-2.4.5.jar:2.4.5]at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.6.jar:5.3.6]at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.45.jar:9.0.45]at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-5.3.6.jar:5.3.6]at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.6.jar:5.3.6]at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) ~[tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) [tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542) [tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143) [tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) [tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357) [tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374) [tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893) [tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1707) [tomcat-embed-core-9.0.45.jar:9.0.45]at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-9.0.45.jar:9.0.45]at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_121]at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_121]at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-9.0.45.jar:9.0.45]at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]

错误的核心信息如下：

[type=illegal_argument_exception, reason=Fielddata is disabled on text fields by default. Set fielddata=true on [color] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.]

大概得意思就是 color 这个字段有问题，上面我们使用 price 字段就可以，为啥 color 字段就有问题呢？

原因是因为 price 是数值类型的字段，而 color 是 text 类型的字段，在 Elasticsearch 中 text 类型的字段不能直接进行分组聚合，我们需要将其标记为 keyword 类型的字符串可以分组聚合。

在字段上增加如下如下注解：

//颜色
@Field(store = true, type = FieldType.Keyword)
private String color;

同时修改查询代码如下：

public void categoryColorCountCar() {TermsAggregationBuilder termsAggregationBuilder = AggregationBuilders.terms("colorCount").field("color.keyword");NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();NativeSearchQuery build = queryBuilder.withQuery(QueryBuilders.matchAllQuery()).addAggregation(termsAggregationBuilder).build();SearchHits<CarDO> search = elasticsearchRestTemplate.search(build, CarDO.class);Aggregations aggregations = search.getAggregations();if (aggregations != null) {Terms terms = (Terms) aggregations.asMap().get("colorCount");for (Terms.Bucket bucket : terms.getBuckets()) {String color = bucket.getKeyAsString();long colorCount = bucket.getDocCount();log.info("汽车颜色:{},有:{} 辆", color, colorCount);}}
}

执行结果如下：

2024-12-30 20:43:58.621  INFO 42108 --- [nio-8086-exec-2] c.o.s.service.impl.CarEsServiceImpl      : 汽车颜色:钻石黑,有:2 辆
2024-12-30 20:43:58.621  INFO 42108 --- [nio-8086-exec-2] c.o.s.service.impl.CarEsServiceImpl      : 汽车颜色:雅灰,有:2 辆

执行结果如何预期，上述编码的重点在 color.keyword，我们在查询的字段后面要带上 keyword。

嵌套聚合查询本篇暂不讲，在实现嵌套聚合查询的时候踩了很多坑，找了多种解决方法才实现，下一篇会对嵌套查询场景单独分享。

总结：本篇简单分享了使用 ElasticsearchRestTemplate 实现 Elasticsearch 的各种聚合查询，希望可以帮助到对 ElasticsearchRestTemplate API 使用不熟悉的朋友们。

如有不正确的地方欢迎各位指出纠正。