Elasticsearch之ik中文分词篇

ik分词器插件

es在7.3版本已经支持中文分词，由于中文分词只能支持到单个字进行分词，不够灵活与适配我们平常使用习惯，所以有很多对应中文分词出现，最近使用的是ik分词器，就说说它吧。

ik分词器安装

安装可以百度下有很多教程，需要注意的是ik分词器的版本要跟es版本对应上，避免出现不必要的兼容问题。

ik分词模式

ik_max_word: 将文本拆分成最细粒度的词语或者字

GET /test_analysis/_analyze
{"text": "是否分词","analyzer": "ik_max_word"
}

结果

{"tokens" : [{"token" : "是否","start_offset" : 0,"end_offset" : 2,"type" : "CN_WORD","position" : 0},{"token" : "是","start_offset" : 0,"end_offset" : 1,"type" : "CN_WORD","position" : 1},{"token" : "否","start_offset" : 1,"end_offset" : 2,"type" : "CN_WORD","position" : 2},{"token" : "分词","start_offset" : 2,"end_offset" : 4,"type" : "CN_WORD","position" : 3},{"token" : "分","start_offset" : 2,"end_offset" : 3,"type" : "CN_WORD","position" : 4},{"token" : "词","start_offset" : 3,"end_offset" : 4,"type" : "CN_WORD","position" : 5}]
}

ik_smart: 将文本拆分成最粗粒的词语

GET /test_analysis/_analyze
{"text": "是否分词","analyzer": "ik_smart"
}

结果

{"tokens" : [{"token" : "是否","start_offset" : 0,"end_offset" : 2,"type" : "CN_WORD","position" : 0},{"token" : "分词","start_offset" : 2,"end_offset" : 4,"type" : "CN_WORD","position" : 1}]
}

一般都用ik_max_word

es ik分词测试

创建索引

PUT /test_analysis
{"mappings": {"properties": {"message": {"type": "text","analyzer": "ik_max_word"},"id": {"type": "keyword"}}}
}

添加数据

POST /test_analysis/_bulk
{"index":{}}
{"id":"111", "message":"我是一个小可爱"}
{"index":{}}
{"id":"222", "message":"只是为了测试一下结果是否分词"}
{"index":{}}
{"id":"333", "message":"测试一下是否进行了ik分词"}
{"index":{}}
{"id":"444", "message":"搞一些假的数据吧"}
{"index":{}}
{"id":"555", "message":"实在不知道再写一些什么了"}
{"index":{}}
{"id":"666", "message":"就这样吧"}

查询

GET /test_analysis/_search
{"query": {"bool": {"must": [{"match": {"message": "是否分词"}}]}}
}

查询分词结果

{"took" : 1,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 3,"relation" : "eq"},"max_score" : 5.104265,"hits" : [{"_index" : "test_analysis","_type" : "_doc","_id" : "MDXEe4wBS_Neyb68FBdy","_score" : 5.104265,"_source" : {"id" : "333","message" : "测试一下是否进行了ik分词"}},{"_index" : "test_analysis","_type" : "_doc","_id" : "LzXEe4wBS_Neyb68FBdy","_score" : 5.0611815,"_source" : {"id" : "222","message" : "只是为了测试一下结果是否分词"}},{"_index" : "test_analysis","_type" : "_doc","_id" : "LjXEe4wBS_Neyb68FBdy","_score" : 0.728194,"_source" : {"id" : "111","message" : "我是一个小可爱"}}]}
}