一起來學ElasticSearch(七)
前言
目前正在出一個Es專題系列教程, 篇幅會較多, 喜歡的話，給個關注❤️ ~
承接上文，本節把上節遺留的條件查詢操作給大家講一下~
為了方便學習, 本節中所有示例沿用上節的索引。本文偏實戰一些，好了, 廢話不多說直接開整吧~
多條件組合查詢
bool
es中使用bool來控制多條件查詢，bool查詢支援以下引數:
must：被查詢的資料必須滿足當前條件mush_not：被查詢的資料必須不滿足當前條件should：被查詢的資料應該滿足當前條件。should查詢被用於修正查詢結果的評分。需要注意的是，如果組合查詢中沒有must，那麼被查詢的資料至少要匹配一條should。如果有must語句，那麼就無須匹配should，should將完全用於修正查詢結果的評分filter：被查詢的資料必須滿足當前條件，但是filter操作不涉及查詢結果評分。僅用於條件過濾
下面通過一個例子來看下如何使用:
json
GET class_1/_search
{
"query": {
"bool": {
"must": [
{"match": {
"name": "apple"
}}
],
"must_not": [
{"term": {
"num": {
"value": "5"
}
}}
],
"should": [
{"match": {
"name": "k"
}}
],"filter": [
{"range": {
"num": {
"gte": 0,
"lte": 10
}
}}
]
}
}
}
結果返回:
```json { "took" : 9, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 3, "relation" : "eq" }, "max_score" : 0.752627, "hits" : [ { "_index" : "class_1", "_type" : "_doc", "_id" : "b8fcCoYB090miyjed7YE", "_score" : 0.752627, "_source" : { "name" : "I eat apple so haochi1~", "num" : 1 } }, { "_index" : "class_1", "_type" : "_doc", "_id" : "ccfcCoYB090miyjed7YE", "_score" : 0.752627, "_source" : { "name" : "I eat apple so haochi3~", "num" : 1 } }, { "_index" : "class_1", "_type" : "_doc", "_id" : "cMfcCoYB090miyjed7YE", "_score" : 0.7389809, "_source" : { "name" : "I eat apple so zhen haochi2~", "num" : 1 } } ] } }
```
constant_score
constant_score查詢可以通過boost指定一個固定的評分，通常來說，constant_score的作用是代替一個只有filter的bool查詢
下面看具體使用：
json
GET class_1/_search
{
"query": {
"constant_score": {
"filter": {
"term": {
"num": 6
}
},
"boost": 1.2
}
}
}
返回:
```json { "took" : 7, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 2, "relation" : "eq" }, "max_score" : 1.2, "hits" : [ { "_index" : "class_1", "_type" : "_doc", "_id" : "h2Fg-4UBECmbBdQA6VLg", "_score" : 1.2, "_source" : { "name" : "b", "num" : 6 } }, { "_index" : "class_1", "_type" : "_doc", "_id" : "1", "_score" : 1.2, "_source" : { "name" : "l", "num" : 6 } } ] } }
```
查詢驗證 & 分析
驗證
es中通過/_validate/query路由來驗證查詢條件的正確性, 這裡要注意是驗證查詢條件是否準確
示例:
json
GET class_1/_validate/query?explain
{
"query": {
"bool": {
"must": [
{"match": {
"name": "apple"
}}
]
}
}
}
正常返回:
json
{
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"valid" : true,
"explanations" : [
{
"index" : "class_1",
"valid" : true,
"explanation" : "+name:apple"
}
]
}
將name欄位改為 name1再查詢:
```json { "_shards" : { "total" : 1, "successful" : 1, "failed" : 0 }, "valid" : true, "explanations" : [ { "index" : "class_1", "valid" : true, "explanation" : """+MatchNoDocsQuery("unmapped fields [name1]")""" } ] }
```
可以看到報了異常錯誤
分析
es中通過/_validate/query?explain路由來進行查詢分析
示例:
json
GET class_1/_validate/query?explain
{
"query": {
"bool": {
"must": [
{"match": {
"name": "apple so"
}}
]
}
}
}
返回:
json
{
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"valid" : true,
"explanations" : [
{
"index" : "class_1",
"valid" : true,
"explanation" : "+(name:apple name:so)"
}
]
}
可以看到"explanation" : "+(name:apple name:so)"，查詢的短語apple so被進行了分詞，分成了name:apple, name: so
排序
預設排序
在前面的幾個例子中，我們可以看到它的預設排序是按照_score降序，也就是匹配度高的比較靠前,但是_socre的計算是很佔用查詢效能的,這個不難理解。
當我們不需要進行_score計算，可以通過filter或constant_score來進行構建查詢條件
filter示例:
json
GET class_1/_search
{
"query": {
"bool": {
"filter": [
{"term": {
"num": 1
}}
]
}
}
}
返回:
```json { "took" : 5, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 3, "relation" : "eq" }, "max_score" : 0.0, "hits" : [ { "_index" : "class_1", "_type" : "_doc", "_id" : "b8fcCoYB090miyjed7YE", "_score" : 0.0, "_source" : { "name" : "I eat apple so haochi1~", "num" : 1 } }, { "_index" : "class_1", "_type" : "_doc", "_id" : "ccfcCoYB090miyjed7YE", "_score" : 0.0, "_source" : { "name" : "I eat apple so haochi3~", "num" : 1 } }, { "_index" : "class_1", "_type" : "_doc", "_id" : "cMfcCoYB090miyjed7YE", "_score" : 0.0, "_source" : { "name" : "I eat apple so zhen haochi2~", "num" : 1 } } ] } }
```
通過查詢結果我們發現score都為0.0了，說明沒有進行score計算
constant_score示例：
json
GET class_1/_search
{
"query": {
"constant_score": {
"filter": {
"term": {
"num": 1
}
},
"boost": 1.2
}
}
}
返回:
```json { "took" : 3, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 3, "relation" : "eq" }, "max_score" : 1.2, "hits" : [ { "_index" : "class_1", "_type" : "_doc", "_id" : "b8fcCoYB090miyjed7YE", "_score" : 1.2, "_source" : { "name" : "I eat apple so haochi1~", "num" : 1 } }, { "_index" : "class_1", "_type" : "_doc", "_id" : "ccfcCoYB090miyjed7YE", "_score" : 1.2, "_source" : { "name" : "I eat apple so haochi3~", "num" : 1 } }, { "_index" : "class_1", "_type" : "_doc", "_id" : "cMfcCoYB090miyjed7YE", "_score" : 1.2, "_source" : { "name" : "I eat apple so zhen haochi2~", "num" : 1 } } ] } }
```
可以看到，對應返回的分值，都是使用boost屬性指定的分值
自定義排序
自定義可以用於大部分場景，那麼es中怎麼進行自定義排序呢? es中使用sort引數來自定義排序順序，預設為升序，那麼降序怎麼操作呢？
- 升序
json
{"sort":["num"]}
- 降序,
desc代表降序
json
{"sort":[{"num":{"order":"desc"}}]}
tips
es中使用doc value列式儲存來實現欄位的排序功能text欄位預設不建立doc value，因此無法針對text欄位進行排序- 可以通過設定
text欄位屬性fielddata=true來開啟對text欄位的排序功能，但是不建議開啟，對text欄位排序及其消耗查詢效能且不符合需求
單欄位排序
json
GET class_1/_search
{
"sort": [
"num"
]
}
返回:
```json { "took" : 6, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 11, "relation" : "eq" }, "max_score" : null, "hits" : [ { "_index" : "class_1", "_type" : "_doc", "_id" : "b8fcCoYB090miyjed7YE", "_score" : null, "_source" : { "name" : "I eat apple so haochi1~", "num" : 1 }, "sort" : [ 1 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "ccfcCoYB090miyjed7YE", "_score" : null, "_source" : { "name" : "I eat apple so haochi3~", "num" : 1 }, "sort" : [ 1 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "cMfcCoYB090miyjed7YE", "_score" : null, "_source" : { "name" : "I eat apple so zhen haochi2~", "num" : 1 }, "sort" : [ 1 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "h2Fg-4UBECmbBdQA6VLg", "_score" : null, "_source" : { "name" : "b", "num" : 6 }, "sort" : [ 6 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "1", "_score" : null, "_source" : { "name" : "l", "num" : 6 }, "sort" : [ 6 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "3", "_score" : null, "_source" : { "num" : 9, "name" : "e", "age" : 9, "desc" : [ "hhhh" ] }, "sort" : [ 9 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "4", "_score" : null, "_source" : { "name" : "f", "age" : 10, "num" : 10 }, "sort" : [ 10 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "RWlfBIUBDuA8yW5cu9wu", "_score" : null, "_source" : { "name" : "一年級", "num" : 20 }, "sort" : [ 20 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "iGFt-4UBECmbBdQAnVJe", "_score" : null, "_source" : { "name" : "g", "age" : 8 }, "sort" : [ 9223372036854775807 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "iWFt-4UBECmbBdQAnVJg", "_score" : null, "_source" : { "name" : "h", "age" : 9 }, "sort" : [ 9223372036854775807 ] } ] } }
```
可以看到是按照num預設升序排序
再看下降序:
json
GET class_1/_search
{
"sort": [
{"num": {"order":"desc"}}
]
}
返回：
```json { "took" : 15, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 11, "relation" : "eq" }, "max_score" : null, "hits" : [ { "_index" : "class_1", "_type" : "_doc", "_id" : "RWlfBIUBDuA8yW5cu9wu", "_score" : null, "_source" : { "name" : "一年級", "num" : 20 }, "sort" : [ 20 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "4", "_score" : null, "_source" : { "name" : "f", "age" : 10, "num" : 10 }, "sort" : [ 10 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "3", "_score" : null, "_source" : { "num" : 9, "name" : "e", "age" : 9, "desc" : [ "hhhh" ] }, "sort" : [ 9 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "h2Fg-4UBECmbBdQA6VLg", "_score" : null, "_source" : { "name" : "b", "num" : 6 }, "sort" : [ 6 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "1", "_score" : null, "_source" : { "name" : "l", "num" : 6 }, "sort" : [ 6 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "b8fcCoYB090miyjed7YE", "_score" : null, "_source" : { "name" : "I eat apple so haochi1~", "num" : 1 }, "sort" : [ 1 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "ccfcCoYB090miyjed7YE", "_score" : null, "_source" : { "name" : "I eat apple so haochi3~", "num" : 1 }, "sort" : [ 1 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "cMfcCoYB090miyjed7YE", "_score" : null, "_source" : { "name" : "I eat apple so zhen haochi2~", "num" : 1 }, "sort" : [ 1 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "iGFt-4UBECmbBdQAnVJe", "_score" : null, "_source" : { "name" : "g", "age" : 8 }, "sort" : [ -9223372036854775808 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "iWFt-4UBECmbBdQAnVJg", "_score" : null, "_source" : { "name" : "h", "age" : 9 }, "sort" : [ -9223372036854775808 ] } ] } }
```
這下就降序排序了
多欄位
json
GET class_1/_search
{
"sort": [
"num", "age"
]
}
scroll分頁
還記得之前給大家講的from+size的分頁方式嗎，es中預設允許from+size的分頁的最大資料量為10000。當我們想要批量獲取更大的資料量時，使用from+size就會十分的耗費效能。
然而大部分應用場景下的資料量是極其龐大的，比如你要查詢某些系統日誌資料。es中可以使用/scorll路由來進行滾動分頁查詢，它類似於在查詢初始時間點建立了一個當前服務叢集的資料快照（包含每一個分片），並保留它一段時間。在時間超過了設定的過期時間以後，快照將在es空閒時被刪除。
需要注意的是，因為是進行快照查詢，因此在快照建立後資料的變更在本次的滾動查詢中，不可見
初始化快照 & 快照儲存10分鐘
查詢示例:
json
GET class_1/_search?scroll=10m
{
"query": {
"match_phrase": {
"name": "apple"
}
},
"size": 2
}
返回：
json
{
"_scroll_id" : "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAXoFjEwWkdOMkxLUTVPZEMzM01ZdHhPc1EAAAAAAAACABZjUy1CemQwQVFfU3BUeGs2OGk0R1Z3AAAAAAAAAgEWY1MtQnpkMEFRX1NwVHhrNjhpNEdWdw==",
"took" : 6,
"timed_out" : false,
"_shards" : {
"total" : 3,
"successful" : 3,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 0.752627,
"hits" : [
{
"_index" : "class_1",
"_type" : "_doc",
"_id" : "b8fcCoYB090miyjed7YE",
"_score" : 0.752627,
"_source" : {
"name" : "I eat apple so haochi1~",
"num" : 1
}
},
{
"_index" : "class_1",
"_type" : "_doc",
"_id" : "ccfcCoYB090miyjed7YE",
"_score" : 0.752627,
"_source" : {
"name" : "I eat apple so haochi3~",
"num" : 1
}
}
]
}
}
如圖，當前共返回2條資料，並且返回了一個快照ID，後續可以根據快照ID進行滾動查詢:
根據快照ID滾動查詢
json
GET /_search/scroll
{
"scroll": "10m",
"scroll_id" : "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAXoFjEwWkdOMkxLUTVPZEMzM01ZdHhPc1EAAAAAAAACABZjUy1CemQwQVFfU3BUeGs2OGk0R1Z3AAAAAAAAAgEWY1MtQnpkMEFRX1NwVHhrNjhpNEdWdw=="
}
返回:
json
{
"_scroll_id" : "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAXoFjEwWkdOMkxLUTVPZEMzM01ZdHhPc1EAAAAAAAACABZjUy1CemQwQVFfU3BUeGs2OGk0R1Z3AAAAAAAAAgEWY1MtQnpkMEFRX1NwVHhrNjhpNEdWdw==",
"took" : 6,
"timed_out" : false,
"_shards" : {
"total" : 3,
"successful" : 3,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 0.752627,
"hits" : [
{
"_index" : "class_1",
"_type" : "_doc",
"_id" : "cMfcCoYB090miyjed7YE",
"_score" : 0.7389809,
"_source" : {
"name" : "I eat apple so zhen haochi2~",
"num" : 1
}
}
]
}
}
在滾動一次:
```json { "_scroll_id" : "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAXoFjEwWkdOMkxLUTVPZEMzM01ZdHhPc1EAAAAAAAACABZjUy1CemQwQVFfU3BUeGs2OGk0R1Z3AAAAAAAAAgEWY1MtQnpkMEFRX1NwVHhrNjhpNEdWdw==", "took" : 1, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 3, "relation" : "eq" }, "max_score" : 0.752627, "hits" : [ ] } }
```
有的小夥伴可能不知道怎麼滾動的，因為後續滾動都是同一個scroll_id,其實通過結果，我們不難發現:
- 首先建立了一個10分鐘的
快照，規定了每次返回的資料量為2條,並且初始化的時候，返回了2條 - 通過
scroll_id進行滾動操作，返回了1條資料,原因是快照的資料量總共只有3條，初始化的時候返回了2條，所以現在只有1條 - 再次滾動的時候，發現返回了空，因為資料已經被查完了
結束語
本節就到此結束了，大家一定要多去練習。下節我們進入進階查詢部分內容 ~
本著把自己知道的都告訴大家，如果本文對您有所幫助，點贊+關注鼓勵一下唄~
相關文章
- 利用docker搭建es叢集
- 一起來學ElasticSearch(一)
- 一起來學ElasticSearch(二)
- 一起來學ElasticSearch(三)
- 一起來學ElasticSearch(四)
- 一起來學ElasticSearch(五)
- 一起來學ElasticSearch(六)
專案原始碼(原始碼已更新 歡迎star⭐️)
往期併發程式設計內容推薦
- Java多執行緒專題之執行緒與程序概述
- Java多執行緒專題之執行緒類和介面入門
- Java多執行緒專題之進階學習Thread(含原始碼分析)
- Java多執行緒專題之Callable、Future與FutureTask(含原始碼分析)
- 面試官: 有了解過執行緒組和執行緒優先順序嗎
- 面試官: 說一下執行緒的生命週期過程
- 面試官: 說一下執行緒間的通訊
- 面試官: 說一下Java的共享記憶體模型
- 面試官: 有了解過指令重排嗎，什麼是happens-before
- 面試官: 有了解過volatile關鍵字嗎 說說看
- 面試官: 有了解過Synchronized嗎 說說看
- Java多執行緒專題之Lock鎖的使用
- 面試官: 有了解過ReentrantLock的底層實現嗎？說說看
- 面試官: 有了解過CAS和原子操作嗎？說說看
- Java多執行緒專題之執行緒池的基本使用
- 面試官: 有了解過執行緒池的工作原理嗎？說說看
- 面試官: 執行緒池是如何做到執行緒複用的？有了解過嗎，說說看
- 面試官: 阻塞佇列有了解過嗎?說說看
- 面試官: 阻塞佇列的底層實現有了解過嗎? 說說看
- 面試官: 同步容器和併發容器有用過嗎? 說說看
- 面試官: CopyOnWrite容器有了解過嗎? 說說看
- 面試官: Semaphore在專案中有使用過嗎？說說看(原始碼剖析)
- 面試官: Exchanger在專案中有使用過嗎？說說看(原始碼剖析)
- 面試官: CountDownLatch有了解過嗎？說說看(原始碼剖析)
- 面試官: CyclicBarrier有了解過嗎？說說看(原始碼剖析)
- 面試官: Phaser有了解過嗎？說說看
- 面試官: Fork/Join 有了解過嗎？說說看（含原始碼分析）
- 面試官: Stream並行流有了解過嗎？說說看
部落格(閱讀體驗較佳)
-
地址: https://github.com/qiuChengleiy/java-thread-all.git
推薦 SpringBoot & SpringCloud (原始碼已更新 歡迎star⭐️)
-
地址: https://github.com/qiuChengleiy/springboot-all.git