一起來學ElasticSearch(七)
前言
目前正在出一個Es專題
系列教程, 篇幅會較多, 喜歡的話,給個關注❤️ ~
承接上文,本節把上節遺留的條件查詢
操作給大家講一下~
為了方便學習, 本節中所有示例沿用上節的索引。本文偏實戰一些,好了, 廢話不多說直接開整吧~
多條件組合查詢
bool
es
中使用bool
來控制多條件
查詢,bool
查詢支援以下引數:
must
:被查詢的資料必須滿足
當前條件mush_not
:被查詢的資料必須不滿足
當前條件should
:被查詢的資料應該滿足
當前條件。should
查詢被用於修正查詢結果的評分。需要注意的是,如果組合查詢中沒有must
,那麼被查詢的資料至少要匹配一條should
。如果有must
語句,那麼就無須匹配should
,should
將完全用於修正查詢結果的評分filter
:被查詢的資料必須滿足
當前條件,但是filter
操作不涉及查詢結果評分。僅用於條件過濾
下面通過一個例子來看下如何使用:
json
GET class_1/_search
{
"query": {
"bool": {
"must": [
{"match": {
"name": "apple"
}}
],
"must_not": [
{"term": {
"num": {
"value": "5"
}
}}
],
"should": [
{"match": {
"name": "k"
}}
],"filter": [
{"range": {
"num": {
"gte": 0,
"lte": 10
}
}}
]
}
}
}
結果返回:
```json { "took" : 9, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 3, "relation" : "eq" }, "max_score" : 0.752627, "hits" : [ { "_index" : "class_1", "_type" : "_doc", "_id" : "b8fcCoYB090miyjed7YE", "_score" : 0.752627, "_source" : { "name" : "I eat apple so haochi1~", "num" : 1 } }, { "_index" : "class_1", "_type" : "_doc", "_id" : "ccfcCoYB090miyjed7YE", "_score" : 0.752627, "_source" : { "name" : "I eat apple so haochi3~", "num" : 1 } }, { "_index" : "class_1", "_type" : "_doc", "_id" : "cMfcCoYB090miyjed7YE", "_score" : 0.7389809, "_source" : { "name" : "I eat apple so zhen haochi2~", "num" : 1 } } ] } }
```
constant_score
constant_score
查詢可以通過boost
指定一個固定的評分,通常來說,constant_score
的作用是代替一個只有filter
的bool
查詢
下面看具體使用:
json
GET class_1/_search
{
"query": {
"constant_score": {
"filter": {
"term": {
"num": 6
}
},
"boost": 1.2
}
}
}
返回:
```json { "took" : 7, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 2, "relation" : "eq" }, "max_score" : 1.2, "hits" : [ { "_index" : "class_1", "_type" : "_doc", "_id" : "h2Fg-4UBECmbBdQA6VLg", "_score" : 1.2, "_source" : { "name" : "b", "num" : 6 } }, { "_index" : "class_1", "_type" : "_doc", "_id" : "1", "_score" : 1.2, "_source" : { "name" : "l", "num" : 6 } } ] } }
```
查詢驗證 & 分析
驗證
es
中通過/_validate/query
路由來驗證查詢條件的正確性
, 這裡要注意是驗證查詢條件
是否準確
示例:
json
GET class_1/_validate/query?explain
{
"query": {
"bool": {
"must": [
{"match": {
"name": "apple"
}}
]
}
}
}
正常返回:
json
{
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"valid" : true,
"explanations" : [
{
"index" : "class_1",
"valid" : true,
"explanation" : "+name:apple"
}
]
}
將name
欄位改為 name1
再查詢:
```json { "_shards" : { "total" : 1, "successful" : 1, "failed" : 0 }, "valid" : true, "explanations" : [ { "index" : "class_1", "valid" : true, "explanation" : """+MatchNoDocsQuery("unmapped fields [name1]")""" } ] }
```
可以看到報了異常錯誤
分析
es
中通過/_validate/query?explain
路由來進行查詢分析
示例:
json
GET class_1/_validate/query?explain
{
"query": {
"bool": {
"must": [
{"match": {
"name": "apple so"
}}
]
}
}
}
返回:
json
{
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"valid" : true,
"explanations" : [
{
"index" : "class_1",
"valid" : true,
"explanation" : "+(name:apple name:so)"
}
]
}
可以看到"explanation" : "+(name:apple name:so)"
,查詢的短語apple so
被進行了分詞,分成了name:apple
, name: so
排序
預設排序
在前面的幾個例子中,我們可以看到它的預設排序是按照_score降序
,也就是匹配度高的比較靠前
,但是_socre
的計算是很佔用查詢效能的,這個不難理解。
當我們不需要進行_score計算
,可以通過filter
或constant_score
來進行構建查詢條件
filter
示例:
json
GET class_1/_search
{
"query": {
"bool": {
"filter": [
{"term": {
"num": 1
}}
]
}
}
}
返回:
```json { "took" : 5, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 3, "relation" : "eq" }, "max_score" : 0.0, "hits" : [ { "_index" : "class_1", "_type" : "_doc", "_id" : "b8fcCoYB090miyjed7YE", "_score" : 0.0, "_source" : { "name" : "I eat apple so haochi1~", "num" : 1 } }, { "_index" : "class_1", "_type" : "_doc", "_id" : "ccfcCoYB090miyjed7YE", "_score" : 0.0, "_source" : { "name" : "I eat apple so haochi3~", "num" : 1 } }, { "_index" : "class_1", "_type" : "_doc", "_id" : "cMfcCoYB090miyjed7YE", "_score" : 0.0, "_source" : { "name" : "I eat apple so zhen haochi2~", "num" : 1 } } ] } }
```
通過查詢結果我們發現score
都為0.0
了,說明沒有進行score
計算
constant_score
示例:
json
GET class_1/_search
{
"query": {
"constant_score": {
"filter": {
"term": {
"num": 1
}
},
"boost": 1.2
}
}
}
返回:
```json { "took" : 3, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 3, "relation" : "eq" }, "max_score" : 1.2, "hits" : [ { "_index" : "class_1", "_type" : "_doc", "_id" : "b8fcCoYB090miyjed7YE", "_score" : 1.2, "_source" : { "name" : "I eat apple so haochi1~", "num" : 1 } }, { "_index" : "class_1", "_type" : "_doc", "_id" : "ccfcCoYB090miyjed7YE", "_score" : 1.2, "_source" : { "name" : "I eat apple so haochi3~", "num" : 1 } }, { "_index" : "class_1", "_type" : "_doc", "_id" : "cMfcCoYB090miyjed7YE", "_score" : 1.2, "_source" : { "name" : "I eat apple so zhen haochi2~", "num" : 1 } } ] } }
```
可以看到,對應返回的分值,都是使用boost
屬性指定的分值
自定義排序
自定義可以用於大部分場景,那麼es
中怎麼進行自定義排序呢? es
中使用sort
引數來自定義排序順序,預設為升序
,那麼降序
怎麼操作呢?
- 升序
json
{"sort":["num"]}
- 降序,
desc
代表降序
json
{"sort":[{"num":{"order":"desc"}}]}
tips
es
中使用doc value
列式儲存來實現欄位的排序功能text
欄位預設不建立doc value
,因此無法針對text
欄位進行排序- 可以通過設定
text
欄位屬性fielddata=true
來開啟對text
欄位的排序功能,但是不建議開啟,對text
欄位排序及其消耗查詢效能且不符合需求
單欄位排序
json
GET class_1/_search
{
"sort": [
"num"
]
}
返回:
```json { "took" : 6, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 11, "relation" : "eq" }, "max_score" : null, "hits" : [ { "_index" : "class_1", "_type" : "_doc", "_id" : "b8fcCoYB090miyjed7YE", "_score" : null, "_source" : { "name" : "I eat apple so haochi1~", "num" : 1 }, "sort" : [ 1 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "ccfcCoYB090miyjed7YE", "_score" : null, "_source" : { "name" : "I eat apple so haochi3~", "num" : 1 }, "sort" : [ 1 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "cMfcCoYB090miyjed7YE", "_score" : null, "_source" : { "name" : "I eat apple so zhen haochi2~", "num" : 1 }, "sort" : [ 1 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "h2Fg-4UBECmbBdQA6VLg", "_score" : null, "_source" : { "name" : "b", "num" : 6 }, "sort" : [ 6 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "1", "_score" : null, "_source" : { "name" : "l", "num" : 6 }, "sort" : [ 6 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "3", "_score" : null, "_source" : { "num" : 9, "name" : "e", "age" : 9, "desc" : [ "hhhh" ] }, "sort" : [ 9 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "4", "_score" : null, "_source" : { "name" : "f", "age" : 10, "num" : 10 }, "sort" : [ 10 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "RWlfBIUBDuA8yW5cu9wu", "_score" : null, "_source" : { "name" : "一年級", "num" : 20 }, "sort" : [ 20 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "iGFt-4UBECmbBdQAnVJe", "_score" : null, "_source" : { "name" : "g", "age" : 8 }, "sort" : [ 9223372036854775807 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "iWFt-4UBECmbBdQAnVJg", "_score" : null, "_source" : { "name" : "h", "age" : 9 }, "sort" : [ 9223372036854775807 ] } ] } }
```
可以看到是按照num
預設升序排序
再看下降序:
json
GET class_1/_search
{
"sort": [
{"num": {"order":"desc"}}
]
}
返回:
```json { "took" : 15, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 11, "relation" : "eq" }, "max_score" : null, "hits" : [ { "_index" : "class_1", "_type" : "_doc", "_id" : "RWlfBIUBDuA8yW5cu9wu", "_score" : null, "_source" : { "name" : "一年級", "num" : 20 }, "sort" : [ 20 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "4", "_score" : null, "_source" : { "name" : "f", "age" : 10, "num" : 10 }, "sort" : [ 10 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "3", "_score" : null, "_source" : { "num" : 9, "name" : "e", "age" : 9, "desc" : [ "hhhh" ] }, "sort" : [ 9 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "h2Fg-4UBECmbBdQA6VLg", "_score" : null, "_source" : { "name" : "b", "num" : 6 }, "sort" : [ 6 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "1", "_score" : null, "_source" : { "name" : "l", "num" : 6 }, "sort" : [ 6 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "b8fcCoYB090miyjed7YE", "_score" : null, "_source" : { "name" : "I eat apple so haochi1~", "num" : 1 }, "sort" : [ 1 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "ccfcCoYB090miyjed7YE", "_score" : null, "_source" : { "name" : "I eat apple so haochi3~", "num" : 1 }, "sort" : [ 1 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "cMfcCoYB090miyjed7YE", "_score" : null, "_source" : { "name" : "I eat apple so zhen haochi2~", "num" : 1 }, "sort" : [ 1 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "iGFt-4UBECmbBdQAnVJe", "_score" : null, "_source" : { "name" : "g", "age" : 8 }, "sort" : [ -9223372036854775808 ] }, { "_index" : "class_1", "_type" : "_doc", "_id" : "iWFt-4UBECmbBdQAnVJg", "_score" : null, "_source" : { "name" : "h", "age" : 9 }, "sort" : [ -9223372036854775808 ] } ] } }
```
這下就降序
排序了
多欄位
json
GET class_1/_search
{
"sort": [
"num", "age"
]
}
scroll分頁
還記得之前給大家講的from+size
的分頁方式嗎,es
中預設允許from+size
的分頁的最大資料量為10000
。當我們想要批量獲取更大的資料量時,使用from+size
就會十分的耗費效能。
然而大部分應用場景下的資料量是極其龐大的,比如你要查詢某些系統日誌資料。es
中可以使用/scorll
路由來進行滾動分頁查詢
,它類似於在查詢初始時間點建立了一個當前服務叢集的資料快照
(包含每一個分片),並保留它一段時間。在時間超過了設定的過期時間以後,快照將在es空閒時被刪除。
需要注意的是,因為是進行快照
查詢,因此在快照
建立後資料的變更在本次的滾動查詢中,不可見
初始化快照 & 快照儲存10分鐘
查詢示例:
json
GET class_1/_search?scroll=10m
{
"query": {
"match_phrase": {
"name": "apple"
}
},
"size": 2
}
返回:
json
{
"_scroll_id" : "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAXoFjEwWkdOMkxLUTVPZEMzM01ZdHhPc1EAAAAAAAACABZjUy1CemQwQVFfU3BUeGs2OGk0R1Z3AAAAAAAAAgEWY1MtQnpkMEFRX1NwVHhrNjhpNEdWdw==",
"took" : 6,
"timed_out" : false,
"_shards" : {
"total" : 3,
"successful" : 3,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 0.752627,
"hits" : [
{
"_index" : "class_1",
"_type" : "_doc",
"_id" : "b8fcCoYB090miyjed7YE",
"_score" : 0.752627,
"_source" : {
"name" : "I eat apple so haochi1~",
"num" : 1
}
},
{
"_index" : "class_1",
"_type" : "_doc",
"_id" : "ccfcCoYB090miyjed7YE",
"_score" : 0.752627,
"_source" : {
"name" : "I eat apple so haochi3~",
"num" : 1
}
}
]
}
}
如圖,當前共返回2
條資料,並且返回了一個快照ID
,後續可以根據快照ID
進行滾動
查詢:
根據快照ID滾動查詢
json
GET /_search/scroll
{
"scroll": "10m",
"scroll_id" : "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAXoFjEwWkdOMkxLUTVPZEMzM01ZdHhPc1EAAAAAAAACABZjUy1CemQwQVFfU3BUeGs2OGk0R1Z3AAAAAAAAAgEWY1MtQnpkMEFRX1NwVHhrNjhpNEdWdw=="
}
返回:
json
{
"_scroll_id" : "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAXoFjEwWkdOMkxLUTVPZEMzM01ZdHhPc1EAAAAAAAACABZjUy1CemQwQVFfU3BUeGs2OGk0R1Z3AAAAAAAAAgEWY1MtQnpkMEFRX1NwVHhrNjhpNEdWdw==",
"took" : 6,
"timed_out" : false,
"_shards" : {
"total" : 3,
"successful" : 3,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 0.752627,
"hits" : [
{
"_index" : "class_1",
"_type" : "_doc",
"_id" : "cMfcCoYB090miyjed7YE",
"_score" : 0.7389809,
"_source" : {
"name" : "I eat apple so zhen haochi2~",
"num" : 1
}
}
]
}
}
在滾動一次:
```json { "_scroll_id" : "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAXoFjEwWkdOMkxLUTVPZEMzM01ZdHhPc1EAAAAAAAACABZjUy1CemQwQVFfU3BUeGs2OGk0R1Z3AAAAAAAAAgEWY1MtQnpkMEFRX1NwVHhrNjhpNEdWdw==", "took" : 1, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 3, "relation" : "eq" }, "max_score" : 0.752627, "hits" : [ ] } }
```
有的小夥伴可能不知道怎麼滾動
的,因為後續滾動都是同一個scroll_id
,其實通過結果,我們不難發現:
- 首先建立了一個10分鐘的
快照
,規定了每次返回的資料量為2條
,並且初始化的時候,返回了2條 - 通過
scroll_id
進行滾動操作,返回了1條
資料,原因是快照的資料量總共只有3條
,初始化的時候返回了2條
,所以現在只有1條
- 再次滾動的時候,發現返回了空,因為資料已經被查完了
結束語
本節就到此結束了,大家一定要多去練習。下節我們進入進階查詢
部分內容 ~
本著把自己知道的都告訴大家,如果本文對您有所幫助,點贊+關注
鼓勵一下唄~
相關文章
- 利用docker搭建es叢集
- 一起來學ElasticSearch(一)
- 一起來學ElasticSearch(二)
- 一起來學ElasticSearch(三)
- 一起來學ElasticSearch(四)
- 一起來學ElasticSearch(五)
- 一起來學ElasticSearch(六)
專案原始碼(原始碼已更新 歡迎star⭐️)
往期併發程式設計內容推薦
- Java多執行緒專題之執行緒與程序概述
- Java多執行緒專題之執行緒類和介面入門
- Java多執行緒專題之進階學習Thread(含原始碼分析)
- Java多執行緒專題之Callable、Future與FutureTask(含原始碼分析)
- 面試官: 有了解過執行緒組和執行緒優先順序嗎
- 面試官: 說一下執行緒的生命週期過程
- 面試官: 說一下執行緒間的通訊
- 面試官: 說一下Java的共享記憶體模型
- 面試官: 有了解過指令重排嗎,什麼是happens-before
- 面試官: 有了解過volatile關鍵字嗎 說說看
- 面試官: 有了解過Synchronized嗎 說說看
- Java多執行緒專題之Lock鎖的使用
- 面試官: 有了解過ReentrantLock的底層實現嗎?說說看
- 面試官: 有了解過CAS和原子操作嗎?說說看
- Java多執行緒專題之執行緒池的基本使用
- 面試官: 有了解過執行緒池的工作原理嗎?說說看
- 面試官: 執行緒池是如何做到執行緒複用的?有了解過嗎,說說看
- 面試官: 阻塞佇列有了解過嗎?說說看
- 面試官: 阻塞佇列的底層實現有了解過嗎? 說說看
- 面試官: 同步容器和併發容器有用過嗎? 說說看
- 面試官: CopyOnWrite容器有了解過嗎? 說說看
- 面試官: Semaphore在專案中有使用過嗎?說說看(原始碼剖析)
- 面試官: Exchanger在專案中有使用過嗎?說說看(原始碼剖析)
- 面試官: CountDownLatch有了解過嗎?說說看(原始碼剖析)
- 面試官: CyclicBarrier有了解過嗎?說說看(原始碼剖析)
- 面試官: Phaser有了解過嗎?說說看
- 面試官: Fork/Join 有了解過嗎?說說看(含原始碼分析)
- 面試官: Stream並行流有了解過嗎?說說看
部落格(閱讀體驗較佳)
-
地址
: https://github.com/qiuChengleiy/java-thread-all.git
推薦 SpringBoot & SpringCloud (原始碼已更新 歡迎star⭐️)
-
地址
: https://github.com/qiuChengleiy/springboot-all.git