StarRocks 運維工具 StarGo

語言: CN / TW / HK

注:本文主要內容均來源 StarRocks 官網 https://docs.starrocks.com/zh-cn/main/administration/stargo

StarGo 是一個用於管理多個 StarRocks 叢集的命令列工具。通過 StarGo,您可以使用簡單的命令列實現多叢集的部署、檢視、升級、啟動與停止等操作。

StarGo 使用 go 開發, github 地址 https://github.com/wangtianyi2004/starrocks-controller

部署 StarGo

在當前使用者路徑下下載並解壓 StarGo 二進位制安裝包。

wget https://raw.githubusercontent.com/wangtianyi2004/starrocks-controller/main/stargo-pkg.tar.gz
tar -xzvf stargo-pkg.tar.gz

安裝包包含以下檔案。

stargo:StarGo 二進位制檔案,無需安裝。

deploy-template.yaml:部署配置檔案模板。

repo.yaml:指定 StarRocks 安裝包下載庫的配置檔案。

StarGo 支援的版本參見: http://starrocks-thirdparty.oss-cn-zhangjiakou.aliyuncs.com/packageVersion.list

目前支援版本如下:

v2.0.1
v2.1.3
v2.1.6
v2.2.0
v2.2.1
v2.2.2
v2.2.3
v2.3.0

部署叢集

前提條件

  • 待部署叢集至少需要一箇中控機節點和三個部署機節點,所有節點可以混合部署於同一臺機器。
  • 中控機上需部署 StarGo。
  • 中控機與部署機間需建立 SSH 互信。

以下示例建立了中控機 sr-dev@r0 與部署機 starrocks@r1、starrocks@r2 以及 starrocks@r3 間的 SSH 互信。

建立 sr-dev@r0 到 starrocks@r1、r2、r3 的 ssh 互信。

[sr-dev@r0 ~]$ ssh-keygen
[sr-dev@r0 ~]$ ssh-copy-id starrocks@r1
[sr-dev@r0 ~]$ ssh-copy-id starrocks@r2
[sr-dev@r0 ~]$ ssh-copy-id starrocks@r3

## 驗證 sr-dev@r0 到 starrocks@r1、r2、r3 的 ssh 互信。
[sr-dev@r0 ~]$ ssh starrocks@r1 date
[sr-dev@r0 ~]$ ssh starrocks@r2 date
[sr-dev@r0 ~]$ ssh starrocks@r3 date

建立配置檔案

根據以下 YAML 模板,建立部署 StarRocks 叢集的拓撲檔案。具體配置項參考引數配置。

[starrocks@bigdata11 stargo-pkg]$ cat deploy-star3.yaml 
global:
    user: "starrocks"
    ssh_port: 22
fe_servers:
  - host: xx.xx.xx.229
    ssh_port: 22
    http_port: 38030
    rpc_port: 39020
    query_port: 39030
    edit_log_port: 39010
    deploy_dir: /opt/StarRocks1/fe
    meta_dir: /data/starrocks1/fe/meta
    log_dir: /data/starrocks1/fe/log
    priority_networks: xx.xx.xx.229
    config:
      sys_log_level: "INFO"
be_servers:
  - host: xx.xx.xx.229
    ssh_port: 22
    be_port: 39060
    webserver_port: 38040
    heartbeat_service_port: 39050
    brpc_port: 38060
    deploy_dir : /opt/StarRocks1/be
    storage_dir: /data/starrocks1/be/storage
    log_dir: /data/starrocks1/be/log
    priority_networks: xx.xx.xx.229/24
    config:
      create_tablet_worker_count: 3
  • 注:伺服器上已經有一套 StarRocks,再部署一套單機的,修改了下程式、埠

建立部署目錄(可選)

如果您在配置檔案中設定的部署路徑不存在,且您有建立該路徑的許可權,StarGo 將根據配置檔案自動建立部署目錄。如果路徑已存在,請確保您有在該路徑下擁有寫入的許可權。您也可以通過以下命令,在各部署節點分別建立部署路徑。

  • 在 FE 節點安裝目錄下上建立 meta 路徑。
mkdir -p /opt/StarRocks1/be
  • 在 BE 節點安裝目錄下上建立 storage 路徑。
mkdir -p /data/starrocks1/be/storage
  • 注意: 請確保以上建立的路徑與配置檔案中的 meta_dir 和 storage_dir 相同。

部署 StarRocks

通過以下命令部署 StarRocks 叢集。

./stargo cluster deploy <cluster_name> <version> <topology_file>
引數 描述
cluster_name 建立的叢集名
version StarRocks 的版本
topology_file 配置檔名

建立成功後,叢集將會自動啟動。當返回 beStatus 和feStatus 為 true 時,叢集部署啟動成功。

示例:

[starrocks@bigdata11 stargo-pkg]$ ./stargo cluster deploy star3 v2.2.2 deploy-star3.yaml
[20220812-113235  OUTPUT] Deploy cluster [clusterName = star3, clusterVersion = v2.2.2, metaFile = deploy-star3.yaml]

[20220812-113236  OUTPUT] PRE CHECK DEPLOY ENV:
PreCheck FE:
server id                  ssh auth         meta dir                        deploy dir                      http port        rpc port         query port       edit log port    open files count
--------------------       ---------------  ------------------------------  ------------------------------  ---------------  ---------------  ---------------  ---------------  ---------------
xx.xx.xx.229:39010         PASS             PASS                            PASS                            PASS             PASS             PASS             PASS             PASS           

PreCheck BE:
server id                  ssh auth         storage dir                     deploy dir                      webSer port      heartbeat port   brpc port        be port          open files count
--------------------       ---------------  ------------------------------  ------------------------------  ---------------  ---------------  ---------------  ---------------  ---------------
xx.xx.xx.229:39060         PASS             PASS                            PASS                            PASS             PASS             PASS             PASS             PASS           


[20220812-113236  OUTPUT] PreCheck successfully. RESPECT
[20220812-113236  OUTPUT] Create the deploy folder ...
[20220812-113237  OUTPUT] Download StarRocks package & jdk ...
[20220812-113322    INFO] The file starrocks-2.2.2-quickstart.tar.gz [1695364308] download successfully
[20220812-113322  OUTPUT] Download done.
[20220812-113322  OUTPUT] Decompress StarRocks pakcage & jdk ...
[20220812-113325    INFO] The tar file /home/starrocks/.stargo/download/starrocks-2.2.2-quickstart.tar.gz has been decompressed under /home/starrocks/.stargo/download
[20220812-113349    INFO] The tar file /home/starrocks/.stargo/download/StarRocks-2.2.2.tar.gz has been decompressed under /home/starrocks/.stargo/download
[20220812-113354    INFO] The tar file /home/starrocks/.stargo/download/jdk-8u301-linux-x64.tar.gz has been decompressed under /home/starrocks/.stargo/download
[20220812-113354  OUTPUT] Distribute FE Dir ...
[20220812-113402    INFO] Upload dir feSourceDir = [/home/starrocks/.stargo/download/StarRocks-2.2.2/fe] to feTargetDir = [/opt/StarRocks1/fe] on FeHost = [xx.xx.xx.229]
[20220812-113406    INFO] Upload dir JDKSourceDir = [/home/starrocks/.stargo/download/jdk1.8.0_301] to JDKTargetDir = [/opt/StarRocks1/fe/jdk] on FeHost = [xx.xx.xx.229]
[20220812-113406    INFO] Modify JAVA_HOME: host = [xx.xx.xx.229], filePath = [/opt/StarRocks1/fe/bin/start_fe.sh]
[20220812-113406  OUTPUT] Distribute BE Dir ...
[20220812-113417    INFO] Upload dir BeSourceDir = [/home/starrocks/.stargo/download/StarRocks-2.2.2/be] to BeTargetDir = [/opt/StarRocks1/be] on BeHost = [xx.xx.xx.229]
[20220812-113417  OUTPUT] Modify configuration for FE nodes & BE nodes ...
############################################# START FE CLUSTER #############################################
############################################# START FE CLUSTER #############################################
[20220812-113417    INFO] Starting leader FE node [host = xx.xx.xx.229, editLogPort = 39010]
[20220812-113438    WARN] The FE node doesn't start, wait for 10s [FeHost = xx.xx.xx.229, FeQueryPort = 39030, error = Process exited with status 1]
[20220812-113438    INFO] Starting leader FE node [host = xx.xx.xx.229, editLogPort = 39010]
[20220812-113454    INFO] The FE node start succefully [host = xx.xx.xx.229, queryPort = 39030]
[20220812-113454    INFO] List all FE status:
                                        feHost = xx.xx.xx.229        feQueryPort = 39030     feStatus = true

############################################# START BE CLUSTER #############################################
############################################# START BE CLUSTER #############################################
[20220812-113454    INFO] Starting BE node [BeHost = xx.xx.xx.229 HeartbeatServicePort = 39050]
[20220812-113515    INFO] The BE node start succefully [host = xx.xx.xx.229, heartbeatServicePort = 39050]
[20220812-113515  OUTPUT] List all BE status:
                                        beHost = xx.xx.xx.229        beHeartbeatServicePort = 39050	beStatus = true

如果執行檢查失敗,會有相應提示,按提示操作即可

[starrocks@bigdata12 stargo-pkg]$ ./stargo cluster deploy star3 v2.2.2 deploy-star3.yaml 
[20220812-102653  OUTPUT] Deploy cluster [clusterName = star3, clusterVersion = v2.2.2, metaFile = deploy-star3.yaml]

[20220812-102658  OUTPUT] PRE CHECK DEPLOY ENV:
PreCheck FE:
server id                  ssh auth         meta dir                        deploy dir                      http port        rpc port         query port       edit log port    open files count
--------------------       ---------------  ------------------------------  ------------------------------  ---------------  ---------------  ---------------  ---------------  ---------------
xx.xx.xx.228:39010         PASS             FAILED: Priv failed             FAILED: Priv failed             PASS             PASS             PASS             PASS             PASS           

PreCheck BE:
server id                  ssh auth         storage dir                     deploy dir                      webSer port      heartbeat port   brpc port        be port          open files count
--------------------       ---------------  ------------------------------  ------------------------------  ---------------  ---------------  ---------------  ---------------  ---------------
xx.xx.xx.228:39060         PASS             FAILED: Dir exist/Priv failed   FAILED: Dir exist/Priv failed   PASS             PASS             PASS             PASS             PASS           
xx.xx.xx.229:39060         PASS             FAILED: Dir exist/Priv failed   FAILED: Dir exist/Priv failed   PASS             PASS             PASS             PASS             PASS           
xx.xx.xx.230:39060         PASS             FAILED: Dir exist               FAILED: Dir exist               PASS             PASS             PASS             PASS             PASS           


[20220812-102658   ERROR] Please use bellowing promption to fix the issue for FE servers:
Detect the FE META FOLDER exist or no privilege. Use bellowing command to check or fix the issue:
  [Host = xx.xx.xx.228]  chown -R starrocks /data/starrocks1/fe/meta
Detect the FE DEPLOY FOLDER exist or no privilege. Use bellowing command to check or fix the issue:
  [Host = xx.xx.xx.228]  chown -R starrocks /opt/StarRocks1/fe

[20220812-102658   ERROR] Please use bellowing promption to fix the issue for BE servers:
Detect the BE STORAGE FOLDER exist or no privilege. Use bellowing command to check or fix the issue:
  [Host = xx.xx.xx.228]  mkdir /data/starrocks1/be/storage.bak && mv /data/starrocks1/be/storage/* /data/starrocks1/be/storage.bak/ && chown -R starrocks /data/starrocks1/be/storage
  [Host = xx.xx.xx.229]  mkdir /data/starrocks1/be/storage.bak && mv /data/starrocks1/be/storage/* /data/starrocks1/be/storage.bak/ && chown -R starrocks /data/starrocks1/be/storage
  [Host = xx.xx.xx.230]  mkdir /data/starrocks1/be/storage.bak && mv /data/starrocks1/be/storage/* /data/starrocks1/be/storage.bak/
Detect the BE DEPLOY FOLDER exist or no privilege. Use bellowing command to check or fix the issue:
  [Host = xx.xx.xx.228]  mkdir /opt/StarRocks1/be.bak && mv /opt/StarRocks1/be/* /opt/StarRocks1/be.bak/ && chown -R starrocks /opt/StarRocks1/be
  [Host = xx.xx.xx.229]  mkdir /opt/StarRocks1/be.bak && mv /opt/StarRocks1/be/* /opt/StarRocks1/be.bak/ && chown -R starrocks /opt/StarRocks1/be
  [Host = xx.xx.xx.230]  mkdir /opt/StarRocks1/be.bak && mv /opt/StarRocks1/be/* /opt/StarRocks1/be.bak/

[20220812-102658   ERROR] PreCheck failed.

檢視叢集資訊

[starrocks@bigdata11 stargo-pkg]$ ./stargo cluster list
[20220812-115246  OUTPUT] List all clusters
ClusterName      Version     User        CreateDate                 MetaPath                                                      PrivateKey                                        
---------------  ----------  ----------  -------------------------  ------------------------------------------------------------  --------------------------------------------------
star3            v2.2.2      starrocks   2022-08-12 11:34:17        /home/starrocks/.stargo/cluster/star3                         /home/starrocks/.ssh/id_rsa

檢視指定叢集資訊

[starrocks@bigdata11 stargo-pkg]$ ./stargo cluster display star3
[20220812-143023  OUTPUT] Display cluster [clusterName = star3]
clusterName = star3
clusterVerison = v2.2.2
ID                          ROLE    HOST                  PORT             STAT        DATADIR                                             DEPLOYDIR                                         
--------------------------  ------  --------------------  ---------------  ----------  --------------------------------------------------  --------------------------------------------------
xx.xx.xx.229:39010          FE      xx.xx.xx.229          39010/39030      UP/L        /opt/StarRocks1/fe                                  /data/starrocks1/fe/meta                          
xx.xx.xx.229:39060          BE      xx.xx.xx.229          39060/39050      UP          /opt/StarRocks1/be                                  /data/starrocks1/be/storage                       
[starrocks@bigdata11 stargo-pkg]$

升級版本

starrocks 2.2.0 升級到 2.3.0

[starrocks@bigdata11 stargo-pkg]$ ./stargo cluster upgrade star3 v2.3.0
[20220812-143227  OUTPUT] Upgrade cluster. [ClusterName = star3, TargetVersion = v2.3.0]
[20220812-143227  OUTPUT] Upgrade StarRocks Cluster star3, from version v2.2.2 to version v2.3.0
[20220812-143227  OUTPUT] Download StarRocks package & jdk ...
[20220812-143340    INFO] The file starrocks-2.3.0-quickstart.tar.gz [1726845129] download successfully
[20220812-143340  OUTPUT] Download done.
[20220812-143340  OUTPUT] Decompress StarRocks pakcage & jdk ...
[20220812-143346    INFO] The tar file /home/starrocks/.stargo/download/starrocks-2.3.0-quickstart.tar.gz has been decompressed under /home/starrocks/.stargo/download
[20220812-143411    INFO] The tar file /home/starrocks/.stargo/download/StarRocks-2.3.0.tar.gz has been decompressed under /home/starrocks/.stargo/download
[20220812-143411    INFO] The tar file /home/starrocks/.stargo/download/jdk-8u301-linux-x64.tar.gz has been decompressed under /home/starrocks/.stargo/download
[20220812-143413  OUTPUT] Starting upgrade BE node. [beId = 0]
[20220812-143413    INFO] upgrade be node - backup be lib. [host = xx.xx.xx.229, sourceDir = /opt/StarRocks1/be/lib, targetDir = /opt/StarRocks1/be/lib.bak-20220812143413]
[20220812-143423    INFO] upgrade be node - upload new be lib. [host = xx.xx.xx.229, sourceDir = /home/starrocks/.stargo/download/StarRocks-2.3.0/be/lib, targetDir = /opt/StarRocks1/be/lib]
[20220812-143423    INFO] Waiting for stoping BE node [BeHost = xx.xx.xx.229]
[20220812-143429    INFO] upgrade be node - stop be node. [host = xx.xx.xx.229, beDeployDir = /opt/StarRocks1/be]
[20220812-143430    INFO] upgrade be node - start be node. [host = xx.xx.xx.229, beDeployDir = /opt/StarRocks1/be]
[20220812-143443    INFO] upgrade be node - start be node. [host = xx.xx.xx.229, beDeployDir = /opt/StarRocks1/be]
[20220812-143443  OUTPUT] The Be node upgrade successfully. [beId = 0, currentVersion = v2.3.0-a9bdb09]
[20220812-143443  OUTPUT] Starting upgrade FE node. [feId = 0]
[20220812-143445    INFO] upgrade FE node - backup FE lib. [host = xx.xx.xx.229, sourceDir = /opt/StarRocks1/fe/lib, targetDir = /opt/StarRocks1/fe/lib.bak-20220812143443]
[20220812-143448    INFO] upgrade FE node - upload new FE lib. [host = xx.xx.xx.229, sourceDir = /home/starrocks/.stargo/download/StarRocks-2.3.0/fe/lib, targetDir = /opt/StarRocks1/fe/lib]
[20220812-143448    INFO] Waiting for stoping FE node [FeHost = xx.xx.xx.229]
[20220812-143451    INFO] upgrade FE node - stop FE node. [host = xx.xx.xx.229, feDeployDir = /opt/StarRocks1/fe]
[20220812-143452    INFO] upgrade FE node - start FE node. [host = xx.xx.xx.229, feDeployDir = /opt/StarRocks1/fe]
[20220812-143452   ERROR] Error in ping db [dbPath = root:@tcp(xx.xx.xx.229:39030)/], error = dial tcp xx.xx.xx.229:39030: connect: connection refused
[20220812-143502    INFO] upgrade FE node - start FE node. [host = xx.xx.xx.229, feDeployDir = /opt/StarRocks1/fe]
[20220812-143503   ERROR] The FE node upgrade failed.  [feId = 0, targetVersion = v2.3.0, currentVersion = v]
  • 注: 是有報錯,但是升級成功了

升級後檢視叢集

[20220812-143555  OUTPUT] List all clusters
ClusterName      Version     User        CreateDate                 MetaPath                                                      PrivateKey                                        
---------------  ----------  ----------  -------------------------  ------------------------------------------------------------  --------------------------------------------------
star3            v2.3.0      starrocks   2022-08-12 14:35:03        /home/starrocks/.stargo/cluster/star3                         /home/starrocks/.ssh/id_rsa

已知缺陷

  1. StarGo 不會部署 Broker
  2. StarGo 安裝包包含 jdk,不會替換安裝

歡迎關注Flink菜鳥公眾號,會不定期更新Flink(開發技術)相關的推文