一背景

利用Gitlab CI实现基础设施编排自动化，用户后续针对基础设施的管理使用Gitlab完成，提交基础设施变更后，会出发pr进行Gitlab CI流水线执行，从而实现基础设施DevOPS流程。

二流程架构

2.1 架构图

2.2 流程

运维研发编写目标云的基于Terraform的资源清单文件，同事项目内管理Gitlab CI流程，在K8s不同NS下注册有对应的runner，当在不同分支下可以触发不同ns下的CI流程。

开发或运维人员提交代码。
部署在对应名称空间下的runner执行流程，创建运行单个Stage的POD来运行Terraform对应命令，例如init/fmt/play/apply等。
如果要对云上资源进行变更，修改代码，再次提交pr，出发更新流水线。
如果需要销毁，根据CI文件配置提交BUILD为destroy，触发云上销毁动作。

三预置条件

Gitlab 服务器
注册有项目的gitlab-runner
K8s集群
腾讯云AK账号

四配置

4.1 Gitlab CI配置

4.1.1 .gitlab.yaml

```yaml variables: # PHASE: BUILD|DESTROY PHASE: DESTROY

squiduser:[email protected]:3128

REGION: "ap-guangzhou" PLAN_JSON: plan.json BACKEND_CONF: "backend_oss.conf"

before_script:

- apk add --no-cache curl git jq

apk add --no-cache jq
export http_proxy=${SQUID_PROXY}
export https_proxy=${SQUID_PROXY}
export TENCENTCLOUD_SECRET_KEY=${TENCENTCLOUD_SECRET_KEY}
export TENCENTCLOUD_SECRET_ID=${TENCENTCLOUD_SECRET_ID}
export TF_REGISTRY_CLIENT_TIMEOUT=120000
export CHECKPOINT_TIMEOUT=500000
export TF_REGISTRY_DISCOVERY_RETRY=5
alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'"

配置缓存

cache: paths: - ${CI_PROJECT_DIR}/.terraform/*

stages: - init - validate - plan - deploy

Init: image: name: hashicorp/terraform:0.14.0 entrypoint: [""] stage: init retry: max: 2 when: - script_failure tags: - gitlab-runner-k8s-new script: - terraform version - terraform init -backend-config=${BACKEND_CONF} only: - dev

Validate: image: name: hashicorp/terraform:0.14.0 entrypoint: [""] stage: validate tags: - gitlab-runner-k8s-new retry: 2 script: - terraform init -backend-config=${BACKEND_CONF} - terraform validate - terraform fmt -check -recursive || echo 0 cache: paths: - ${CI_PROJECT_DIR}/.terraform/* policy: pull allow_failure: true

Plan: image: name: hashicorp/terraform:0.14.0 entrypoint: [""] stage: plan retry: 2 tags: - gitlab-runner-k8s-new artifacts: paths: - plan.bin - app_config.zip expire_in: 2 week script: - terraform init -backend-config=${BACKEND_CONF} - terraform plan -input=false -out=plan.bin -var region=${REGION} - terraform show --json "plan.bin" | convert_report > ${PLAN_JSON} - cat ${PLAN_JSON} only: variables: - $PHASE == "BUILD"

Apply: image: name: hashicorp/terraform:0.14.0 entrypoint: [""] when: manual stage: deploy retry: 2 tags: - gitlab-runner-k8s-new script: - terraform init -backend-config=${BACKEND_CONF} - terraform apply -auto-approve -input=false plan.bin only: variables: - $PHASE == "BUILD" environment: name: snunv

Destroy: image: name: hashicorp/terraform:0.14.0 entrypoint: [""] stage: deploy retry: 2 tags: - gitlab-runner-k8s-new script: - terraform init -backend-config=${BACKEND_CONF} - terraform destroy -auto-approve -var region=${REGION} only: variables: - $PHASE == "DESTROY" ```

4.1.2 环境配置

利用Gitlab CI/CD的Environment进行环境管理。

4.2 Terraform资源

```shell provider "tencentcloud" { region = var.region }

terraform { required_providers { tencentcloud = { source = "registry.terraform.io/tencentcloudstack/tencentcloud" version = ">=1.61.5" } } backend "cos" {} }

输入变量

variable "region" { type = string }

再次仅为一个查询示例

data "tencentcloud_instances" "cvm" { }

输出

output "result" { value = { cvm_result = { for k, v in data.tencentcloud_instances.cvm : k => v }, count = data.tencentcloud_instances.cvm.instance_list[*] } } ```

为了terraform后端backend安全，将其存储为单独文件，可不同分支或环境进行修改

shell region = "ap-beijing" bucket = "tfproject-1253329830" prefix = "samxxxxitlab/dexxxxxt"