编译mips64le架构的consul

语言: CN / TW / HK

时间 2021-07-16 14:42:52 Zhangguanzhang

git clone http://github.com/hashicorp/consul.git
cd consul

线上使用的是 v1.8 版本，这里我以 v1.8.14 (2021/07/19 发布的)搞的。

git checkout v1.8.14

准备工作

拉取需要的镜像。貌似 golang 1.16 更好的支持 mips64 了，所以条件允许的话，这里可以下改下 golang 的版本试试

$ head -n2 build-support/docker/Build-Go.dockerfile
ARG GOLANG_VERSION=1.14.11
FROM golang:${GOLANG_VERSION}
$ docker pull golang:1.14.11

开始编译

相关变量可以查看 GNUmakefile ，实际会在容器里运行 build-support/functions/20-build.sh 里的 function build_consul ：

local container_id=$(docker create -it \
   ${volume_mount} \
   -e CGO_ENABLED=0 \
   -e GOLDFLAGS="${GOLDFLAGS}" \
   -e GOTAGS="${GOTAGS}" \
   ${image_name} \
   ./build-support/scripts/build-local.sh -o "${XC_OS}" -a "${XC_ARCH}")

而在容器里执行 build-support/scripts/build-local.sh -o "${XC_OS}" -a "${XC_ARCH}" 会执行 build-support/functions/20-build.sh 里的 function build_consul_local ，主要编译是下面的这行：

debug_run env CGO_ENABLED=0 GOOS=${os} GOARCH=${arch} go install -ldflags "${GOLDFLAGS}" -tags "${GOTAGS}" && cp "${MAIN_GOPATH}/bin/${GOBIN_EXTRA}${binname}" "${outdir}/${binname}"

开始编译

export DOCKER_BUILD_QUIET=0
export XC_OS=linux XC_ARCH=mips64le
make consul-docker


==> Building Consul - OSes: linux, Architectures: mips64le
Building sequentially with go install
--->   linux/mips64le
# github.com/boltdb/bolt
/go/pkg/mod/github.com/boltdb/[email protected]/bolt_unix.go:62:15: undefined: maxMapSize
/go/pkg/mod/github.com/boltdb/[email protected]/bucket.go:135:15: undefined: brokenUnaligned
/go/pkg/mod/github.com/boltdb/[email protected]/db.go:101:13: undefined: maxMapSize
/go/pkg/mod/github.com/boltdb/[email protected]/db.go:317:12: undefined: maxMapSize
/go/pkg/mod/github.com/boltdb/[email protected]/db.go:335:10: undefined: maxMapSize
/go/pkg/mod/github.com/boltdb/[email protected]/db.go:336:8: undefined: maxMapSize
/go/pkg/mod/github.com/boltdb/[email protected]/freelist.go:169:19: undefined: maxAllocSize
/go/pkg/mod/github.com/boltdb/[email protected]/freelist.go:176:14: undefined: maxAllocSize
/go/pkg/mod/github.com/boltdb/[email protected]/freelist.go:204:17: undefined: maxAllocSize
/go/pkg/mod/github.com/boltdb/[email protected]/freelist.go:207:7: undefined: maxAllocSize
/go/pkg/mod/github.com/boltdb/[email protected]/freelist.go:207:7: too many errors
ERROR: Failed to build Consul for linux/mips64le
make: *** [consul-docker] Error 1

编译报错，是因为 boltdb 的库现在 archive 了。而且相关文件用 golang 的 build tag 区分架构了，没有预置 mips64le 的，搜了几个 issue 后搞了下编译还是失败了，仔细看上面的 path，没有读取 vendor 目录，得改编译容器里的 mod 目录下的 boltdb 文件。所以得 hack 下。

hack

先产生临时文件 /tmp/hack_consul.sh

cat > /tmp/hack_consul.sh << 'consul_eof'
         if [ $arch == "mips64le" ];then
cat > $(go env GOPATH)/pkg/mod/github.com/boltdb/[email protected]/bolt_mips64x.go <<'EOF'
// +build mips64 mips64le

package bolt

// maxMapSize represents the largest mmap size supported by Bolt.
const maxMapSize = 0x8000000000 // 512GB

// maxAllocSize is the size used when creating array pointers.
const maxAllocSize = 0x7FFFFFFF

// Are unaligned load/stores broken on this arch?
var brokenUnaligned = false
EOF

cat > $(go env GOPATH)/pkg/mod/github.com/boltdb/[email protected]/bolt_mips.go <<'EOF'
// +build mips mipsle

package bolt

// maxMapSize represents the largest mmap size supported by Bolt.
const maxMapSize = 0x40000000 // 1GB

// maxAllocSize is the size used when creating array pointers.
const maxAllocSize = 0xFFFFFFF

// Are unaligned load/stores broken on this arch?
var brokenUnaligned = false
EOF
         fi
consul_eof

取 go install 命令的行数，在前面一行插入 hack 的脚本

CONSUL_HACK_NUM=$(grep -Pn 'debug_run.+?go install' build-support/functions/20-build.sh | cut -d: -f 1)

sed -i "$[CONSUL_HACK_NUM-1]r /tmp/hack_consul.sh" build-support/functions/20-build.sh

继续编译

$ make consul-docker
Building Golang build container
Sending build context to Docker daemon  2.048kB
Step 1/5 : ARG GOLANG_VERSION=1.14.11
Step 2/5 : FROM golang:${GOLANG_VERSION}
 ---> f93db70cba35
Step 3/5 : ARG GOTOOLS="github.com/elazarl/go-bindata-assetfs/...    github.com/hashicorp/go-bindata/...    golang.org/x/tools/cmd/cover    golang.org/x/tools/cmd/stringer    github.com/axw/gocov/gocov    gopkg.in/matm/v1/gocov-html"
 ---> Using cache
 ---> e0979c09e72f
Step 4/5 : RUN GO111MODULE=on go get -v ${GOTOOLS} && mkdir -p /consul
 ---> Using cache
 ---> 9cb7abf49b86
Step 5/5 : WORKDIR /consul
 ---> Using cache
 ---> fb1467e4623c
Successfully built fb1467e4623c
Successfully tagged consul-build-go:latest
==> Building Consul
Ensuring Go modules are up to date
Creating the Go Build Container with image: consul-build-go
Copying the source from '/root/go/src/github.com/hashicorp/consul' to /consul
Running build in container
==> Building Consul - OSes: linux, Architectures: mips64le
Building sequentially with go install
--->   linux/mips64le
Copying back artifacts

$ ls -l pkg/bin/linux_mips64le/
total 112192
-rwxr-xr-x 1 root root 114881061 Jul 16 12:34 consul

测试

拷贝到龙芯机器上测试下可否运行

$ lscpu
Architecture:          mips64el
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    1
Core(s) per socket:    4
座：                 2
NUMA 节点：         2
型号名称：        Loongson-3A R4 (Loongson-3B4000)
CPU max MHz:           1800.0000
CPU min MHz:           900.0000
BogoMIPS：            3594.02
L1d 缓存：          64K
L1i 缓存：          64K
L2 缓存：           256K
L3 缓存：           2048K
NUMA 节点0 CPU：    0-3
NUMA 节点1 CPU：    4-7
$ /tmp/consul --version
Consul v1.8.14
Revision 94c1bdd3b+CHANGES
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)

参考

「其他文章」