重寫Nacos服務發現:多個伺服器如何跨名稱空間,訪問公共服務?

語言: CN / TW / HK

一、問題背景

在開發某個公共應用時,筆者發現該公共應用的資料是所有測試環境(假設存在 dev/dev2/dev3)通用的。

這就意味著只需部署一個應用,就能滿足所有測試環境的需求;也意味著所有測試環境都需要呼叫該公共應用,而不同測試環境的應用註冊在不同的 Nacos 名稱空間。

二、兩種解決方案

如果所有測試環境都需要呼叫該公共應用,有兩種可行的方案。第一種,將該公共服務同時註冊到不同的測試環境所對應的名稱空間中。

第二種,將公共應用註冊到單獨的名稱空間,不同的測試環境能夠跨名稱空間訪問該應用。

三、詳細的問題解決過程

先行交代筆者的版本號配置。Nacos 客戶端版本號為 NACOS 1.4.1;Java 專案的 Nacos 版本號如下。

最初想法是將該公共應用同時註冊到多個名稱空間下。在查詢資料的過程中,團隊成員在 GitHub 上發現了一篇類似問題的部落格分享:Registration Center: Can services in different namespaces be called from each other? #1176,原文傳送器:https://github.com/alibaba/nacos/issues/1176。

01 註冊多個名稱空間

從該部落格中,我們看到其他程式設計師朋友也遇到了類似的公共服務的需求。在本篇文章中,筆者將進一步分享實現思路以及示例程式碼。

說明:以下程式碼內容來自使用者 chuntaojun 的分享。

shareNamespace={namespaceId[:group]},{namespaceId[:group]} 


@RunWith(SpringRunner.class)
@SpringBootTest(classes = NamingApp.class, properties = {"server.servlet.context-path=/nacos"},
    webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class SelectServiceInShareNamespace_ITCase {

    private NamingService naming1;
    private NamingService naming2;
    @LocalServerPort
    private int port;
    @Before
    public void init() throws Exception{
        NamingBase.prepareServer(port);
        if (naming1 == null) {
            Properties properties = new Properties();
            properties.setProperty(PropertyKeyConst.SERVER_ADDR, "127.0.0.1"+":"+port);
            properties.setProperty(PropertyKeyConst.SHARE_NAMESPACE, "57425802-3058-4507-9a73-3229b9f00a36");
            naming1 = NamingFactory.createNamingService(properties);

            Properties properties2 = new Properties();
            properties2.setProperty(PropertyKeyConst.SERVER_ADDR, "127.0.0.1"+":"+port);
            properties2.setProperty(PropertyKeyConst.NAMESPACE, "57425802-3058-4507-9a73-3229b9f00a36");
            naming2 = NamingFactory.createNamingService(properties2);
        }
        while (true) {
            if (!"UP".equals(naming1.getServerStatus())) {
                Thread.sleep(1000L);
                continue;
            }
            break;
        }
    }

    @Test
    public void testSelectInstanceInShareNamespaceNoGroup() throws NacosException, InterruptedException {
        String service1 = randomDomainName();
        String service2 = randomDomainName();
        naming1.registerInstance(service1, "127.0.0.1", 90);
        naming2.registerInstance(service2, "127.0.0.2", 90);

        Thread.sleep(1000);

        List<Instance> instances = naming1.getAllInstances(service2);
        Assert.assertEquals(1, instances.size());
        Assert.assertEquals(service2, NamingUtils.getServiceName(instances.get(0).getServiceName()));
    }

    @Test
    public void testSelectInstanceInShareNamespaceWithGroup() throws NacosException, InterruptedException {
        String service1 = randomDomainName();
        String service2 = randomDomainName();
        naming2.registerInstance(service1, groupName, "127.0.0.1", 90);
        naming3.registerInstance(service2, "127.0.0.2", 90);

        Thread.sleep(1000);

        List<Instance> instances = naming3.getAllInstances(service1);
        Assert.assertEquals(1, instances.size());
        Assert.assertEquals(service1, NamingUtils.getServiceName(instances.get(0).getServiceName()));
        Assert.assertEquals(groupName, NamingUtils.getServiceName(NamingUtils.getGroupName(instances.get(0).getServiceName())));
    }

}


進一步考慮後發現該解決方案可能不太契合當前遇到的問題。公司目前的開發測試環境有很多個,並且不確定以後會不會繼續增加。

如果每增加一個環境,都需要修改一次公共服務的配置,並且重啟一次公共服務,著實太麻煩了。倒不如反其道而行,讓其他的伺服器實現跨名稱空間訪問公共服務。

02 跨名稱空間訪問

針對實際問題查詢資料時,我們找到了類似的參考分享《重寫 Nacos 服務發現邏輯動態修改遠端服務IP地址》,原文傳送器:https://www.cnblogs.com/changxy-codest/p/14632574.html。

跟著部落格思路看程式碼,筆者瞭解到服務發現的主要相關類是 NacosNamingService, NacosDiscoveryProperties, NacosDiscoveryAutoConfiguration

然後,筆者將部落格的示例程式碼複製過來,試著進行如下除錯:

@Slf4j
@Configuration
@ConditionalOnNacosDiscoveryEnabled
@ConditionalOnProperty(
        name = {"spring.profiles.active"},
        havingValue = "dev"
)
@AutoConfigureBefore({NacosDiscoveryClientAutoConfiguration.class})
public class DevEnvironmentNacosDiscoveryClient {

    @Bean
    @ConditionalOnMissingBean
    public NacosDiscoveryProperties nacosProperties() {
        return new DevEnvironmentNacosDiscoveryProperties();
    }

    static class DevEnvironmentNacosDiscoveryProperties extends NacosDiscoveryProperties {

        private NamingService namingService;

        @Override
        public NamingService namingServiceInstance() {
            if (null != this.namingService) {
                return this.namingService;
            } else {
                Properties properties = new Properties();
                properties.put("serverAddr", super.getServerAddr());
                properties.put("namespace", super.getNamespace());
                properties.put("com.alibaba.nacos.naming.log.filename", super.getLogName());
                if (super.getEndpoint().contains(":")) {
                    int index = super.getEndpoint().indexOf(":");
                    properties.put("endpoint", super.getEndpoint().substring(0, index));
                    properties.put("endpointPort", super.getEndpoint().substring(index + 1));
                } else {
                    properties.put("endpoint", super.getEndpoint());
                }

                properties.put("accessKey", super.getAccessKey());
                properties.put("secretKey", super.getSecretKey());
                properties.put("clusterName", super.getClusterName());
                properties.put("namingLoadCacheAtStart", super.getNamingLoadCacheAtStart());

                try {
                    this.namingService = new DevEnvironmentNacosNamingService(properties);
                } catch (Exception var3) {
                    log.error("create naming service error!properties={},e=,", this, var3);
                    return null;
                }

                return this.namingService;
            }
        }

    }

    static class DevEnvironmentNacosNamingService extends NacosNamingService {

        public DevEnvironmentNacosNamingService(Properties properties) {
            super(properties);
        }

        @Override
        public List<Instance> selectInstances(String serviceName, List<String> clusters, boolean healthy) throws NacosException {
            List<Instance> instances = super.selectInstances(serviceName, clusters, healthy);
            instances.stream().forEach(instance -> instance.setIp("10.101.232.24"));
            return instances;
        }
    }

}


除錯後發現部落格提供的程式碼並不能滿足筆者的需求,還得進一步深入探索。

但幸運的是,除錯過程發現 Nacos 服務發現的關鍵類是 com.alibaba.cloud.nacos.discovery.NacosServiceDiscovery,其中的關鍵方法是 getInstances()getServices(),即「返回指定服務 ID 的所有服務例項」和「獲取所有服務的名稱」。

也就是說,對 getInstances() 方法進行重寫肯定能實現本次目標——跨名稱空間訪問公共服務。

/**
 * Return all instances for the given service.
 * @param serviceId id of service
 * @return list of instances
 * @throws NacosException nacosException
 */
public List<ServiceInstance> getInstances(String serviceId) throws NacosException {
        String group = discoveryProperties.getGroup();
        List<Instance> instances = discoveryProperties.namingServiceInstance()
                        .selectInstances(serviceId, group, true);
        return hostToServiceInstanceList(instances, serviceId);
}

/**
 * Return the names of all services.
 * @return list of service names
 * @throws NacosException nacosException
 */
public List<String> getServices() throws NacosException {
        String group = discoveryProperties.getGroup();
        ListView<String> services = discoveryProperties.namingServiceInstance()
                        .getServicesOfServer(1, Integer.MAX_VALUE, group);
        return services.getData();
}


03 最終解決思路及程式碼示例

具體的解決方案思路大致如下:

  1. 生成一個共享配置類NacosShareProperties,用來配置共享公共服務的 namespacegroup
  2. 重寫配置類 NacosDiscoveryProperties (新:NacosDiscoveryPropertiesV2),將新增的共享配置類作為屬性放進該配置類,後續會用到;
  3. 重寫服務發現類 NacosServiceDiscovery (新:NacosServiceDiscoveryV2),這是最關鍵的邏輯;
  4. 重寫自動配置類 NacosDiscoveryAutoConfiguration,將自定義相關類比 Nacos 原生類更早的注入容器。

最終程式碼中用到了一些工具類,可以自行補充完整。

/**
 * <pre>
 *  @description: 共享nacos屬性
 *  @author: rookie0peng
 *  @date: 2022/8/29 15:22
 *  </pre>
 */
@Configuration
@ConfigurationProperties(prefix = "nacos.share")
public class NacosShareProperties {

    private final Map<String, Set<String>> NAMESPACE_TO_GROUP_NAME_MAP = new ConcurrentHashMap<>();

    /**
     * 共享nacos實體列表
     */
    private List<NacosShareEntity> entities;

    public List<NacosShareEntity> getEntities() {
        return entities;
    }

    public void setEntities(List<NacosShareEntity> entities) {
        this.entities = entities;
    }

    public Map<String, Set<String>> getNamespaceGroupMap() {
        safeStream(entities).filter(entity -> nonNull(entity) && nonNull(entity.getNamespace()))
                .forEach(entity -> {
                    Set<String> groupNames = NAMESPACE_TO_GROUP_NAME_MAP.computeIfAbsent(entity.getNamespace(), k -> new HashSet<>());
                    if (nonNull(entity.getGroupNames()))
                        groupNames.addAll(entity.getGroupNames());
                });
        return new HashMap<>(NAMESPACE_TO_GROUP_NAME_MAP);
    }

    @Override
    public String toString() {
        return "NacosShareProperties{" +
                "entities=" + entities +
                '}';
    }

    /**
     * 共享nacos實體
     */
    public static final class NacosShareEntity {

        /**
         * 名稱空間
         */
        private String namespace;

        /**
         * 分組
         */
        private List<String> groupNames;

        public String getNamespace() {
            return namespace;
        }

        public void setNamespace(String namespace) {
            this.namespace = namespace;
        }

        public List<String> getGroupNames() {
            return groupNames;
        }

        public void setGroupNames(List<String> groupNames) {
            this.groupNames = groupNames;
        }

        @Override
        public String toString() {
            return "NacosShareEntity{" +
                    "namespace='" + namespace + '\'' +
                    ", groupNames=" + groupNames +
                    '}';
        }
    }
}



/**
 * @description: naocs服務發現屬性重寫
 * @author: rookie0peng
 * @date: 2022/8/30 1:19
 */
public class NacosDiscoveryPropertiesV2 extends NacosDiscoveryProperties {

    private static final Logger log = LoggerFactory.getLogger(NacosDiscoveryPropertiesV2.class);

    private final NacosShareProperties nacosShareProperties;

    private static final Map<String, NamingService> NAMESPACE_TO_NAMING_SERVICE_MAP = new ConcurrentHashMap<>();

    public NacosDiscoveryPropertiesV2(NacosShareProperties nacosShareProperties) {
        super();
        this.nacosShareProperties = nacosShareProperties;
    }

    public Map<String, NamingService> shareNamingServiceInstances() {
        if (!NAMESPACE_TO_NAMING_SERVICE_MAP.isEmpty()) {
            return new HashMap<>(NAMESPACE_TO_NAMING_SERVICE_MAP);
        }
        List<NacosShareProperties.NacosShareEntity> entities = Optional.ofNullable(nacosShareProperties)
                .map(NacosShareProperties::getEntities).orElse(Collections.emptyList());
        entities.stream().filter(entity -> nonNull(entity) && nonNull(entity.getNamespace()))
                .filter(PredicateUtil.distinctByKey(NacosShareProperties.NacosShareEntity::getNamespace))
                .forEach(entity -> {
                    try {
                        NamingService namingService = NacosFactory.createNamingService(getNacosProperties(entity.getNamespace()));
                        if (namingService != null) {
                            NAMESPACE_TO_NAMING_SERVICE_MAP.put(entity.getNamespace(), namingService);
                        }
                    } catch (Exception e) {
                        log.error("create naming service error! properties={}, e=", this, e);
                    }
                });
        return new HashMap<>(NAMESPACE_TO_NAMING_SERVICE_MAP);
    }

    private Properties getNacosProperties(String namespace) {
        Properties properties = new Properties();
        properties.put(SERVER_ADDR, getServerAddr());
        properties.put(USERNAME, Objects.toString(getUsername(), ""));
        properties.put(PASSWORD, Objects.toString(getPassword(), ""));
        properties.put(NAMESPACE, namespace);
        properties.put(UtilAndComs.NACOS_NAMING_LOG_NAME, getLogName());
        String endpoint = getEndpoint();
        if (endpoint.contains(":")) {
            int index = endpoint.indexOf(":");
            properties.put(ENDPOINT, endpoint.substring(0, index));
            properties.put(ENDPOINT_PORT, endpoint.substring(index + 1));
        }
        else {
            properties.put(ENDPOINT, endpoint);
        }

        properties.put(ACCESS_KEY, getAccessKey());
        properties.put(SECRET_KEY, getSecretKey());
        properties.put(CLUSTER_NAME, getClusterName());
        properties.put(NAMING_LOAD_CACHE_AT_START, getNamingLoadCacheAtStart());

//        enrichNacosDiscoveryProperties(properties);
        return properties;
    }
}



/**
 * @description: naocs服務發現重寫
 * @author: rookie0peng
 * @date: 2022/8/30 1:10
 */
public class NacosServiceDiscoveryV2 extends NacosServiceDiscovery {

    private final NacosDiscoveryPropertiesV2 discoveryProperties;

    private final NacosShareProperties nacosShareProperties;

    private final NacosServiceManager nacosServiceManager;

    public NacosServiceDiscoveryV2(NacosDiscoveryPropertiesV2 discoveryProperties, NacosShareProperties nacosShareProperties, NacosServiceManager nacosServiceManager) {
        super(discoveryProperties, nacosServiceManager);
        this.discoveryProperties = discoveryProperties;
        this.nacosShareProperties = nacosShareProperties;
        this.nacosServiceManager = nacosServiceManager;
    }

    /**
     * Return all instances for the given service.
     * @param serviceId id of service
     * @return list of instances
     * @throws NacosException nacosException
     */
    public List<ServiceInstance> getInstances(String serviceId) throws NacosException {
        String group = discoveryProperties.getGroup();
        List<Instance> instances = discoveryProperties.namingServiceInstance()
                .selectInstances(serviceId, group, true);
        if (isEmpty(instances)) {
            Map<String, Set<String>> namespaceGroupMap = nacosShareProperties.getNamespaceGroupMap();
            Map<String, NamingService> namespace2NamingServiceMap = discoveryProperties.shareNamingServiceInstances();
            for (Map.Entry<String, NamingService> entry : namespace2NamingServiceMap.entrySet()) {
                String namespace;
                NamingService namingService;
                if (isNull(namespace = entry.getKey()) || isNull(namingService = entry.getValue()))
                    continue;
                Set<String> groupNames = namespaceGroupMap.get(namespace);
                List<Instance> shareInstances;
                if (isEmpty(groupNames)) {
                    shareInstances = namingService.selectInstances(serviceId, group, true);
                    if (nonEmpty(shareInstances))
                        break;
                } else {
                    shareInstances = new ArrayList<>();
                    for (String groupName : groupNames) {
                        List<Instance> subShareInstances = namingService.selectInstances(serviceId, groupName, true);
                        if (nonEmpty(subShareInstances)) {
                            shareInstances.addAll(subShareInstances);
                        }
                    }
                }
                if (nonEmpty(shareInstances)) {
                    instances = shareInstances;
                    break;
                }
            }
        }
        return hostToServiceInstanceList(instances, serviceId);
    }

    /**
     * Return the names of all services.
     * @return list of service names
     * @throws NacosException nacosException
     */
    public List<String> getServices() throws NacosException {
        String group = discoveryProperties.getGroup();
        ListView<String> services = discoveryProperties.namingServiceInstance()
                .getServicesOfServer(1, Integer.MAX_VALUE, group);
        return services.getData();
    }

    public static List<ServiceInstance> hostToServiceInstanceList(
            List<Instance> instances, String serviceId) {
        List<ServiceInstance> result = new ArrayList<>(instances.size());
        for (Instance instance : instances) {
            ServiceInstance serviceInstance = hostToServiceInstance(instance, serviceId);
            if (serviceInstance != null) {
                result.add(serviceInstance);
            }
        }
        return result;
    }

    public static ServiceInstance hostToServiceInstance(Instance instance,
                                                        String serviceId) {
        if (instance == null || !instance.isEnabled() || !instance.isHealthy()) {
            return null;
        }
        NacosServiceInstance nacosServiceInstance = new NacosServiceInstance();
        nacosServiceInstance.setHost(instance.getIp());
        nacosServiceInstance.setPort(instance.getPort());
        nacosServiceInstance.setServiceId(serviceId);

        Map<String, String> metadata = new HashMap<>();
        metadata.put("nacos.instanceId", instance.getInstanceId());
        metadata.put("nacos.weight", instance.getWeight() + "");
        metadata.put("nacos.healthy", instance.isHealthy() + "");
        metadata.put("nacos.cluster", instance.getClusterName() + "");
        metadata.putAll(instance.getMetadata());
        nacosServiceInstance.setMetadata(metadata);

        if (metadata.containsKey("secure")) {
            boolean secure = Boolean.parseBoolean(metadata.get("secure"));
            nacosServiceInstance.setSecure(secure);
        }
        return nacosServiceInstance;
    }

    private NamingService namingService() {
        return nacosServiceManager
                .getNamingService(discoveryProperties.getNacosProperties());
    }
}



/**
 * @description: 重寫nacos服務發現的自動配置
 * @author: rookie0peng
 * @date: 2022/8/30 1:08
 */
@Configuration(proxyBeanMethods = false)
@ConditionalOnDiscoveryEnabled
@ConditionalOnNacosDiscoveryEnabled
@AutoConfigureBefore({NacosDiscoveryAutoConfiguration.class})
public class NacosDiscoveryAutoConfigurationV2 {

    @Bean
    @ConditionalOnMissingBean
    public NacosDiscoveryPropertiesV2 nacosProperties(NacosShareProperties nacosShareProperties) {
        return new NacosDiscoveryPropertiesV2(nacosShareProperties);
    }

    @Bean
    @ConditionalOnMissingBean
    public NacosServiceDiscovery nacosServiceDiscovery(
            NacosDiscoveryPropertiesV2 discoveryPropertiesV2, NacosShareProperties nacosShareProperties, NacosServiceManager nacosServiceManager
    ) {
        return new NacosServiceDiscoveryV2(discoveryPropertiesV2, nacosShareProperties, nacosServiceManager);
    }
}



本以為問題到這就結束了,但最後自測時發現程式根本不走 Nacos 的服務發現邏輯,而是執行 Ribbon 的負載均衡邏輯com.netflix.loadbalancer.AbstractLoadBalancerRule

不過實現類是 com.alibaba.cloud.nacos.ribbon.NacosRule,繼續基於 NacosRule 重寫負載均衡。

/**
 * @description: 共享nacos名稱空間規則
 * @author: rookie0peng
 * @date: 2022/8/31 2:04
 */
public class ShareNacosNamespaceRule extends AbstractLoadBalancerRule {

    private static final Logger LOGGER = LoggerFactory.getLogger(ShareNacosNamespaceRule.class);

    @Autowired
    private NacosDiscoveryPropertiesV2 nacosDiscoveryPropertiesV2;
    @Autowired
    private NacosShareProperties nacosShareProperties;

    /**
     * 重寫choose方法
     *
     * @param key
     * @return
     */
    @SneakyThrows
    @Override
    public Server choose(Object key) {
        try {
            String clusterName = this.nacosDiscoveryPropertiesV2.getClusterName();
            DynamicServerListLoadBalancer loadBalancer = (DynamicServerListLoadBalancer) getLoadBalancer();
            String name = loadBalancer.getName();

            NamingService namingService = nacosDiscoveryPropertiesV2
                    .namingServiceInstance();
            List<Instance> instances = namingService.selectInstances(name, true);
            if (CollectionUtils.isEmpty(instances)) {
                LOGGER.warn("no instance in service {}, then to get share service's instance", name);
                List<Instance> shareNamingService = this.getShareNamingService(name);
                if (nonEmpty(shareNamingService))
                    instances = shareNamingService;
                else
                    return null;
            }
            List<Instance> instancesToChoose = instances;
            if (org.apache.commons.lang3.StringUtils.isNotBlank(clusterName)) {
                List<Instance> sameClusterInstances = instances.stream()
                        .filter(instance -> Objects.equals(clusterName,
                                instance.getClusterName()))
                        .collect(Collectors.toList());
                if (!CollectionUtils.isEmpty(sameClusterInstances)) {
                    instancesToChoose = sameClusterInstances;
                }
                else {
                    LOGGER.warn(
                            "A cross-cluster call occurs,name = {}, clusterName = {}, instance = {}",
                            name, clusterName, instances);
                }
            }

            Instance instance = ExtendBalancer.getHostByRandomWeight2(instancesToChoose);

            return new NacosServer(instance);
        }
        catch (Exception e) {
            LOGGER.warn("NacosRule error", e);
            return null;
        }
    }


    @Override
    public void initWithNiwsConfig(IClientConfig iClientConfig) {

    }

    private List<Instance> getShareNamingService(String serviceId) throws NacosException {
        List<Instance> instances = Collections.emptyList();
        Map<String, Set<String>> namespaceGroupMap = nacosShareProperties.getNamespaceGroupMap();
        Map<String, NamingService> namespace2NamingServiceMap = nacosDiscoveryPropertiesV2.shareNamingServiceInstances();
        for (Map.Entry<String, NamingService> entry : namespace2NamingServiceMap.entrySet()) {
            String namespace;
            NamingService namingService;
            if (isNull(namespace = entry.getKey()) || isNull(namingService = entry.getValue()))
                continue;
            Set<String> groupNames = namespaceGroupMap.get(namespace);
            List<Instance> shareInstances;
            if (isEmpty(groupNames)) {
                shareInstances = namingService.selectInstances(serviceId, true);
                if (nonEmpty(shareInstances))
                    break;
            } else {
                shareInstances = new ArrayList<>();
                for (String groupName : groupNames) {
                    List<Instance> subShareInstances = namingService.selectInstances(serviceId, groupName, true);
                    if (nonEmpty(subShareInstances)) {
                        shareInstances.addAll(subShareInstances);
                    }
                }
            }
            if (nonEmpty(shareInstances)) {
                instances = shareInstances;
                break;
            }
        }
        return instances;
    }
}


至此問題得以解決。

Nacos 上配置好共享 namespacegroup 後,就能夠進行跨名稱空間訪問了。


# nacos共享名稱空間配置 示例
nacos.share.entities[0].namespace=e6ed2017-3ed6-4d9b-824a-db626424fc7b
nacos.share.entities[0].groupNames[0]=DEFAULT_GROUP
# 指定服務使用共享的負載均衡規則,service-id是註冊到nacos上的服務id,ShareNacosNamespaceRule需要寫全限定名
service-id.ribbon.NFLoadBalancerRuleClassName=***.***.***.ShareNacosNamespaceRule



注意:如果 Java 專案的 nacos discovery 版本用的是 2021.1,則不需要重寫 Ribbon 的負載均衡類,因為該版本的 Nacos 不依賴 Ribbon。

2.2.1.RELEASE 版本nacos discovery 依賴 Ribbon.

2021.1 版本的 nacos discovery 不依賴 Ribbon。

四、總結

為了達到共享名稱空間的預期,構思、查詢資料、實現邏輯、除錯,前後一共花費 4 天左右。

但該功能仍然存在共享服務快取等可優化空間,留待後續實現。

五、參考文獻

[1] Registration Center: Can services in different namespaces be called from each other? [EB/OL]. https://github.com/alibaba/nacos/issues/1176, 2019-05-07/2022-11-29.

[2] 重寫Nacos服務發現邏輯動態修改遠端服務IP地址 [EB/OL]. https://www.cnblogs.com/changxy-codest/p/14632574.html, 2021-04-08/2022-11-29.

瞭解更多敏捷開發、專案管理、行業動態等訊息,關注我們 [LigaAI@oschina](https://my.oschina.net/u/5057806) 或點選LigaAI - 新一代智慧研發協作平臺,線上申請體驗我們的產品。