Installing Jenkins X on Alibaba Cloud Container Service

Editor's Note: This blog post was originally published on the Jenkins X website .

Jenkins X has just added initial support to install in Alibaba Container Service , its Kubernetes offering.

The following instructions allow installation in a managed Kubernetes cluster in any region outside of mainland China, where more configuration is needed avoid using Google services blocked by chinese authorities (Docker images in GCR). There is a pending issue for that.

Alibaba requires several things in order to create a Kubernetes cluster, so it is easier to do it through the web UI the first time.

The following services need to be activated: Container Service, Resource Orchestration Service (ROS), RAM, and Auto Scaling service, and created the Container Service roles .

If we want to use the command line we can install the aliyun cli. I have added all the steps needed below in case you want to use it.

<code data-lang="shell">brew install aliyun-cli
aliyun configure
REGION=ap-southeast-1 

The clusters need to be created in a VPC, so that needs to be created with VSwitches for each zone to be used.

<code data-lang="shell">aliyun vpc CreateVpc \
    --VpcName jx \
    --Description "Jenkins X" \
    --RegionId ${REGION} \
    --CidrBlock 172.16.0.0/12

{
    "ResourceGroupId": "rg-acfmv2nomuaaaaa",
    "RequestId": "2E795E99-AD73-4EA7-8BF5-F6F391000000",
    "RouteTableId": "vtb-t4nesimu804j33p4aaaaa",
    "VRouterId": "vrt-t4n2w07mdra52kakaaaaa",
    "VpcId": "vpc-t4nszyte14vie746aaaaa"
}

VPC=vpc-t4nszyte14vie746aaaaa

aliyun vpc CreateVSwitch \
    --VSwitchName jx \
    --VpcId ${VPC} \
    --RegionId ${REGION} \
    --ZoneId ${REGION}a \
    --Description "Jenkins X" \
    --CidrBlock 172.16.0.0/24

{
    "RequestId": "89D9AB1F-B4AB-4B4B-8CAA-F68F84417502",
    "VSwitchId": "vsw-t4n7uxycbwgtg14maaaaa"
}

VSWITCH=vsw-t4n7uxycbwgtg14maaaaa 

Next, a keypair (or password) is needed for the cluster instances.

<code data-lang="shell">aliyun ecs ImportKeyPair \
    --KeyPairName jx \
    --RegionId ${REGION} \
    --PublicKeyBody "$(cat ~/.ssh/id_rsa.pub)" 

The last step is to create the cluster using the just created VPC, VSwitch and Keypair. It’s important to select the option Expose API Server with EIP (public_slb in the API json) to be able to connect to the API from the internet.

<code data-lang="shell">echo &#x3C;&#x3C; EOF > cluster.json
{
    "name": "jx-rocks",
    "cluster_type": "ManagedKubernetes",
    "disable_rollback": true,
    "timeout_mins": 60,
    "region_id": "${REGION}",
    "zoneid": "${REGION}a",
    "snat_entry": true,
    "cloud_monitor_flags": false,
    "public_slb": true,
    "worker_instance_type": "ecs.c4.xlarge",
    "num_of_nodes": 3,
    "worker_system_disk_category": "cloud_efficiency",
    "worker_system_disk_size": 120,
    "worker_instance_charge_type": "PostPaid",
    "vpcid": "${VPC}",
    "vswitchid": "${VSWITCH}",
    "container_cidr": "172.20.0.0/16",
    "service_cidr": "172.21.0.0/20",
    "key_pair": "jx"
}
EOF

aliyun cs  POST /clusters \
    --header "Content-Type=application/json" \
    --body "$(cat create.json)"

{
    "cluster_id": "cb643152f97ae4e44980f6199f298f223",
    "request_id": "0C1E16F8-6A9E-4726-AF6E-A8F37CDDC50C",
    "task_id": "T-5cd93cf5b8ff804bb40000e1",
    "instanceId": "cb643152f97ae4e44980f6199f298f223"
}

CLUSTER=cb643152f97ae4e44980f6199f298f223 

We can now download kubectl configuration with

<code data-lang="shell">aliyun cs GET /k8s/${CLUSTER}/user_config | jq -r .config > ~/.kube/config-alibaba
export KUBECONFIG=$KUBECONFIG:~/.kube/config-alibaba 

Another detail before being able to install applications that use PersistentVolumeClaims is to configure a default storage class . There are several volume options that can be listed with kubectl get storageclass .

<code data-lang="shell">NAME                          PROVISIONER     AGE
alicloud-disk-available       alicloud/disk   44h
alicloud-disk-common          alicloud/disk   44h
alicloud-disk-efficiency      alicloud/disk   44h
alicloud-disk-ssd             alicloud/disk   44h 

Each of them matches the following cloud disks:

• alicloud-disk-common: basic cloud disk (minimum size 5GiB). Only available in some zones (us-west-1a, cn-beijing-b,…)

• alicloud-disk-efficiency: high-efficiency cloud disk, ultra disk (minimum size 20GiB).

• alicloud-disk-ssd: SSD disk (minimum size 20GiB).

• alicloud-disk-available: provides highly available options, first attempts to create a high-efficiency cloud disk. If the corresponding AZ’s efficient cloud disk resources are sold out, tries to create an SSD disk. If the SSD is sold out, tries to create a common cloud disk.

To set SSDs as the default:

<code data-lang="shell">kubectl patch storageclass alicloud-disk-ssd \
    -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class":"true"}}}' 

NOTE: Alibaba cloud disks must be more than 5GiB (basic) or 20GiB (SSD and Ultra) ) so we will need to configure any service that is deployed with PVCs to have that size as a minimum or the PersistentVolume provision will fail. The Jenkins X deployments are already configured with this in mind .

Alibaba Kubernetes clusters won’t be able to pull images from the insecure docker registry included in Jenkins X, we need to use Alibaba’s Container Registry by going to https://cr.console.aliyun.com and setting a password.

Then we need to create a Container Registry namespace that allows us to push any image and make them public by default.

From the web UI we can create a Docker login password that we will be using later.

<code data-lang="shell">NAMESPACE=jenkins-x-$(cat /dev/urandom | env LC_CTYPE=C tr -dc 'a-z' | fold -w 6 | head -n 1)
cat &#x3C;&#x3C; EOF > namespace.json
{
    "Namespace": {
        "Namespace": "${NAMESPACE}"
    }
}
EOF
aliyun cr PUT /namespace \
    --header "Content-Type=application/json" \
    --body "$(cat namespace.json)"

cat &#x3C;&#x3C; EOF > namespace.json
{
    "namespace": {
        "namespace": "${NAMESPACE}",
        "defaultVisibility": "PUBLIC",
        "autoCreate": true
    }
}
EOF

aliyun cr POST /namespace/${NAMESPACE} \
    --header "Content-Type=application/json" \
    --body "$(cat namespace.json)" 

Now we can install Jenkins X as usual, passing the --provider alibaba flag.

<code data-lang="shell">jx install \
    --provider alibaba \
    --default-admin-password=admin \
    --default-environment-prefix=jx-rocks \
    --tekton \
    --docker-registry=registry.${REGION}.aliyuncs.com \
    --docker-registry-org=${NAMESPACE} \
    -b 

After installation we need to add the Container Registry credentials to the cluster, the Docker password we have previously created.

<code data-lang="bash">AUTH=$(echo -n "${DOCKER_USERNAME}:${DOCKER_PASSWORD}" | base64)
DATA=$(cat &#x3C;&#x3C; EOF | base64
{
    "auths": {
        "registry.${REGION}.aliyuncs.com": {
            "auth": "${AUTH}"
        }
    }
}
EOF
)
cat &#x3C;&#x3C; EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: jenkins-docker-cfg
  namespace: jx
data:
  config.json: ${DATA}
EOF 

Addons can be installed normally with the caveat mentioned above about PVC minimum size. Which means that we may need to pass some Helm values to the create addon command, depending on the chart we are installing.

For instance to install Prometheus with 20Gi disks:

<code data-lang="shell">jx create addon prometheus \
    -s alertmanager.persistentVolume.size=20Gi,pushgateway.persistentVolume.size=20Gi,server.persistentVolume.size=20Gi 

Tekton builds need to be configured to use PVCs bigger than 20Gi due to the same reasons. The default is to use 5GiB PVCs.

<code data-lang="shell">cat &#x3C;&#x3C; EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: config-artifact-pvc
  namespace: jx
data:
  size: 20Gi
EOF 

• Learn more about Kubernetes and its impact on the cloud

• Discover more about Jenkins X

• Read how to deliver secure apps on Kubernetes