自定义Jenkins-slave镜像

概述:

在前面章节,我们提到过通过Jenkins对接Kubernetes能实现对Jenkins slave的弹性部署和回收,能非常有效的利用资源,在实际使用的过程中,我们会根据不同的编程语言去构建一个对应的jnlp,或在一个jnlp镜像内包含全部编程语言。下面讲解一下如何定制自己的jnlp镜像。

软件版本:
os:ubuntu 16.04
Kubernetes:1.12.0
jenkins:2.89.4

制做方法:

基于标准操作系统base镜像构建,我这里使用debian,当然也可以使用centos,ubuntu,alpine等其他镜像(以java语言环境为例,构建maven编译环境,其他开发语言类似)

1、安装配置JAVA环境和maven编译环境,下载jdk,maven
https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
下载Linux x64 tar包,解压到Dockerfile目录

http://mirror.bit.edu.cn/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
解压tar包,到Dockerfile目录

2、安装配置Jenkins-slave
https://github.com/jenkinsci/docker-jnlp-slave/blob/master/jenkins-slave
下载jenkins-slave启动脚本,并放置到 /usr/local/bin/目录

3、安装slave.jar包
slave.jar包主要用于Jenkins-slave去连接Jenkins-server所用
https://repo.jenkins-ci.org/public/org/jenkins-ci/main/remoting/3.16/remoting-3.16.jar
下载slave.jar包并放置到/usr/share/jenkins/目录

4、安装配置docker-in-docker
下载docker二进制包
https://download.docker.com/linux/static/stable/x86_64/
docker-17.03.2-ce.tgz
解压后,放到Dockerfile目录

5、下载kubectl二进制包
https://dl.k8s.io/v1.11.3/kubernetes-client-linux-amd64.tar.gz

解压后,放到Dockerfile目录

6、放置连接Kubernetes集群的kubeconfig文件
拷贝连接集群的kubeconfig文件,重命名为config

1
mv kubeconfig config

基于以下Dockerfile构建镜像

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
FROM debian:9
USER root
ARG AGENT_WORKDIR=/home/jenkins/agent
RUN mkdir -p /home/jenkins/.jenkins && mkdir -p ${AGENT_WORKDIR}
COPY jdk1.8.0_181 /usr/local/jdk1.8.0_181
COPY maven3.3.9 /usr/local/maven3.3.9
COPY jenkins-slave /usr/local/bin/jenkins-slave
COPY slave.jar /usr/share/jenkins/
COPY docker /usr/local/docker
COPY kubectl /usr/bin/
COPY config /root
ENV PATH=/usr/local/jdk1.8.0_181/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/maven3.3.9/bin:/usr/local/docker
RUN chmod +x /usr/local/bin/jenkins-slave
ENV HOME /home/jenkins
RUN chmod 755 /usr/share/jenkins && chmod 644 /usr/share/jenkins/slave.jar
USER root
ENV AGENT_WORKDIR=${AGENT_WORKDIR}
WORKDIR /home/jenkins
ENTRYPOINT ["jenkins-slave"]

测试用例
https://github.com/wanshaoyuan/pipeline-exemple.git

测试的Jenkins的pipeline
注意:deployment.yaml里面应用部署在sock-shop这个命名空间内,需要提前创建好。
jenkins-pipeline

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
podTemplate(label: 'jnlp-slave', cloud: 'kubernetes') {
node('jnlp-slave') {
stage('git clone'){
//clone CODE根据实际情况修改
git credentialsId: 'ba80cc36-77b1-42d0-8838-f834de346fe1', url: 'https://github.com/wanshaoyuan/pipeline-exemple.git'
}
stage('code build'){
sh '''/usr/local/maven3.3.9/bin/mvn -DskipTests package -f /home/jenkins/workspace/test-jnlp2'''
}
stage('image build'){
sh '''echo $BUILD_ID
GROUP=172.31.164.66/library COMMIT=$BUILD_ID /home/jenkins/workspace/test-jnlp2/scripts/build.sh'''
}
stage('upload'){
//上传镜像
sh '''docker login 172.31.164.66 -u=admin -p=123456
docker push 172.31.164.66/library/shipping:$BUILD_ID'''
}
stage('deploy'){
//执行部署脚本
sh 'sed -i "s/172.31.164.66\\/library\\/shipping:.*/172.31.164.66\\/library\\/shipping:$BUILD_ID/g" /home/jenkins/workspace/test-jnlp2/deployment.yml'
sh 'kubectl apply -f /home/jenkins/workspace/test-jnlp2/deployment.yml --kubeconfig /root/config'
}
}
}

配置jnlp映射宿主机的docker.sock文件,不然无法实现容器镜像build

执行完毕

最后访问集群30001

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
kubectl get svc -n sock-shop
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
carts ClusterIP 10.43.23.102 <none> 80/TCP 1d
carts-db ClusterIP 10.43.229.245 <none> 27017/TCP 1d
catalogue ClusterIP 10.43.133.136 <none> 80/TCP 1d
catalogue-db ClusterIP 10.43.156.93 <none> 3306/TCP 1d
front-end NodePort 10.43.130.209 <none> 80:30001/TCP 1d
orders ClusterIP 10.43.212.179 <none> 80/TCP 1d
orders-db ClusterIP 10.43.178.3 <none> 27017/TCP 1d
payment ClusterIP 10.43.205.54 <none> 80/TCP 1d
queue-master ClusterIP 10.43.178.73 <none> 80/TCP 1d
rabbitmq ClusterIP 10.43.233.8 <none> 5672/TCP 1d
shipping ClusterIP 10.43.198.185 <none> 80/TCP 1d
user ClusterIP 10.43.213.139 <none> 80/TCP 1d
user-db ClusterIP 10.43.94.66 <none> 27017/TCP 1d

http://host_ip:30001