Coreos集群中安装Kubernetes的步骤(2)

Master节点Coreos版本最好在1097以上,至少需要保证rkt 1.2.1以上。

创建统一的SSL目录
sudo mkdir -p /etc/kubernetes/ssl  

拷贝以下证书到SSL目录中

- /etc/kubernetes/ssl/ca.pem
- /etc/kubernetes/ssl/apiserver.pem
- /etc/kubernetes/ssl/apiserver-key.pem

修改访问权限

sudo chmod 600 /etc/kubernetes/ssl/*-key.pem  
sudo chown root:root /etc/kubernetes/ssl/*-key.pem  
网络配置

建议采用Flannel网络

创建 /etc/flannel/options.env

FLANNELD_IFACE=${ADVERTISE_IP}  
FLANNELD_ETCD_ENDPOINTS=${ETCD_ENDPOINTS}

- ADVERTISE_IP Master节点IP
- ETCD_ENDPOINTS ETCD集群地址,例如:http://10.12.1.104:2379,http://10.12.1.105:2379,http://10.12.1.106:2379

创建/etc/systemd/system/flanneld.service.d/40-ExecStartPre-symlink.conf

[Service]
ExecStartPre=/usr/bin/ln -sf /etc/flannel/options.env /run/flannel/options.env  
Docker配置

因为采用Flannel网络,所以Docker需要支持Flannel.

创建/etc/systemd/system/docker.service.d/40-flannel.conf

[Unit]
Requires=flanneld.service  
After=flanneld.service  
[Service]
EnvironmentFile=/etc/kubernetes/cni/docker_opts_cni.env  

创建/etc/kubernetes/cni/dockeroptscni.env

DOCKER_OPT_BIP=""  
DOCKER_OPT_IPMASQ=""  

创建/etc/kubernetes/cni/net.d/10-flannel.conf

{
    "name": "podnet",
    "type": "flannel",
    "delegate": {
        "isDefaultGateway": true
    }
}
创建kubelet 服务
  • 创建/etc/systemd/system/kubelet.service
- K8S_VER hyperkube版本, 官方推荐为v1.6.1_coreos.0
- ADVERTISE_IP Master节点IP
- DNS_SERVICE_IP 10.3.0.10
- NETWORK_PLUGIN 因此采用Flannel,因此不需要修改

[Service]
Environment=KUBELET_IMAGE_TAG=${K8S_VER}  
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/run/kubelet-pod.uuid \  
  --volume var-log,kind=host,source=/var/log \
  --mount volume=var-log,target=/var/log \
  --volume dns,kind=host,source=/etc/resolv.conf \
  --mount volume=dns,target=/etc/resolv.conf"
ExecStartPre=/usr/bin/mkdir -p /etc/kubernetes/manifests  
ExecStartPre=/usr/bin/mkdir -p /var/log/containers  
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/run/kubelet-pod.uuid  
ExecStart=/usr/lib/coreos/kubelet-wrapper \  
  --api-servers=http://127.0.0.1:8080 \
  --register-schedulable=false \
  --cni-conf-dir=/etc/kubernetes/cni/net.d \
  --network-plugin=${NETWORK_PLUGIN} \
  --container-runtime=docker \
  --allow-privileged=true \
  --pod-manifest-path=/etc/kubernetes/manifests \
  --hostname-override=${ADVERTISE_IP} \
  --cluster_dns=${DNS_SERVICE_IP} \
  --cluster_domain=cluster.local
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/run/kubelet-pod.uuid  
Restart=always  
RestartSec=10

[Install]
WantedBy=multi-user.target  
  • 创建/etc/kubernetes/manifests/kube-apiserver.yaml
- ETCD_ENDPOINTS 与上面配置的ETCD地址保持一致
- SERVICE_IP_RANGE 10.3.0.0/24 
- ADVERTISE_IP Master节点IP

apiVersion: v1  
kind: Pod  
metadata:  
  name: kube-apiserver
  namespace: kube-system
spec:  
  hostNetwork: true
  containers:
  - name: kube-apiserver
    image: quay.io/coreos/hyperkube:v1.6.1_coreos.0
    command:
    - /hyperkube
    - apiserver
    - --bind-address=0.0.0.0
    - --etcd-servers=${ETCD_ENDPOINTS}
    - --allow-privileged=true
    - --service-cluster-ip-range=${SERVICE_IP_RANGE}
    - --secure-port=443
    - --advertise-address=${ADVERTISE_IP}
    - --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota
    - --tls-cert-file=/etc/kubernetes/ssl/apiserver.pem
    - --tls-private-key-file=/etc/kubernetes/ssl/apiserver-key.pem
    - --client-ca-file=/etc/kubernetes/ssl/ca.pem
    - --service-account-key-file=/etc/kubernetes/ssl/apiserver-key.pem
    - --runtime-config=extensions/v1beta1/networkpolicies=true
    - --anonymous-auth=false
    livenessProbe:
      httpGet:
        host: 127.0.0.1
        port: 8080
        path: /healthz
      initialDelaySeconds: 15
      timeoutSeconds: 15
    ports:
    - containerPort: 443
      hostPort: 443
      name: https
    - containerPort: 8080
      hostPort: 8080
      name: local
    volumeMounts:
    - mountPath: /etc/kubernetes/ssl
      name: ssl-certs-kubernetes
      readOnly: true
    - mountPath: /etc/ssl/certs
      name: ssl-certs-host
      readOnly: true
  volumes:
  - hostPath:
      path: /etc/kubernetes/ssl
    name: ssl-certs-kubernetes
  - hostPath:
      path: /usr/share/ca-certificates
    name: ssl-certs-host
  • 创建/etc/kubernetes/manifests/kube-proxy.yaml
apiVersion: v1  
kind: Pod  
metadata:  
  name: kube-proxy
  namespace: kube-system
spec:  
  hostNetwork: true
  containers:
  - name: kube-proxy
    image: quay.io/coreos/hyperkube:v1.6.1_coreos.0
    command:
    - /hyperkube
    - proxy
    - --master=http://127.0.0.1:8080
    securityContext:
      privileged: true
    volumeMounts:
    - mountPath: /etc/ssl/certs
      name: ssl-certs-host
      readOnly: true
  volumes:
  - hostPath:
      path: /usr/share/ca-certificates
    name: ssl-certs-host
  • 创建/etc/kubernetes/manifests/kube-controller-manager.yaml
apiVersion: v1  
kind: Pod  
metadata:  
  name: kube-controller-manager
  namespace: kube-system
spec:  
  hostNetwork: true
  containers:
  - name: kube-controller-manager
    image: quay.io/coreos/hyperkube:v1.6.1_coreos.0
    command:
    - /hyperkube
    - controller-manager
    - --master=http://127.0.0.1:8080
    - --leader-elect=true
    - --service-account-private-key-file=/etc/kubernetes/ssl/apiserver-key.pem
    - --root-ca-file=/etc/kubernetes/ssl/ca.pem
    resources:
      requests:
        cpu: 200m
    livenessProbe:
      httpGet:
        host: 127.0.0.1
        path: /healthz
        port: 10252
      initialDelaySeconds: 15
      timeoutSeconds: 15
    volumeMounts:
    - mountPath: /etc/kubernetes/ssl
      name: ssl-certs-kubernetes
      readOnly: true
    - mountPath: /etc/ssl/certs
      name: ssl-certs-host
      readOnly: true
  volumes:
  - hostPath:
      path: /etc/kubernetes/ssl
    name: ssl-certs-kubernetes
  - hostPath:
      path: /usr/share/ca-certificates
    name: ssl-certs-host
  • 创建/etc/kubernetes/manifests/kube-scheduler.yaml
apiVersion: v1  
kind: Pod  
metadata:  
  name: kube-scheduler
  namespace: kube-system
spec:  
  hostNetwork: true
  containers:
  - name: kube-scheduler
    image: quay.io/coreos/hyperkube:v1.6.1_coreos.0
    command:
    - /hyperkube
    - scheduler
    - --master=http://127.0.0.1:8080
    - --leader-elect=true
    resources:
      requests:
        cpu: 100m
    livenessProbe:
      httpGet:
        host: 127.0.0.1
        path: /healthz
        port: 10251
      initialDelaySeconds: 15
      timeoutSeconds: 15
启动服务
  • Reload
sudo systemctl daemon-reload  
  • 配置flannel
curl -X PUT -d "value={\"Network\":\"$POD_NETWORK\",\"Backend\":{\"Type\":\"vxlan\"}}" "$ETCD_SERVER/v2/keys/coreos.com/network/config"

- POD_NETWORK 私网网段10.2.0.0/16
- ETCD_SERVER 任意一台ETCD服务地址

例如:
curl -X PUT -d "value={\"Network\":\"10.2.0.0/16\",\"Backend\":{\"Type\":\"vxlan\"}}" "http://10.12.1.104:2379/v2/keys/coreos.com/network/config"  
  • 启动Kubelet
sudo systemctl start kubelet

sudo systemctl start kubelet  
  • 验证服务
curl http://127.0.0.1:8080/version

输出应该与下面类似:
{
  "major": "1",
  "minor": "4",
  "gitVersion": "v1.5.2+coreos.0",
  "gitCommit": "ec2b52fabadf824a42b66b6729fe4cff2c62af8c",
  "gitTreeState": "clean",
  "buildDate": "2016-11-14T19:42:00Z",
  "goVersion": "go1.6.3",
  "compiler": "gc",
  "platform": "linux/amd64"
}
curl -s localhost:10255/pods | jq -r '.items[].metadata.name'

输出类似:
kube-scheduler-$node  
kube-apiserver-$node  
kube-controller-$node  
kube-proxy-$node

如果验证成功,那么master节点安装完成。下面开始安装worker节点。