摘自 Kubernetes 从上手到实践 | 掘金小册

目录

1. Kubernetes 基础概念

  • Node:工作节点,可以是物理机或虚拟机,当状态满足要求后达到Ready状态
  • Deployment部署,一种对期望状态的描述
  • Pod:集群中可调度的最小调度单元,可包含多个容器
  • Container Runtime容器运行时,这里默认为 Docker

2. Kubernetes 整体架构

C/S 架构

从宏观上看,K8S 遵循 C/S 架构,可以用下面的图来表示:

                               +-------------+                              
                               |             |                              
                               |             |               +---------------+
                               |             |       +-----> |     Node 1    |
                               | Kubernetes  |       |       +---------------+
+-----------------+            |   Server    |       |                      
|       CLI       |            |             |       |       +---------------+
|    (Kubectl)    |----------->| ( Master )  |<------+-----> |     Node 2    |
|                 |            |             |       |       +---------------+
+-----------------+            |             |       |       
                               |             |       |       +---------------+
                               |             |       +-----> |     Node 3    |
                               |             |               +---------------+
                               +-------------+

Master

Master是整个 K8S 集群的大脑,他有几个重要的功能:

  • 接收外部的请求和集群内部的通知反馈
  • 发布:对集群整体的调度和管理
  • 存储:存储集群所需持久化的状态信息

上述功能通过一些组件来共同完成,我们称其为Control Plane

+----------------------------------------------------------+          
| Master                                                   |          
|              +-------------------------+                 |          
|     +------->|        API Server       |<--------+       |          
|     |        |                         |         |       |          
|     v        +-------------------------+         v       |          
|   +----------------+     ^      +--------------------+   |          
|   |                |     |      |                    |   |          
|   |   Scheduler    |     |      | Controller Manager |   |          
|   |                |     |      |                    |   |          
|   +----------------+     v      +--------------------+   |          
| +------------------------------------------------------+ |          
| |                                                      | |          
| |                Cluster state store                   | |          
| |                                                      | |          
| +------------------------------------------------------+ |          
+----------------------------------------------------------+

Master主要包含以下几个重要的组成部分

1. Cluster state store

用来存储集群所有需要持久化的状态,并且提供watch的功能支持,可以快速的通知各组件的变更等操作

目前 Kubernetes 的存储层选择是etcd,所以一般情况下,我们直接etcd来代表集群状态存储服务,即将所有状态存储到etcd实例中

2. API Server

这是整个集群的入口,类似于人体的感官,接收外部的信号和请求,并将相应的信息写入到etcd

为了保证安全,API Server 还提供了认证相关的功能,用于判断客户端是否有权限进行操作。API Server 支持多种认证方法,不过一般情况下,我们使用x509证书来进行认证

API Server 的目标是成为一个极简的 Server只提供REST操作更新etcd,并充当着集群的网关。至于其他的业务逻辑,则通过插件或者其他组件来实现

3. Controller Manager

Controller Manager 大概是 K8S 集群中最繁忙的部分,它在后台运行着许多不同的控制器进程,用来调节集群的状态

集群的配置发生改变时,控制器就会朝着预期的状态开始工作

4. Scheduler

Scheduler集群的调度器,它会持续关注集群中未被调度的 Pod,并根据资源可用性、节点亲和性或是其他一些限制条件,通过绑定的 API 将 Pod 调度/绑定到 Node 上

在这个过程中,调度程序一般只考虑调度开始时 Node 的状态,而不考虑在调度过程中 Node 的状态变化

Node

Master相对应,可以将Node简单理解为加入集群中的机器,它有以下几个核心组件

+--------------------------------------------------------+       
| +---------------------+        +---------------------+ |       
| |      kubelet        |        |     kube-proxy      | |       
| |                     |        |                     | |       
| +---------------------+        +---------------------+ |       
| +----------------------------------------------------+ |       
| | Container Runtime (Docker)                         | |       
| | +---------------------+    +---------------------+ | |       
| | |Pod                  |    |Pod                  | | |       
| | | +-----+ +-----+     |    |+-----++-----++-----+| | |       
| | | |C1   | |C2   |     |    ||C1   ||C2   ||C3   || | |       
| | | |     | |     |     |    ||     ||     ||     || | |       
| | | +-----+ +-----+     |    |+-----++-----++-----+| | |       
| | +---------------------+    +---------------------+ | |       
| +----------------------------------------------------+ |       
+--------------------------------------------------------+
1. kubelet

Kubelet实现了集群中最重要的关于 Node 和 Pod 的控制功能

K8S 原生的执行模式操作应用程序的容器,基于这种模式,可以让应用程序之间相互隔离,互不影响

此外,由于是操作容器,所以应用程序和主机之间也是相互隔离的,在任何容器运行时(比如 Docker)上都可以部署和运行

K8S 将Pod作为可调度的基本单位Pod可以是一组容器(也可以包含存储卷),它分离开了构建时和部署时的关注点

  • 构建时:重点关注某个容器是否能正确构建,如何快速构建
  • 部署时:关心某个应用程序的服务是否可用,是否符合预期,依赖的相关资源是否都能访问到

这种隔离的模式,可以很方便的将应用程序与底层的基础设施解耦,极大提高了集群扩/缩容、迁移的灵活性

之前提到MasterScheduler组件,它会调度未绑定的 Pod 到符合条件的 Node 上。至于最终该 Pod 是否能运行于 Node 上,则是kubelet来决定

2. Container Runtime

容器运行时最主要的功能是下载镜像运行容器最常见的实现是Docker,目前还有其他一些实现,例如rktcri-o等。

K8S 提供了一套通用的容器运行时接口 CRI (Container Runtime Interface),凡是符合这套标准的容器运行时实现,都可以在 K8S 上使用。

3. kube-proxy

要想访问某个服务,要么通过域名,要么通过 IP 地址。而每个 Pod 在创建后都有一个虚拟 IP,在 K8S 中有一个抽象的概念叫做Servicekube-proxy提供的便是代理的服务,让我们可以通过Service访问到 Pod

实际的工作原理是:K8S 会在每个 Node 上启动一个kube-proxy进程,通过编排iptables规则来达到上述效果

3. 搭建 Kubernetes 集群

略。参见 使用 kubeadm 搭建 Kubernetes 集群 | 苏易北

4. 使用 kubectl 管理集群

首先在终端下执行kubectl

> kubectl
kubectl controls the Kubernetes cluster manager. 

Find more information at: https://kubernetes.io/docs/reference/kubectl/overview/

Basic Commands (Beginner):
  create         Create a resource from a file or from stdin.
  expose         Take a replication controller, service, deployment or pod and expose it as a new Kubernetes Service
  run            Run a particular image on the cluster
  set            Set specific features on objects

Basic Commands (Intermediate):
  explain        Documentation of resources
  get            Display one or many resources
  edit           Edit a resource on the server
  delete         Delete resources by filenames, stdin, resources and names, or by resources and label selector

Deploy Commands:
  rollout        Manage the rollout of a resource
  scale          Set a new size for a Deployment, ReplicaSet, Replication Controller, or Job
  autoscale      Auto-scale a Deployment, ReplicaSet, or ReplicationController

Cluster Management Commands:
  certificate    Modify certificate resources.
  cluster-info   Display cluster info
  top            Display Resource (CPU/Memory/Storage) usage.
  cordon         Mark node as unschedulable
  uncordon       Mark node as schedulable
  drain          Drain node in preparation for maintenance
  taint          Update the taints on one or more nodes

Troubleshooting and Debugging Commands:
  describe       Show details of a specific resource or group of resources
  logs           Print the logs for a container in a pod
  attach         Attach to a running container
  exec           Execute a command in a container
  port-forward   Forward one or more local ports to a pod
  proxy          Run a proxy to the Kubernetes API server
  cp             Copy files and directories to and from containers.
  auth           Inspect authorization

Advanced Commands:
  diff           Diff live version against would-be applied version
  apply          Apply a configuration to a resource by filename or stdin
  patch          Update field(s) of a resource using strategic merge patch
  replace        Replace a resource by filename or stdin
  wait           Experimental: Wait for a specific condition on one or many resources.
  convert        Convert config files between different API versions

Settings Commands:
  label          Update the labels on a resource
  annotate       Update the annotations on a resource
  completion     Output shell completion code for the specified shell (bash or zsh)

Other Commands:
  api-resources  Print the supported API resources on the server
  api-versions   Print the supported API versions on the server, in the form of "group/version"
  config         Modify kubeconfig files
  plugin         Provides utilities for interacting with plugins.
  version        Print the client and server version information

Usage:
  kubectl [flags] [options]

Use "kubectl <command> --help" for more information about a given command.
Use "kubectl options" for a list of global command-line options (applies to all commands).

基础配置

首先来看~/.kube/config配置文件的内容(以minikube为例):

> ls $HOME/.kube/config
/home/tao/.kube/config
> cat $HOME/.kube/config
apiVersion: v1
clusters:
- cluster:
    certificate-authority: /home/tao/.minikube/ca.crt
    server: https://192.168.99.101:8443
  name: minikube
contexts:
- context:
    cluster: minikube
    user: minikube
  name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
  user:
    client-certificate: /home/tao/.minikube/client.crt
    client-key: /home/tao/.minikube/client.key

可以看出,$HOME/.kube/config中主要包含:

  • K8S 集群的 API 地址
  • 用于认证的证书地址

如果想指定配置文件路径,可以使用--kubeconfig或者环境变量KUBECONFIG来传递。

另外如果你并不想使用配置文件的话,也可以在命令行直接传递相关参数来使用:

> kubectl --client-key='/home/tao/.minikube/client.key' --client-certificate='/home/tao/.minikube/client.crt' --server='https://192.168.99.101:8443' get nodes
NAME       STATUS    ROLES     AGE       VERSION
minikube   Ready     master    2d        v1.11.3

kubectl get

使用kubectl get nodes命令查看集群中的所有节点

> kubectl get nodes
NAME             STATUS   ROLES    AGE   VERSION
abelsu7-ubuntu   Ready    master   20d   v1.13.3
centos-1         Ready    <none>   20d   v1.13.3
centos-2         Ready    <none>   20d   v1.13.3

传递-o wide/yaml/json可以得到不同格式的输出

> kubectl get nodes -o wide
NAME             STATUS   ROLES    AGE   VERSION   INTERNAL-IP       EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION               CONTAINER-RUNTIME
abelsu7-ubuntu   Ready    master   20d   v1.13.3   xxx.xxx.xxx.xxx   <none>        Ubuntu 18.04.1 LTS      4.15.0-38-generic            docker://18.6.1
centos-1         Ready    <none>   20d   v1.13.3   xxx.xxx.xxx.xxx   <none>        CentOS Linux 7 (Core)   3.10.0-862.14.4.el7.x86_64   docker://18.9.2
centos-2         Ready    <none>   20d   v1.13.3   xxx.xxx.xxx.xxx   <none>        CentOS Linux 7 (Core)   3.10.0-862.14.4.el7.x86_64   docker://18.9.1

使用-o json将内容以 JSON 格式输出时,可以配合jq进行内容提取,例如:

> kubectl get nodes -o json | jq ".items[] | {name: .metadata.name} + .status.nodeInfo"
{
  "name": "abelsu7-ubuntu",
  "architecture": "amd64",
  "bootID": "efeb7bb0-9c57-40d1-b592-47c0974db9c5",
  "containerRuntimeVersion": "docker://18.6.1",
  "kernelVersion": "4.15.0-38-generic",
  "kubeProxyVersion": "v1.13.3",
  "kubeletVersion": "v1.13.3",
  "machineID": "c7eeb3f409394ad79a08c27afcc8958c",
  "operatingSystem": "linux",
  "osImage": "Ubuntu 18.04.1 LTS",
  "systemUUID": "314AB8CC-38BC-11E6-9C43-BC0000500000"
}
{
  "name": "centos-1",
  "architecture": "amd64",
  "bootID": "f109d499-2dea-45e8-834a-907f78d267bc",
  "containerRuntimeVersion": "docker://18.9.2",
  "kernelVersion": "3.10.0-862.14.4.el7.x86_64",
  "kubeProxyVersion": "v1.13.3",
  "kubeletVersion": "v1.13.3",
  "machineID": "b9d5ec44cf284913b48d1ca1a7662c83",
  "operatingSystem": "linux",
  "osImage": "CentOS Linux 7 (Core)",
  "systemUUID": "E364DF48-5F20-11E6-8BF7-57717FCC0F00"
}
{
  "name": "centos-2",
  "architecture": "amd64",
  "bootID": "8fadfee7-e09b-4ee9-81c2-d5464f60a4c0",
  "containerRuntimeVersion": "docker://18.9.1",
  "kernelVersion": "3.10.0-862.14.4.el7.x86_64",
  "kubeProxyVersion": "v1.13.3",
  "kubeletVersion": "v1.13.3",
  "machineID": "bbe91187ab474caebff29ffc64bcd487",
  "operatingSystem": "linux",
  "osImage": "CentOS Linux 7 (Core)",
  "systemUUID": "A1C63AE4-5F04-11E6-88F2-108D9A211300"
}

此方法可以得到Node基础信息

kubectl api-resources

可以使用kubectl api-resources查看服务端支持的 API 资源及其别名、描述等信息:

> kubectl api-resources
NAME                              SHORTNAMES   APIGROUP                       NAMESPACED   KIND
bindings                                                                      true         Binding
componentstatuses                 cs                                          false        ComponentStatus
configmaps                        cm                                          true         ConfigMap
endpoints                         ep                                          true         Endpoints
events                            ev                                          true         Event
limitranges                       limits                                      true         LimitRange
namespaces                        ns                                          false        Namespace
nodes                             no                                          false        Node
persistentvolumeclaims            pvc                                         true         PersistentVolumeClaim
persistentvolumes                 pv                                          false        PersistentVolume
pods                              po                                          true         Pod
podtemplates                                                                  true         PodTemplate
replicationcontrollers            rc                                          true         ReplicationController
resourcequotas                    quota                                       true         ResourceQuota
secrets                                                                       true         Secret
serviceaccounts                   sa                                          true         ServiceAccount
services                          svc                                         true         Service
mutatingwebhookconfigurations                  admissionregistration.k8s.io   false        MutatingWebhookConfiguration
validatingwebhookconfigurations                admissionregistration.k8s.io   false        ValidatingWebhookConfiguration
customresourcedefinitions         crd,crds     apiextensions.k8s.io           false        CustomResourceDefinition
apiservices                                    apiregistration.k8s.io         false        APIService
controllerrevisions                            apps                           true         ControllerRevision
daemonsets                        ds           apps                           true         DaemonSet
deployments                       deploy       apps                           true         Deployment
replicasets                       rs           apps                           true         ReplicaSet
statefulsets                      sts          apps                           true         StatefulSet
tokenreviews                                   authentication.k8s.io          false        TokenReview
localsubjectaccessreviews                      authorization.k8s.io           true         LocalSubjectAccessReview
selfsubjectaccessreviews                       authorization.k8s.io           false        SelfSubjectAccessReview
selfsubjectrulesreviews                        authorization.k8s.io           false        SelfSubjectRulesReview
subjectaccessreviews                           authorization.k8s.io           false        SubjectAccessReview
horizontalpodautoscalers          hpa          autoscaling                    true         HorizontalPodAutoscaler
cronjobs                          cj           batch                          true         CronJob
jobs                                           batch                          true         Job
certificatesigningrequests        csr          certificates.k8s.io            false        CertificateSigningRequest
leases                                         coordination.k8s.io            true         Lease
events                            ev           events.k8s.io                  true         Event
daemonsets                        ds           extensions                     true         DaemonSet
deployments                       deploy       extensions                     true         Deployment
ingresses                         ing          extensions                     true         Ingress
networkpolicies                   netpol       extensions                     true         NetworkPolicy
podsecuritypolicies               psp          extensions                     false        PodSecurityPolicy
replicasets                       rs           extensions                     true         ReplicaSet
networkpolicies                   netpol       networking.k8s.io              true         NetworkPolicy
poddisruptionbudgets              pdb          policy                         true         PodDisruptionBudget
podsecuritypolicies               psp          policy                         false        PodSecurityPolicy
clusterrolebindings                            rbac.authorization.k8s.io      false        ClusterRoleBinding
clusterroles                                   rbac.authorization.k8s.io      false        ClusterRole
rolebindings                                   rbac.authorization.k8s.io      true         RoleBinding
roles                                          rbac.authorization.k8s.io      true         Role
priorityclasses                   pc           scheduling.k8s.io              false        PriorityClass
storageclasses                    sc           storage.k8s.io                 false        StorageClass
volumeattachments                              storage.k8s.io                 false        VolumeAttachment

kubectl explain

可以使用kubectl explain <API>查看 API 的相应说明

> kubectl explain nodes
KIND:     Node
VERSION:  v1

DESCRIPTION:
     Node is a worker node in Kubernetes. Each node will have a unique
     identifier in the cache (i.e. in etcd).

FIELDS:
   apiVersion    <string>
     APIVersion defines the versioned schema of this representation of an
     object. Servers should convert recognized schemas to the latest internal
     value, and may reject unrecognized values. More info:
     https://git.k8s.io/community/contributors/devel/api-conventions.md#resources

   kind    <string>
     Kind is a string value representing the REST resource this object
     represents. Servers may infer this from the endpoint the client submits
     requests to. Cannot be updated. In CamelCase. More info:
     https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds

   metadata    <Object>
     Standard object's metadata. More info:
     https://git.k8s.io/community/contributors/devel/api-conventions.md#metadata

   spec    <Object>
     Spec defines the behavior of a node.
     https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status

   status    <Object>
     Most recently observed status of the node. Populated by the system.
     Read-only. More info:
     https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status

kubectl run

之前提到,Pod 是 K8s 中最小的调度单元,所以我们无法直接在 K8s 中运行一个 container,但是可以运行一个只包含一个 container 的 Pod

kubectl run基础用法如下:

Usage:
  kubectl run NAME --image=image [--env="key=value"] [--port=port] [--replicas=replicas] [--dry-run=bool] [--overrides=inline-json] [--command] -- [COMMAND] [args...] [options]

NAME--image是必须项,分别代表此次部署的名字所使用的镜像。而在实际使用时,推荐编写配置文件并通过kubectl create进行部署。

例如部署一个Redis实例

> kubectl run redis --image='redis:alpine'
deployment.apps/redis created

可以看到已创建部署deployment.apps/redis created。使用kubectl get all查看发生了什么:

> kubectl get all
NAME                         READY     STATUS    RESTARTS   AGE
pod/redis-7c7545cbcb-2m6rp   1/1       Running   0          30s

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   32s

NAME                    DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/redis   1         1         1            1           30s

NAME                               DESIRED   CURRENT   READY     AGE
replicaset.apps/redis-7c7545cbcb   1         1         1         30s

使用kubectl get all输出内容的格式,/前代表类型/后代表名称

Deployment

Deployment是一种高级别的抽象,允许我们进行扩容、滚动更新降级等操作。我们使用kubectl run redis --image='redis:alpine'命令便创建了一个名为redisDeployment,并指向了其使用的镜像为redis:alpine

同时 K8S 会默认为其增加一些标签Label,可以添加-o wide选项进行查看

> kubectl get deployment.apps/redis -o wide
NAME      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE       CONTAINERS   IMAGES         SELECTOR
redis     1         1         1            1           40s       redis        redis:alpine   run=redis

> kubectl get deploy redis -o wide
NAME      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE       CONTAINERS   IMAGES         SELECTOR
redis     1         1         1            1           40s       redis        redis:alpine   run=redis

可以将这些Label作为选择条件使用:

> kubectl get deploy -l run=redis -o wide
NAME      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE       CONTAINERS   IMAGES         SELECTOR
redis     1         1         1            1           11h       redis        redis:alpine   run=redis

Deployment的创建除了使用上述方式之外,更推荐的方式是使用yaml格式的配置文件。在配置文件中主要是声名一种预期的状态,而其他组件则负责协同调度并最终达成这种预期的状态。最后,Deployment会将Pod托管给下面将要介绍的ReplicaSet

ReplicaSet

ReplicaSet是一种较低级别的结构允许进行扩容

之前提到了Deployment主要是声明一种预期的状态,并且会Pod托管给ReplicaSet,而ReplicaSet则会检查当前的Pod数量及状态是否符合预期,并尽量满足这一预期。

ReplicaSet可简写为rs,通过以下命令查看:

> kubectl get rs -o wide
NAME               DESIRED   CURRENT   READY     AGE       CONTAINERS   IMAGES         SELECTOR                           
redis-7c7545cbcb   1         1         1         11h       redis        redis:alpine   pod-template-hash=3731017676,run=redis

Service

简单来说,Service就是提供稳定访问入口的一组Pod,通过Service可以很方便的实现服务发现负载均衡

> kubectl get service -o wide
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE       SELECTOR
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   16m        <none>

Service目前有 4 种类型

  • ClusterIP目前 K8s 默认的Service类型,将Service暴露于一个仅集群内可访问的虚拟 IP 上
  • NodePort:通过在集群内所有Node上都绑定固定端口的方式将服务暴露出来
  • LoadBalancer:是通过Cloud Provider创建一个外部的负载均衡器,将服务暴露出来,并且会自动创建外部负载均衡器路由请求所需的NodePortClusterIP
  • ExternalName:将服务DNS CNAME的方式转发到指定的域名上将服务暴露出来

kubectl expose

> kubectl expose deploy/redis --port=6379 --protocol=TCP --target-port=6379 --name=redis-server
service/redis-server exposed

> kubectl get svc -o wide
NAME           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE       SELECTOR
kubernetes     ClusterIP   10.96.0.1       <none>        443/TCP    49m       <none>
redis-server   ClusterIP   10.108.105.63   <none>        6379/TCP   4s        run=redis

现在redis-server这个Service使用的是默认类型ClusterIP,所以并不能直接从外部进行访问。需要使用port-forward的方式让它可以在集群外部访问到

> kubectl port-forward svc/redis-server 6379:6379
Forwarding from 127.0.0.1:6379 -> 6379
Forwarding from [::1]:6379 -> 6379
Handling connection for 6379

这样在另一个本地终端上就可以通过redis-cli工具进行连接:

> redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> ping
PONG

当然,也可以使用NodePort方式对外暴露服务

> kubectl expose deploy/redis --port=6379 --protocol=TCP --target-port=6379 --name=redis-server-nodeport --type=NodePort
service/redis-server-nodeport exposed

> kubectl get service/redis-server-nodeport -o wide
NAME                    TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE       SELECTOR
redis-server-nodeport   NodePort   10.109.248.204   <none>        6379:31913/TCP   11s       run=redis

这样就可以通过任意Node上的31913端口访问到redis服务。

更新中…

参考文章

  1. Kubernetes 从上手到实践 | 掘金小册