linuxea:kubernetes Ingress nginx http以及7层https配置 (17)


在之前的一篇中简单的安装配置了Ingress ControllerIngress Controller概述,唯独缺少后端的配置和https七层的配置,这里涉及到secret,secret存储卷在后面将会提到,先看下如何配置一个ingress backend规则
那么现在,按照之前的配置Ingress Controller nginx提供两个端口,分别是30088和30443作为七层代理,分别代理http和https

I. 代理nginx

  • 配置后端backend pod
    准备一个名称为myapp的service和7个使用Deployment的pod进行测试.这里定义的myapp service后面在ingress的backend会被调用,文件如下:
[root@linuxea ingress]# cat deploy-demt.yaml 
apiVersion: v1
kind: Service
metadata:
  name: myapp
  namespace: default
spec:
  selector:
    app: linuxea_app
    version: v0.1.32
  ports:
  - name: http
    targetPort: 80
    port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: dpment-linuxea
  namespace: default
spec:
  replicas: 7
  selector:
    matchLabels:
      app: linuxea_app
      version: v0.1.32
  template:
    metadata:
      labels:
        app: linuxea_app
        version: v0.1.32
    spec:
      containers:
      - name: nginx-a
        image: marksugar/nginx:1.14.b
        ports:
        - name: http
          containerPort: 80

pod 已经run起来

[root@linuxea deploy]# kubectl get pods
NAME                              READY     STATUS    RESTARTS   AGE
dpment-linuxea-648d599b5f-fxn7s   1/1       Running   0          8m
dpment-linuxea-648d599b5f-lrz4r   1/1       Running   0          8m
dpment-linuxea-648d599b5f-m5p2f   1/1       Running   0          8m
dpment-linuxea-648d599b5f-qhrtf   1/1       Running   0          8m
dpment-linuxea-648d599b5f-tgwnx   1/1       Running   0          8m
dpment-linuxea-648d599b5f-vkcj6   1/1       Running   0          8m
dpment-linuxea-648d599b5f-zccrg   1/1       Running   0          8m

而后get svc

[root@linuxea deploy]# kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP   1h
myapp        ClusterIP   10.106.239.216   <none>        80/TCP    8m

这组后端节点通过部署的ingress向外提供访问,也就是说ingress来接入流量,因此,创建一个ingress的nodePort来接入外部流量

定义ingress backend规则

将myapp通过ingress发布,myapp上面已经准备好了,servicenamemyapp,serviceport是80,host唯一是myapp.linuxea.com
其他:

apiVersion: extensions/v1beta1 : 独特标识,扩展群组
kind: Ingress

注解的填写便于ingress调度,以便匹配到相应使用的规则,如:nginx,则使用nginx模式,生成nignx配置等

  annotations: 
    kubernetes.io/ingress.class: "nginx"
  • 这里用了host,也就说使用了虚拟主机名来代理,那么paths默认是根"/"。如果你会用nginx,你肯定明白了。

如下

[root@linuxea ingress]# cat  myapp-ingress.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: myapp-ingress
  namespace: default
  annotations: 
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: myapp.linuxea.com
    http:
      paths:
      - path: 
        backend:
          serviceName: myapp
          servicePort: 80

稍后,便用这个host的域名来进行访问,要清楚的是,这个域名需要解析到这台ingress的机器上

  • 这里的serviceName: myapp是之前准备好的7台pod节点的创建的service名称,这些pod将作为ingress nginx的代理后端节点

应用

定义完成apply起来

[root@linuxea deploy]# kubectl apply -f myapp-ingress.yaml 

get查看

[root@linuxea deploy]# kubectl get ingress
NAME            HOSTS               ADDRESS   PORTS     AGE
myapp-ingress   myapp.linuxea.com             80        3m

查看详情kubectl describe ingress myapp-ingress,myapp.linuxea.com已经配置

[root@linuxea deploy]# kubectl describe ingress myapp-ingress
Name:             myapp-ingress
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host               Path  Backends
  ----               ----  --------
  myapp.linuxea.com  
                        myapp:80 (<none>)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"myapp-ingress","namespace":"default"},"spec":{"rules":[{"host":"myapp.linuxea.com","http":{"paths":[{"backend":{"serviceName":"myapp","servicePort":80},"path":null}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:                         <none>
  • 一旦apply应用后,配置就会注入到ingress nginx中转换成配置文件
[root@linuxea deploy]# kubectl get pods -n ingress-nginx
NAME                                        READY     STATUS    RESTARTS   AGE
default-http-backend-6586bc58b6-n9qbt       1/1       Running   0          5m
nginx-ingress-controller-6bd7c597cb-krz4m   1/1       Running   0          5m

进入到容器内查看

[root@linuxea deploy]# kubectl exec -n ingress-nginx -it nginx-ingress-controller-6bd7c597cb-krz4m -- /bin/bash

过滤下myapp.linuxea.com

www-data@nginx-ingress-controller-6bd7c597cb-krz4m:/etc/nginx$ grep myapp.linuxea.com nginx.conf
    ## start server myapp.linuxea.com
        server_name myapp.linuxea.com ;
    ## end server myapp.linuxea.com
www-data@nginx-ingress-controller-6bd7c597cb-krz4m:/etc/nginx$ 
  • 外部访问
[root@DS-VM-Node_10_0_1_61 ~]# while true;do for url in myapp.linuxea.com;do curl $url:30088;sleep 1; done; done
linuxea-dpment-linuxea-648d599b5f-qhrtf.com-127.0.0.1/8 172.16.4.211/24
linuxea-dpment-linuxea-648d599b5f-fxn7s.com-127.0.0.1/8 172.16.3.7/24
linuxea-dpment-linuxea-648d599b5f-tgwnx.com-127.0.0.1/8 172.16.5.74/24
linuxea-dpment-linuxea-648d599b5f-fxn7s.com-127.0.0.1/8 172.16.3.7/24
linuxea-dpment-linuxea-648d599b5f-vkcj6.com-127.0.0.1/8 172.16.3.6/24
linuxea-dpment-linuxea-648d599b5f-tgwnx.com-127.0.0.1/8 172.16.5.74/24
linuxea-dpment-linuxea-648d599b5f-qhrtf.com-127.0.0.1/8 172.16.4.211/24
linuxea-dpment-linuxea-648d599b5f-tgwnx.com-127.0.0.1/8 172.16.5.74/24
linuxea-dpment-linuxea-648d599b5f-fxn7s.com-127.0.0.1/8 172.16.3.7/24
linuxea-dpment-linuxea-648d599b5f-vkcj6.com-127.0.0.1/8 172.16.3.6/24
linuxea-dpment-linuxea-648d599b5f-fxn7s.com-127.0.0.1/8 172.16.3.7/24
linuxea-dpment-linuxea-648d599b5f-m5p2f.com-127.0.0.1/8 172.16.4.210/24
linuxea-dpment-linuxea-648d599b5f-lrz4r.com-127.0.0.1/8 172.16.5.73/24
linuxea-dpment-linuxea-648d599b5f-vkcj6.com-127.0.0.1/8 172.16.3.6/24
linuxea-dpment-linuxea-648d599b5f-m5p2f.com-127.0.0.1/8 172.16.4.210/24
linuxea-dpment-linuxea-648d599b5f-m5p2f.com-127.0.0.1/8 172.16.4.210/24
linuxea-dpment-linuxea-648d599b5f-zccrg.com-127.0.0.1/8 172.16.5.72/24
linuxea-dpment-linuxea-648d599b5f-lrz4r.com-127.0.0.1/8 172.16.5.73/24
linuxea-dpment-linuxea-648d599b5f-zccrg.com-127.0.0.1/8 172.16.5.72/24
linuxea-dpment-linuxea-648d599b5f-vkcj6.com-127.0.0.1/8 172.16.3.6/24
linuxea-dpment-linuxea-648d599b5f-m5p2f.com-127.0.0.1/8 172.16.4.210/24
linuxea-dpment-linuxea-648d599b5f-qhrtf.com-127.0.0.1/8 172.16.4.211/24
linuxea-dpment-linuxea-648d599b5f-lrz4r.com-127.0.0.1/8 172.16.5.73/24
linuxea-dpment-linuxea-648d599b5f-qhrtf.com-127.0.0.1/8 172.16.4.211/24
linuxea-dpment-linuxea-648d599b5f-tgwnx.com-127.0.0.1/8 172.16.5.74/24
linuxea-dpment-linuxea-648d599b5f-zccrg.com-127.0.0.1/8 172.16.5.72/24

II. 代理httpd

创建7个pod,并且创建一个名称为linuxea-shop-backend的service

[root@linuxea ingress]# cat httpd.yaml
apiVersion: v1
kind: Service
metadata:
  name: linuxea-shop-backend
  namespace: default
spec:
  selector:
    app: linuxea-shopapp
    version: v3.2
  ports:
  - name: http
    targetPort: 80
    port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: linuxea-backend-group
  namespace: default
spec:
  replicas: 7
  selector:
    matchLabels:
      app: linuxea-shopapp
      version: v3.2
  template:
    metadata:
      labels:
        app: linuxea-shopapp
        version: v3.2
    spec:
      containers:
      - name: linuxea-shopapp
        image: httpd:2.4.34-alpine
        ports:
        - name: http
          containerPort: 80
[root@linuxea ingress]# kubectl apply -f httpd.yaml 
service/linuxea-shop-backend created
deployment.apps/linuxea-backend-group created
[root@linuxea ingress]# kubectl get pods -l version=v3.2
NAME                                     READY     STATUS    RESTARTS   AGE
linuxea-backend-group-7fb757ff95-88tzq   1/1       Running   0          37s
linuxea-backend-group-7fb757ff95-9jkhf   1/1       Running   0          37s
linuxea-backend-group-7fb757ff95-br4d8   1/1       Running   0          37s
linuxea-backend-group-7fb757ff95-cqjxm   1/1       Running   0          37s
linuxea-backend-group-7fb757ff95-kmlnb   1/1       Running   0          37s
linuxea-backend-group-7fb757ff95-lfjvr   1/1       Running   0          37s
linuxea-backend-group-7fb757ff95-vrlb5   1/1       Running   0          37s

而后验证下端口是否启动kubectl exec linuxea-backend-group-7fb757ff95-lfjvr -- netstat -tnl

[root@linuxea ingress]#  kubectl exec linuxea-backend-group-7fb757ff95-lfjvr -- netstat -tnl
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       
tcp        0      0 :::80                   :::*                    LISTEN   

httpd-ingress

将创建的service添加到ingress的backend的serviceName里,配置hosts域名

[root@linuxea ingress]# cat httpd-ingress.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: httpd-ingress
  namespace: default
  annotations: 
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: shop.linuxea.com
    http:
      paths:
      - path: 
        backend:
          serviceName: linuxea-shop-backend
          servicePort: 80
[root@linuxea ingress]# kubectl apply -f httpd-ingress.yaml 
ingress.extensions/httpd-ingress created

这里在service中linuxea-shop-backend已经创建,端口已经被映射

[root@linuxea ingress]# kubectl get svc
NAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes             ClusterIP   10.96.0.1        <none>        443/TCP   7h
linuxea-shop-backend   ClusterIP   10.100.237.244   <none>        80/TCP    1m
myapp                  ClusterIP   10.101.103.203   <none>        80/TCP    4m

ingress 也创建成功

[root@linuxea ingress]#  kubectl get ingress
NAME            HOSTS               ADDRESS   PORTS     AGE
httpd-ingress   shop.linuxea.com              80        33s
myapp-ingress   myapp.linuxea.com             80        8m

而后使用kubectl describe ingress tomcat-linuxea查看详情

[root@linuxea ingress]# kubectl describe ingress httpd-ingress
Name:             httpd-ingress
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host              Path  Backends
  ----              ----  --------
  shop.linuxea.com  
                       linuxea-shop-backend:80 (<none>)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"httpd-ingress","namespace":"default"},"spec":{"rules":[{"host":"shop.linuxea.com","http":{"paths":[{"backend":{"serviceName":"linuxea-shop-backend","servicePort":80},"path":null}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:
  Type    Reason  Age   From                      Message
  ----    ------  ----  ----                      -------
  Normal  CREATE  1m    nginx-ingress-controller  Ingress default/httpd-ingress
  • 从集群外部访问
[root@DS-VM-Node_10_0_1_61 ~]# while true;do curl shop.linuxea.com:30088;sleep 1;done
<html><body><h1>It works!</h1></body></html>
<html><body><h1>It works!</h1></body></html>
<html><body><h1>It works!</h1></body></html>
<html><body><h1>It works!</h1></body></html>
<html><body><h1>It works!</h1></body></html>
<html><body><h1>It works!</h1></body></html>
<html><body><h1>It works!</h1></body></html>
<html><body><h1>It works!</h1></body></html>

III. 代理tomcat

创建7个tomcat镜像的pod,并且创建名称为linuxea-tomcat的serivce

[root@linuxea ingress]# cat tomcat.yaml 
apiVersion: v1
kind: Service
metadata:
  name: linuxea-tomcat
  namespace: default
spec:
  selector:
    app: linuxea-tomcat
    version: v3.2
  ports:
  - name: http
    targetPort: 8080
    port: 8080
  - name: ajp
    targetPort: 8009
    port: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: linuxea-tomcat-group
  namespace: default
spec:
  replicas: 7
  selector:
    matchLabels:
      app: linuxea-tomcat
      version: v3.2
  template:
    metadata:
      labels:
        app: linuxea-tomcat
        version: v3.2
    spec:
      containers:
      - name: linuxea-tomcat
        image: tomcat:9.0.12-jre8-alpine
        ports:
        - name: http
          containerPort: 8080
        - name: ajp
          containerPort: 8009
[root@linuxea ingress]# kubectl apply -f tomcat.yaml
[root@linuxea ingress]# kubectl get pods -l app=linuxea-tomcat
NAME                                   READY     STATUS    RESTARTS   AGE
linuxea-tomcat-group-b77666d76-4jmjh   1/1       Running   0          30s
linuxea-tomcat-group-b77666d76-4pbn2   1/1       Running   0          30s
linuxea-tomcat-group-b77666d76-56fvr   1/1       Running   0          30s
linuxea-tomcat-group-b77666d76-6vph2   1/1       Running   0          30s
linuxea-tomcat-group-b77666d76-8r8qg   1/1       Running   0          30s
linuxea-tomcat-group-b77666d76-h6nfd   1/1       Running   0          30s
linuxea-tomcat-group-b77666d76-rv74d   1/1       Running   0          30s

tomcat-ingress

创建完成后仍然需要修改关键的hosts,backend。这里的servicePort端口是pod内应用端口

[root@linuxea ingress]# cat tomcat-ingress.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: tomcat-ingress
  namespace: default
  annotations: 
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: tomcat.linuxea.com
    http:
      paths:
      - path: 
        backend:
          serviceName: linuxea-tomcat
          servicePort: 8080
[root@linuxea ingress]# kubectl apply -f tomcat-ingress.yaml 
ingress.extensions/tomcat-ingress created
[root@linuxea ingress]# kubectl get ingress
NAME             HOSTS                ADDRESS   PORTS     AGE
httpd-ingress    shop.linuxea.com               80        8m
myapp-ingress    myapp.linuxea.com              80        15m
tomcat-ingress   tomcat.linuxea.com             80        12s
[root@linuxea ingress]# kubectl describe ingress  tomcat-ingress
Name:             tomcat-ingress
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host                Path  Backends
  ----                ----  --------
  tomcat.linuxea.com  
                         linuxea-tomcat:8080 (<none>)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"tomcat-ingress","namespace":"default"},"spec":{"rules":[{"host":"tomcat.linuxea.com","http":{"paths":[{"backend":{"serviceName":"linuxea-tomcat","servicePort":8080},"path":null}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:
  Type    Reason  Age   From                      Message
  ----    ------  ----  ----                      -------
  Normal  CREATE  26s   nginx-ingress-controller  Ingress default/tomcat-ingress
[root@linuxea ingress]# 

外部访问

IV. https

直接自签一个证书,来测试tomcat的https用法

[root@linuxea ingress]# openssl genrsa -out linuxea.key 2048 
Generating RSA private key, 2048 bit long modulus
...........................+++
..............................................+++
e is 65537 (0x10001)
[root@linuxea ingress]# openssl req -new -x509 -key linuxea.key -out linuxea.crt -subj /C=PH/ST=Manila/L=Pasa/O=DevOps/CN=tomcat.linuxea.com

转换格式,创建secret存储卷

[root@linuxea ingress]# kubectl create secret tls tomcat-ingress-secret --cert=linuxea.crt --key=linuxea.key
secret/tomcat-ingress-secret created
[root@linuxea ingress]# kubectl get secret
NAME                    TYPE                                  DATA      AGE
default-token-k25gj     kubernetes.io/service-account-token   3         8h
tomcat-ingress-secret   kubernetes.io/tls                     2         23s

kubectl describe secret tomcat-ingress-secret这些信息由特殊的格式进行"隐藏起来",base64编码

[root@linuxea ingress]# kubectl describe secret tomcat-ingress-secret
Name:         tomcat-ingress-secret
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  kubernetes.io/tls

Data
====
tls.crt:  1285 bytes
tls.key:  1679 bytes
[root@linuxea ingress]# 

在spec.tls中有secretName字段和hosts字段来设置

  • 创建tls.yaml文件
[root@linuxea ingress]# cat tomcat-tls.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: tomcat-ingress
  namespace: default
  annotations: 
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - tomcat.linuxea.com
    secretName: tomcat-ingress-secret
  rules:
  - host: tomcat.linuxea.com
    http:
      paths:
      - path: 
        backend:
          serviceName: linuxea-tomcat
          servicePort: 8080
[root@linuxea ingress]# kubectl apply -f tomcat-tls.yaml 
ingress.extensions/tomcat-ingress configured
[root@linuxea ingress]# kubectl get ingress
NAME             HOSTS                ADDRESS   PORTS     AGE
httpd-ingress    shop.linuxea.com               80        33m
myapp-ingress    myapp.linuxea.com              80        41m
tomcat-ingress   tomcat.linuxea.com             80, 443   25m
[root@linuxea ingress]# kubectl describe ingress tomcat-ingress

当apply启动后,配置会注入到ingress nginx中,配置文件就会发生改变,tls文件已经加入到nginx配置文件中,如下:

www-data@nginx-ingress-controller-6bd7c597cb-krz4m:/etc/nginx$ grep tomcat nginx.conf
    ## start server tomcat.linuxea.com
        server_name tomcat.linuxea.com ;
        ssl_certificate                         /etc/ingress-controller/ssl/default-tomcat-ingress-secret.pem;
        ssl_certificate_key                     /etc/ingress-controller/ssl/default-tomcat-ingress-secret.pem;
        ssl_trusted_certificate                 /etc/ingress-controller/ssl/default-tomcat-ingress-secret-full-chain.pem;
            set $ingress_name   "tomcat-ingress";
            set $service_name   "linuxea-tomcat";
            set $proxy_upstream_name "default-linuxea-tomcat-8080";
    ## end server tomcat.linuxea.com
www-data@nginx-ingress-controller-6bd7c597cb-krz4m:/etc/nginx$ 

通过浏览器访问即可(这里访问的是映射的443端口,也就是30443)

0 分享

您可以选择一种方式赞助本站

支付宝扫码赞助

支付宝扫码赞助

日期: 2018-10-03分类: kubernetes

标签: kubernetes

发表评论