linuxea: flannel udp不同节点的pod通信

marksugar
2022-06-12 / 0 评论 / 183 阅读 / 正在检测是否收录...
温馨提示:
本文最后更新于2022年06月12日,已超过56天没有更新,若内容或图片失效,请留言反馈。

如下图所示

image-20220226144955250.png

我们先查看下pod的调度情况,分别是10.11.1.45和10.11.0.10在不同的两个节点,并且是不同的网段

  • 不同的网段,那么久需要查找路由表,看路由信息
[root@master1 ~]# kubectl get pod -o wide
NAME                                    READY   STATUS    RESTARTS   AGE   IP           NODE   
marksugar-deployment-578cdd567b-968gg   1/1     Running   4          8d    10.11.1.45   node1  
marksugar-deployment-578cdd567b-fw5rh   1/1     Running   4          8d    10.11.1.48   node1  
marksugar-deployment-578cdd567b-nfhtt   1/1     Running   8          12d   10.11.0.10   master1

里面分别有两个网卡,分别是eth0和lo

  • pod尾968gg开头的mac地址是9e:66:19:aa:f6:c7,ip是10.11.1.45
[root@master1 ~]# kubectl exec -it marksugar-deployment-578cdd567b-968gg  -- ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: eth0@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue state UP group default 
    link/ether 9e:66:19:aa:f6:c7 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.11.1.45/24 brd 10.11.1.255 scope global eth0
       valid_lft forever preferred_lft forever
  • pod尾nfhtt开头的mac地址是ee:a2:33:a2:b6:69,ip是10.11.0.10
[root@master1 ~]# kubectl exec -it marksugar-deployment-578cdd567b-nfhtt  -- ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: eth0@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue state UP group default 
    link/ether ee:a2:33:a2:b6:69 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.11.0.10/24 brd 10.11.0.255 scope global eth0
       valid_lft forever preferred_lft forever

路由匹配原则

我们在上面知道,不同的网段通讯,需要查找路由表,看路由信息,我们就可以查询下路由信息

[root@master1 ~]# kubectl exec -it marksugar-deployment-578cdd567b-968gg  -- route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.11.1.1       0.0.0.0         UG    0      0        0 eth0
10.11.1.0       0.0.0.0         255.255.255.0   U     0      0        0 eth0
10.244.0.0      10.11.1.1       255.255.0.0     UG    0      0        0 eth0
  • 路由匹配是最长匹配原则,大致意思是,越精确的信息优先

路由匹配是最长匹配原则,大致意思是,越精细的信息优先,什么意思呢
如果是一个ip就优先匹配,而后在到0.0.0.0的默认路由,如果有就走,如过没有就丢掉

当我们ping的时候,就会走默认路由,将信息发送到网关,也就是10.11.1.1

要ping通,就需要源ip,目标ip,源mac,目标mac

路由转发前提

路由转发中ip是不变的,mac地址每经过一条都会发送变化
也就是说源ip和目标ip是不发生改变的,但是源mac和目的mac是一直在变的

如上,10.11.1.45的路由表的信息的下一条是10.11.1.1,他们属于同一个网段,因此只需要解析到二层即可。解析的mac地址在宿主机上是ee:e9:19:55:93:d1,如下

7: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue state UP group default qlen 1000
    link/ether ee:e9:19:55:93:d1 brd ff:ff:ff:ff:ff:ff
    inet 10.11.1.1/24 brd 10.11.1.255 scope global cni0
       valid_lft forever preferred_lft forever
    inet6 fe80::ece9:19ff:fe55:93d1/64 scope link 
       valid_lft forever preferred_lft forever

开始抓包

我们进行抓包来查看两个pod的mac地址是不是和我们上面说的那样。

  • 不同节点的pod网络通讯ip不变,而mac地址一直在发生变化

我们在master节点抓nfhtt的包,nfhtt是10.11.0.10的ip,mac地址是ee:a2:33:a2:b6:69

[root@master1 ~]# kubectl exec -it marksugar-deployment-578cdd567b-nfhtt -- tcpdump -n -e -i eth0

我们并且上面开始Ping,ping的是968gg的pod,ip地址是10.11.1.45,mac地址是9e:66:19:aa:f6:c7

[root@master1 ~]# kubectl exec -it marksugar-deployment-578cdd567b-nfhtt -- ping 10.11.1.45
PING 10.11.1.45 (10.11.1.45): 56 data bytes
64 bytes from 10.11.1.45: seq=0 ttl=60 time=0.613 ms
64 bytes from 10.11.1.45: seq=1 ttl=60 time=0.257 ms
64 bytes from 10.11.1.45: seq=2 ttl=60 time=0.206 ms
64 bytes from 10.11.1.45: seq=3 ttl=60 time=0.235 ms

^C
--- 10.11.1.45 ping statistics ---
13 packets transmitted, 13 packets received, 0% packet loss
round-trip min/avg/max = 0.191/0.254/0.613 ms

查看抓包的结果

[root@master1 ~]# kubectl exec -it marksugar-deployment-578cdd567b-nfhtt -- tcpdump -n -e -i eth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
06:56:41.779184 ee:a2:33:a2:b6:69 > a2:a2:7f:a0:0d:91, ethertype IPv4 (0x0800), length 98: 10.11.0.10 >  64
06:56:41.779377 a2:a2:7f:a0:0d:91 > ee:a2:33:a2:b6:69, ethertype IPv4 (0x0800), length 98: 10.11.1.45 > 4
06:56:42.779249 ee:a2:33:a2:b6:69 > a2:a2:7f:a0:0d:91, ethertype IPv4 (0x0800), length 98: 10.11.0.10 >  64
06:56:42.779448 a2:a2:7f:a0:0d:91 > ee:a2:33:a2:b6:69, ethertype IPv4 (0x0800), length 98: 10.11.1.45 > 4
06:56:43.779308 ee:a2:33:a2:b6:69 > a2:a2:7f:a0:0d:91, ethertype IPv4 (0x0800), length 98: 10.11.0.10 >  64
06:56:43.779500 a2:a2:7f:a0:0d:91 > ee:a2:33:a2:b6:69, ethertype IPv4 (0x0800), length 98: 10.11.1.45 > 4
06:56:44.042429 ee:a2:33:a2:b6:69 > a2:a2:7f:a0:0d:91, ethertype ARP (0x0806), length 42: Request who-ha
06:56:44.042437 a2:a2:7f:a0:0d:91 > ee:a2:33:a2:b6:69, ethertype ARP (0x0806), length 42: Request who-ha
06:56:44.042440 ee:a2:33:a2:b6:69 > a2:a2:7f:a0:0d:91, ethertype ARP (0x0806), length 42: Reply 10.11.0.
06:56:44.042463 a2:a2:7f:a0:0d:91 > ee:a2:33:a2:b6:69, ethertype ARP (0x0806), length 42: Reply 10.11.0.
^C
10 packets captured
10 packets received by filter
0 packets dropped by kernel

查看抓包的结果

我们现在在看下cni0的网关的ip和mac地址如下

  • cni0在这里表示的是pod网络的网关

10.11.1.1网段

7: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue state UP group default qlen 1000
    link/ether ee:e9:19:55:93:d1 brd ff:ff:ff:ff:ff:ff
    inet 10.11.1.1/24 brd 10.11.1.255 scope global cni0
       valid_lft forever preferred_lft forever
    inet6 fe80::ece9:19ff:fe55:93d1/64 scope link 
       valid_lft forever preferred_lft forever

和10.11.0.1网段

7: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue state UP group default qlen 1000
    link/ether a2:a2:7f:a0:0d:91 brd ff:ff:ff:ff:ff:ff
    inet 10.11.0.1/24 brd 10.11.0.255 scope global cni0
       valid_lft forever preferred_lft forever
    inet6 fe80::a0a2:7fff:fea0:d91/64 scope link 
       valid_lft forever preferred_lft forever

如上,我们看到抓包结果中两个mac地址是ee:a2:33:a2:b6:69 > a2:a2:7f:a0:0d:91

pod尾968gg开头的mac地址是9e:66:19:aa:f6:c7,ip是10.11.1.45
pod尾nfhtt开头的mac地址是ee:a2:33:a2:b6:69,ip是10.11.0.10

我们从 10.11.0.10 ping 10.11.1.45, 10.11.0.10 的mac地址是ee:a2:33:a2:b6:69 ,10.11.1.45的pod的mac地址是9e:66:19:aa:f6:c7,而抓包走的则是10.11.0.10 的mac地址是ee:a2:33:a2:b6:69和10.11.0.1的cni0网卡的mac地址a2:a2:7f:a0:0d:91,返回的也是a2:a2:7f:a0:0d:91ee:a2:33:a2:b6:69

我们通过pod nfhtt(ee:a2:33:a2:b6:69 10.11.0.10)ping pod为9688GG(9e:66:19:aa:f6:c7 10.11.1.45)ip
而抓包显示,返回的信息的mac地址是10.11.0.1的mac地址。而10.11.0.1是cni0网关

这篇延续上面几篇基础

1

评论

博主关闭了当前页面的评论