网站首页 > 技术教程 正文
Netronome网卡Firewall设计
本文Netronome Firewall主要是验证Netronome智能网卡P4功能，以Firewall功能特性实现简单的ACL防火墙功能，整体上分为数据平面和控制平面。
如上图所示，数据平面使用P4设计了基于v1model架构的pipeline，控制平面基于ONOS控制器，由于Netronome提供nfp-sdk6-rte对P4runtime支持较差(有人验证Ternary matching和Packet-Out都不支持，具体支持个人也没验证过，不过nfp-sdk版本多年不更新...)，这里基于nfp-sdk6-rte中的rtecli命令行在ONOS中包了个南向协议，并且加了个驱动来纳管Netronome智能网卡，实现了基本的设备连接、pipeline配置和流表增删功能。
这里驱动没有按ONOS规范设计，仅简化为了满足ONOS纳管设备要求，其实按驱动行为规范实现，我这里图简便简化处理了，也有人在智能网卡侧跑一个代理将Openflow或P4runtime转成rtecli来管理，实现复杂了点，我没采用该方案。
Firewall测试直接用netns隔离vf，并配置ip，实现L2互通，然后配置ACL来验证，更复杂的组网场景理论上结合智能网卡的真实物理口(p0&p4)和丰富的vf接口和pipeline中fwd表项或对pipeline重新设计也能得到应用。
数据面设计
数据面基于v1model架构，设计包含两个表的P4 Pipeline: t_acl -> t_fwd，先执行t_acl表判断流量是否满足acl规则，再执行t_fwd表进行转发(目前仅实现端口转发和二层转发)，具体局部table设计如下:
- acl表
direct_counter(CounterType.packets_and_bytes) t_acl_counter;
action allow() {
t_acl_counter.count();
}
action deny() {
mark_to_drop();
t_acl_counter.count();
}
action nop() {
t_acl_counter.count();
}
// acl table
table t_acl {
key = {
standard_metadata.ingress_port : ternary;
hdr.ethernet.src_addr : ternary;
hdr.ethernet.dst_addr : ternary;
hdr.ethernet.ether_type : ternary;
hdr.ipv4.src_addr : ternary;
hdr.ipv4.dst_addr : ternary;
hdr.ipv4.protocol : ternary;
hdr.tcp.src_port : ternary;
hdr.tcp.dst_port : ternary;
hdr.udp.src_port : ternary;
hdr.udp.dst_port : ternary;
}
actions = {
allow;
deny;
nop;
}
size = 1024;
default_action = nop();
counters = t_acl_counter;
}- fwd表
direct_counter(CounterType.packets_and_bytes) t_fwd_counter;
action fwd(port_t port) {
standard_metadata.egress_spec = port;
t_fwd_counter.count();
}
table t_fwd {
key = {
standard_metadata.ingress_port : ternary;
hdr.ethernet.dst_addr : ternary;
}
actions = {
fwd;
}
default_action = fwd(0);
counters = t_fwd_counter;
size = 1024;
}控制面设计
控制面基于ONOS，实现基本的设备纳管，Pipeline配置和流表增删功能，主要包括：rtecli南向协议、rtecli驱动和实现Firewall的app，如下几个模块的实现描述.
- rtecli南向协议
rtecli南向协议通过将rtecli命令行包装了一下，提供基本的design load(pipeline配置)、设备连接、流表增删等功能.
- rtecli驱动
rtecli驱动基于onos中驱动行为机制，编写了rtecli驱动使用的xml并实现基本的驱动行为，实现基本的Handshaker.
- Firewall app
Firewall onos app为标准的ONOS app，onos版本基于2.4.0，提供onos cli命令对ACL防火墙策略和端口/L2转发配置提供命令行(后面可能会增加北向接口和UI)，Firewall服务提供核心的ACL防火墙流表和端口/L2转发流表实现(通过rtecli控制智能网卡，将acl策略和fwd表下发/删除流表).
nfp-sdk6-rte启动
# mod reload
sudo modprobe -r -v nfp && sudo modprobe nfp nfp_pf_netdev=0 nfp_dev_cpp=1
# start nfp-sdk6-rte
sudo systemctl start nfp-sdk6-rte
# check nfp-sdk6-rte status
sudo systemctl status nfp-sdk6-rte通过ip a已经可以看到默认创建的4个vf接口，可启动nfp-sdk6-rte设置NUM_VFS环境变量来设置启动vf数目
10: vf0_0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether d6:61:9c:56:1c:92 brd ff:ff:ff:ff:ff:ff
11: vf0_1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether fe:a2:d2:49:f6:cd brd ff:ff:ff:ff:ff:ff
12: vf0_2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 26:1d:8e:b3:36:13 brd ff:ff:ff:ff:ff:ff
13: vf0_3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 6a:ec:a4:ba:13:f4 brd ff:ff:ff:ff:ff:ffonos & firewall app启动
# install nfp sdk, 由于onos南向通过rtecli来和智能网卡通信，所以也要安装nfp-sdk
sudo dpkg -i nfp-sdk_6.1.0.1-preview-3243-2_amd64.deb
# onos 启动
make onos-run
# firewall app编译
make app-build
# firewall app加载
make app-reloadnetcfg配置
make netcfg-firewallfirewall-app/src/main/resources/nfp_nics.json配置文件：
{
"apps": {
"cn.pcl.firewall": {
"nfp_nics": [
{
"dp_id": "rtecli:nfp-nic1",
"driver": "rtecli",
"rte_host": "172.16.30.15",
"rte_port": "20206",
"nffw_path": "path of firewall.nffw",
"design_path": "path of pif_design.json",
"p4cfg_path": ""
}
]
}
}
}
vf接口netns隔离
sudo ip netns add ns0
sudo ip netns add ns1
sudo ip netns add ns2
sudo ip netns add ns3
sudo ip link set vf0_0 netns ns0
sudo ip link set vf0_1 netns ns1
sudo ip link set vf0_2 netns ns2
sudo ip link set vf0_3 netns ns3配置ip及checksum
# 配置ip
sudo ip netns exec ns0 /bin/bash
ifconfig vf0_0 192.168.10.5 netmask 255.255.255.0 up
# checksum off
ethtool -K vf0_0 tx off
ethtool -K vf0_0 rx off# 配置ip
sudo ip netns exec ns1 /bin/bash
ifconfig vf0_0 192.168.10.10 netmask 255.255.255.0 up
# checksum off
ethtool -K vf0_1 tx off
ethtool -K vf0_1 rx off测试
L2互通
# login onos cli
make onos-cli
# apply fwd flow rules
fwd -d=00:15:4d:00:00:00 add rtecli:nfp-nic1 fwd 768
fwd -d=00:15:4d:00:00:01 add rtecli:nfp-nic1 fwd 769
# show fwd flow rules
fwd show rtecli:nfp-nic1 09:54:46
===================================================================
| ID | MATCH | ACTION |
===================================================================
| ID | IngressPort | DstMac | Action | Output |
-------------------------------------------------------------------
| FWD-1000 | | 00:15:4d:00:00:00 | fwd | 768 |
| FWD-1001 | | 00:15:4d:00:00:01 | fwd | 769 |
-------------------------------------------------------------------- icmp
sudo ip netns exec ns0 /bin/bash
ping 192.168.10.10PING 192.168.10.10 (192.168.10.10) 56(84) bytes of data.
64 bytes from 192.168.10.10: icmp_seq=1 ttl=64 time=0.273 ms
64 bytes from 192.168.10.10: icmp_seq=2 ttl=64 time=0.220 ms
64 bytes from 192.168.10.10: icmp_seq=3 ttl=64 time=0.215 ms
64 bytes from 192.168.10.10: icmp_seq=4 ttl=64 time=0.212 ms
64 bytes from 192.168.10.10: icmp_seq=5 ttl=64 time=0.209 ms
64 bytes from 192.168.10.10: icmp_seq=6 ttl=64 time=0.213 mssudo ip netns exec ns1 /bin/bash
ping 192.168.10.5PING 192.168.10.5 (192.168.10.5) 56(84) bytes of data.
64 bytes from 192.168.10.5: icmp_seq=1 ttl=64 time=0.252 ms
64 bytes from 192.168.10.5: icmp_seq=2 ttl=64 time=0.218 ms
64 bytes from 192.168.10.5: icmp_seq=3 ttl=64 time=0.212 ms
64 bytes from 192.168.10.5: icmp_seq=4 ttl=64 time=0.213 ms
64 bytes from 192.168.10.5: icmp_seq=5 ttl=64 time=0.213 ms
64 bytes from 192.168.10.5: icmp_seq=6 ttl=64 time=0.208 ms
64 bytes from 192.168.10.5: icmp_seq=7 ttl=64 time=0.212 ms- tcp
sudo ip netns exec ns1 /bin/bash
iperf -s -i 1sudo ip netns exec ns0 /bin/bash
iperf -c 192.168.10.10 -i 1------------------------------------------------------------
Client connecting to 192.168.10.10, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.10.5 port 35252 connected with 192.168.10.10 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 958 MBytes 8.03 Gbits/sec
[ 3] 1.0- 2.0 sec 972 MBytes 8.15 Gbits/sec
[ 3] 2.0- 3.0 sec 934 MBytes 7.83 Gbits/sec
[ 3] 3.0- 4.0 sec 938 MBytes 7.87 Gbits/sec
[ 3] 4.0- 5.0 sec 926 MBytes 7.77 Gbits/sec
[ 3] 5.0- 6.0 sec 927 MBytes 7.78 Gbits/sec
[ 3] 6.0- 7.0 sec 918 MBytes 7.70 Gbits/sec
[ 3] 7.0- 8.0 sec 921 MBytes 7.73 Gbits/sec
[ 3] 8.0- 9.0 sec 913 MBytes 7.66 Gbits/sec
[ 3] 0.0-10.0 sec 9.12 GBytes 7.83 Gbits/sec
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 128 KByte (default)
------------------------------------------------------------
[ 4] local 192.168.10.10 port 5001 connected with 192.168.10.5 port 35252
[ ID] Interval Transfer Bandwidth
[ 4] 0.0- 1.0 sec 954 MBytes 8.01 Gbits/sec
[ 4] 1.0- 2.0 sec 973 MBytes 8.16 Gbits/sec
[ 4] 2.0- 3.0 sec 933 MBytes 7.83 Gbits/sec
[ 4] 3.0- 4.0 sec 939 MBytes 7.88 Gbits/sec
[ 4] 4.0- 5.0 sec 927 MBytes 7.78 Gbits/sec
[ 4] 5.0- 6.0 sec 925 MBytes 7.76 Gbits/sec
[ 4] 6.0- 7.0 sec 918 MBytes 7.70 Gbits/sec
[ 4] 7.0- 8.0 sec 921 MBytes 7.73 Gbits/sec
[ 4] 8.0- 9.0 sec 913 MBytes 7.66 Gbits/sec
[ 4] 9.0-10.0 sec 927 MBytes 7.78 Gbits/sec
[ 4] 0.0-10.0 sec 9.12 GBytes 7.83 Gbits/sec- udp
sudo ip netns exec ns1 /bin/bash
iperf -s -i 1 -usudo ip netns exec ns0 /bin/bash
iperf -c 192.168.10.10 -i 1 -u------------------------------------------------------------
Client connecting to 192.168.10.10, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size: 208 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.10.5 port 35162 connected with 192.168.10.10 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 129 KBytes 1.06 Mbits/sec
[ 3] 1.0- 2.0 sec 128 KBytes 1.05 Mbits/sec
[ 3] 2.0- 3.0 sec 128 KBytes 1.05 Mbits/sec
[ 3] 3.0- 4.0 sec 128 KBytes 1.05 Mbits/sec
[ 3] 4.0- 5.0 sec 128 KBytes 1.05 Mbits/sec
[ 3] 5.0- 6.0 sec 128 KBytes 1.05 Mbits/sec
[ 3] 6.0- 7.0 sec 129 KBytes 1.06 Mbits/sec
[ 3] 7.0- 8.0 sec 128 KBytes 1.05 Mbits/sec
[ 3] 8.0- 9.0 sec 128 KBytes 1.05 Mbits/sec
[ 3] 9.0-10.0 sec 128 KBytes 1.05 Mbits/sec
[ 3] 0.0-10.0 sec 1.25 MBytes 1.05 Mbits/sec
[ 3] Sent 893 datagrams
[ 3] Server Report:
[ 3] 0.0-10.0 sec 1.25 MBytes 1.05 Mbits/sec 0.001 ms 0/ 893 (0%)------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size: 208 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.10.10 port 5001 connected with 192.168.10.5 port 35162
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0- 1.0 sec 128 KBytes 1.05 Mbits/sec 0.003 ms 0/ 89 (0%)
[ 3] 1.0- 2.0 sec 128 KBytes 1.05 Mbits/sec 0.002 ms 0/ 89 (0%)
[ 3] 2.0- 3.0 sec 128 KBytes 1.05 Mbits/sec 0.002 ms 0/ 89 (0%)
[ 3] 3.0- 4.0 sec 128 KBytes 1.05 Mbits/sec 0.001 ms 0/ 89 (0%)
[ 3] 4.0- 5.0 sec 128 KBytes 1.05 Mbits/sec 0.001 ms 0/ 89 (0%)
[ 3] 5.0- 6.0 sec 129 KBytes 1.06 Mbits/sec 0.001 ms 0/ 90 (0%)
[ 3] 6.0- 7.0 sec 128 KBytes 1.05 Mbits/sec 0.002 ms 0/ 89 (0%)
[ 3] 7.0- 8.0 sec 128 KBytes 1.05 Mbits/sec 0.001 ms 0/ 89 (0%)
[ 3] 8.0- 9.0 sec 128 KBytes 1.05 Mbits/sec 0.002 ms 0/ 89 (0%)
[ 3] 9.0-10.0 sec 128 KBytes 1.05 Mbits/sec 0.001 ms 0/ 89 (0%)
[ 3] 0.0-10.0 sec 1.25 MBytes 1.05 Mbits/sec 0.001 ms 0/ 893 (0%)ACL防火墙
场景1
# login onos cli
make onos-cli
# apply acl flow rules
acl -t=ipv4 --ipDst=192.168.10.10 add rtecli:nfp-nic1 deny
# show acl flow rules
acl show rtecli:nfp-nic1
========================================================================================================================================================
| ID | MATCH | ACTION |
========================================================================================================================================================
| ID | IngressPort | SrcMac | DstMac | EthType | Protocol | SrcIp | DstIp | SrcPort | DstPort | Action |
--------------------------------------------------------------------------------------------------------------------------------------------------------
| ACL-1000 | | | | ipv4 | | | 192.168.10.10 | | | deny |
--------------------------------------------------------------------------------------------------------------------------------------------------------- imcp
ping 192.168.10.10
PING 192.168.10.10 (192.168.10.10) 56(84) bytes of data.
^C
--- 192.168.10.10 ping statistics ---
11 packets transmitted, 0 received, 100% packet loss, time 10218msping 192.168.10.5
PING 192.168.10.5 (192.168.10.5) 56(84) bytes of data.
^C
--- 192.168.10.5 ping statistics ---
9 packets transmitted, 0 received, 100% packet loss, time 8186ms- tcp
sudo ip netns exec ns1 /bin/bash
iperf -s -i 1sudo ip netns exec ns0 /bin/bash
iperf -c 192.168.10.10 -i 1# iperf client
connect failed: Connection timed out
# iperf server 看不到client端连接日志- udp
sudo ip netns exec ns1 /bin/bash
iperf -s -i 1 -usudo ip netns exec ns0 /bin/bash
iperf -c 192.168.10.10 -i 1 -u# iperf client 正常发包(udp无连接)
# iperf server 看不到client端连接日志场景2
# login onos cli
make onos-cli
acl -t=ipv4 --protocol=icmp --ipDst=192.168.10.10 add rtecli:nfp-nic1 deny
acl show rtecli:nfp-nic1
========================================================================================================================================================
| ID | MATCH | ACTION |
========================================================================================================================================================
| ID | IngressPort | SrcMac | DstMac | EthType | Protocol | SrcIp | DstIp | SrcPort | DstPort | Action |
--------------------------------------------------------------------------------------------------------------------------------------------------------
| ACL-1000 | | | | ipv4 | icmp | | 192.168.10.10 | | | deny |
--------------------------------------------------------------------------------------------------------------------------------------------------------- icmp
ping 192.168.10.10
PING 192.168.10.10 (192.168.10.10) 56(84) bytes of data.
^C
--- 192.168.10.10 ping statistics ---
11 packets transmitted, 0 received, 100% packet loss, time 10235msping 192.168.10.5
PING 192.168.10.5 (192.168.10.5) 56(84) bytes of data.
^C
--- 192.168.10.5 ping statistics ---
9 packets transmitted, 0 received, 100% packet loss, time 8186ms- tcp
iperf -c 192.168.10.10 -i 1
------------------------------------------------------------
Client connecting to 192.168.10.10, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.10.5 port 35256 connected with 192.168.10.10 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 883 MBytes 7.41 Gbits/sec
[ 3] 1.0- 2.0 sec 917 MBytes 7.69 Gbits/sec
[ 3] 2.0- 3.0 sec 918 MBytes 7.70 Gbits/sec
[ 3] 3.0- 4.0 sec 915 MBytes 7.67 Gbits/sec
[ 3] 4.0- 5.0 sec 921 MBytes 7.72 Gbits/sec
[ 3] 5.0- 6.0 sec 910 MBytes 7.63 Gbits/sec
[ 3] 6.0- 7.0 sec 926 MBytes 7.77 Gbits/sec
[ 3] 7.0- 8.0 sec 919 MBytes 7.71 Gbits/sec
[ 3] 8.0- 9.0 sec 920 MBytes 7.72 Gbits/sec
[ 3] 9.0-10.0 sec 906 MBytes 7.60 Gbits/sec
[ 3] 0.0-10.0 sec 8.92 GBytes 7.66 Gbits/sec
iperf -s -i 1
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 128 KByte (default)
------------------------------------------------------------
[ 4] local 192.168.10.10 port 5001 connected with 192.168.10.5 port 35256
[ ID] Interval Transfer Bandwidth
[ 4] 0.0- 1.0 sec 881 MBytes 7.39 Gbits/sec
[ 4] 1.0- 2.0 sec 916 MBytes 7.68 Gbits/sec
[ 4] 2.0- 3.0 sec 918 MBytes 7.70 Gbits/sec
[ 4] 3.0- 4.0 sec 916 MBytes 7.68 Gbits/sec
[ 4] 4.0- 5.0 sec 919 MBytes 7.71 Gbits/sec
[ 4] 5.0- 6.0 sec 911 MBytes 7.64 Gbits/sec
[ 4] 6.0- 7.0 sec 925 MBytes 7.76 Gbits/sec
[ 4] 7.0- 8.0 sec 920 MBytes 7.72 Gbits/sec
[ 4] 8.0- 9.0 sec 921 MBytes 7.72 Gbits/sec
[ 4] 9.0-10.0 sec 906 MBytes 7.60 Gbits/sec
[ 4] 0.0-10.0 sec 8.92 GBytes 7.66 Gbits/sec场景3
# login onos cli
make onos-cli
acl -t=ipv4 --protocol=tcp --ipDst=192.168.10.10 --tpDst=5001 add rtecli:nfp-nic1 deny
acl show rtecli:nfp-nic1 09:32:35
========================================================================================================================================================
| ID | MATCH | ACTION |
========================================================================================================================================================
| ID | IngressPort | SrcMac | DstMac | EthType | Protocol | SrcIp | DstIp | SrcPort | DstPort | Action |
--------------------------------------------------------------------------------------------------------------------------------------------------------
| ACL-1000 | | | | ipv4 | icmp | | 192.168.10.10 | | | deny |
| ACL-1001 | | | | ipv4 | tcp | | 192.168.10.10 | | 5001 | deny |
--------------------------------------------------------------------------------------------------------------------------------------------------------iperf -c 192.168.10.10 -i 1
connect failed: Connection timed out- 上一篇： 处理Win10网络连接里没有以太网图标的问题
- 下一篇： Linux系统下配置VLAN
猜你喜欢
- 2024-11-21 织梦DEDECMS首页制作分页的图文教程
- 2024-11-21 Linux运维获取内存、cpu、磁盘IO信息
- 2024-11-21 服务器配置如何选择 (2)
- 2024-11-21 分布式系统核心:面向服务的分布式架构，基于Web服务的SOA
- 2024-11-21 黑苹果:戴尔7020sff安装bigsur体验
- 2024-11-21 如何从0移植uboot支持exynos4412?
- 2024-11-21 .NET 全面拥抱 Javascript，Jint 火了
- 2024-11-21 Windows下ftp服务配置和管理
- 2024-11-21 微软 .NET 9正式发布:性能大幅提升，提供18个月标准期限支持
- 2024-11-21 手把手教你进入移动光猫的超级管理员后台，终于可以修改设置了
欢迎 你 发表评论:
- 05-30壁纸|Q版灌篮高手
- 05-30Q版插画|今天的小姑娘有点小情绪，不爱笑，还是笑起来招人喜欢
- 05-30头像背景图|十二生肖Q版水墨(上)
- 05-308张Q版动漫人物-可爱的男生女生马克笔设计绘画，色彩漂亮鲜艳
- 05-30头像‖q卡通呆萌版
- 05-30Q版古风美人，十二花神系列，每一张都很可爱
- 05-30次元|动漫|Q版|萝莉|头像
- 05-30Q版古风人物|红楼梦系列 天上掉下来个林妹妹(附线稿)
- 最近发表
- 标签列表
-
- sd分区 (65)
- raid5数据恢复 (81)
- 地址转换 (73)
- 手机存储卡根目录 (55)
- tcp端口 (74)
- project server (59)
- 双击ctrl (55)
- 鼠标 单击变双击 (67)
- debugview (59)
- 字符动画 (65)
- flushdns (57)
- ps复制快捷键 (57)
- 清除系统垃圾代码 (58)
- web服务器的架设 (67)
- 16进制转换 (69)
- xclient (55)
- ps源文件 (67)
- filezilla server (59)
- 句柄无效 (56)
- word页眉页脚设置 (59)
- ansys实例 (56)
- 6 1 3固件 (59)
- sqlserver2000挂起 (59)
- vm虚拟主机 (55)
- config (61)
本文暂时没有评论，来添加一个吧(●'◡'●)