Validating Distributed Firewall rulesets in NSX

This post is intended to demonstrate what happens when the ‘Applied to’ command is used within VMware NSX’s Distributed Firewall. ‘Applied to’ allows the dvFilter to apply firewall policy to the vNIC of Virtual Machines associated with the abstraction – in this case – Security Groups. I am using a standard three-tier application. Due to the abstractions I am using this could simply be enforced on a flat network or existing VLAN back port-groups. You don’t always have to VXLAN all the things!

My topology

1

The following Security Groups are built and have these membership

  • Web VM’s are associated to SG-WEB
  • App VM is associated to SG-APP
  • DB VM is associated to SG-DB
  • SG-APP1 contains SG-WEB, SG-APP, and SG-DB
  • Protocols are listed below
  • All filters are applied to the pertinent SG

The Security Groups

This is show by the Distributed Firewall table below. The relevant Security Groups have their permitted protocols. There is a ‘ring-fence’ Security group, SG-APP1 which has SG-WEB, SG-APP and SG-DB nested inside it.

2

By hovering over the Security Groups you can see members of the relevant groups. Whilst my logs show PASS and DROP based on my rules I want to validate further. All  Allows in this topology are being logged.

3

I can see a IN TCP 172.16.10.10 -> 172.16.10.11/443. This is a permit from the LB on the Web Tier to web-srv-01a at 172.16.10.11 on 443. There are a couple of things worth noting

  • What is domain-c25?
  • What is 1016?

How do I found out about these and how do I know my security polices relevant to the web-srv-01a are being enforced? There is a way.

Down the rabbit hole

! Heads up – commands used here are not supported by VMware. These are engineering commands and not validated. These commands could change or be deprecated at any time so remember, there be dragons!

The kernel module for the Distributed Firewall is really known as a dvFilter. dvFilters are what are impressed onto the vNIC of a VM for rule enforcement. This section will take yo though how to find the filter you’re after, use the filter information to correlate abstractions, and subsequently resolve abstractions into objects.

I need to log into the vSphere host the workload is on. Issue the following command and grep web and nic as arguments to ensure you don’t wade through dozens of VMs dvFilters.

~ # summarize-dvfilter | grep -e web -e nic
   name: nic-0-eth4294967295-ESXi-Firewall.0
   name: nic-0-eth4294967295-ESXi-Firewall.0
   name: nic-0-eth4294967295-ESXi-Firewall.0
   name: nic-0-eth4294967295-ESXi-Firewall.0
   name: nic-0-eth4294967295-ESXi-Firewall.0
world 35671 vmm0:web-sv-01a vcUuid:'50 26 b7 4d c5 6c 1e d9-47 c0 09 25 95 80 2f ad'
 port 50331656 web-sv-01a.eth0
   name: nic-35671-eth0-vmware-sfw.2
   name: nic-35671-eth0-dvfilter-generic-vmware-swsec.1

Our filter we are after is nic-35671-eth0-vmware-sfw.2

~ # vsipioctl getfwrules -f nic-35671-eth0-vmware-sfw.2
ruleset domain-c25 {
  # Filter rules
  rule 1015 at 1 inout protocol tcp from addrset ip-securitygroup-20 to addrset ip-securitygroup-20 port 80 accept with log;
  rule 1017 at 2 inout protocol tcp from addrset ip-securitygroup-20 to addrset ip-securitygroup-21 port 22 accept with log;
  rule 1017 at 3 inout protocol tcp from addrset ip-securitygroup-20 to addrset ip-securitygroup-21 port 8443 accept with log;
  rule 1016 at 4 inout protocol tcp from any to addrset ip-securitygroup-20 port 443 accept with log;
  rule 1019 at 5 inout protocol tcp from addrset ip-ipset-2 to addrset dst1019 port 22 accept;
  rule 1014 at 6 inout protocol ipv6-icmp icmptype 135 from any to any accept;
  rule 1014 at 7 inout protocol ipv6-icmp icmptype 136 from any to any accept;
  rule 1013 at 8 inout protocol udp from any to any port 67 accept;
  rule 1013 at 9 inout protocol udp from any to any port 68 accept;
  rule 1012 at 10 inout protocol any from any to any drop with log;

So there are a couple of things here

  • Line 2 – ‘rulesetdomain-c25’
    • We have a match on domain-c25. This is actually the cluster in which the object is residing.
  • Line 7 – ‘rule 1016 at 4’
    • We have out rule matching out log that was found in Log Insight.
    • It permits tcp from any to an Address set ‘ip-securitygroup-20’ on port 443 with logs

This is a good start. We can see the dvFilter ruleset for the web-sv-01a matches up nicely to what is in the GUI. I

What am I matching?

f you need to find out the address set abstractions are resolving to you can find this out. It will resolve the address sets. Address sets such as IP-sets are a great way to map legacy IP models or enforce physical to virtual connectivity. You can show what an address set is resolving to for all address sets associated to a dvFilter

~ # vsipioctl getaddrsets -f  nic-35678-eth0-vmware-sfw.2
addrset dst1019 {
ip 172.16.10.11,
ip 172.16.10.12,
ip 172.16.20.11,
}
addrset ip-ipset-2 {
ip 192.168.110.10,
}
addrset ip-securitygroup-20 {
ip 172.16.10.11,
ip 172.16.10.12,
}
addrset ip-securitygroup-21 {
ip 172.16.20.11,
}
addrset ip-securitygroup-26 {
}

dst1019 is another way of myself matching three end-point VMs (2 web, 1 app). ip-ipset-2 is the control-VM used for jumphost functions in the lab. I’ve called them out via VM

Bean counting

You can get more information like stats on the filter by adding the -s flag before defining a filter.

~ # vsipioctl getrules -s -f nic-35678-eth0-vmware-sfw.2
rule  1015: 16607 evals, in 0 out 0 pkts, in 0 out 0 bytes
rule  1017: 8308 evals, in 0 out 0 pkts, in 0 out 0 bytes
rule  1017: 8308 evals, in 74728 out 90995 pkts, in 24918073 out 10939925 bytes
rule  1016: 8299 evals, in 77763 out 82819 pkts, in 9072423 out 24250921 bytes
rule  1019: 1 evals, in 70 out 88 pkts, in 18156 out 28600 bytes
rule  1022: 0 evals, in 0 out 0 pkts, in 0 out 0 bytes
rule  1021: 0 evals, in 0 out 0 pkts, in 0 out 0 bytes
rule  1014: 0 evals, in 0 out 0 pkts, in 0 out 0 bytes
rule  1014: 0 evals, in 0 out 0 pkts, in 0 out 0 bytes
rule  1013: 0 evals, in 0 out 0 pkts, in 0 out 0 bytes
rule  1013: 0 evals, in 0 out 0 pkts, in 0 out 0 bytes
rule  1012: 0 evals, in 0 out 0 pkts, in 0 out 0 bytes
rule  1011: 8 evals, in 152757 out 174005 pkts, in 34235096 out 35422400 bytes
~ #

Good old rule 1016 is getting a thrashing. If you’re on a host you can then view active flows for the respective filter.

Chasing the flow

If you want to see active sessions and connections via dvFilter it is possible.

~ # vsipioctl getflows -f nic-35678-eth0-vmware-sfw.2
Count retrieved from kernel active(L3,L4)=21, active(L2)+inactive(L3,L4)=3, drop(L2,L3,L4)=0
5507b0df0007bbc1 Active tcp 0800 1 1019 0 0  192.168.110.10:Unknown(55993) 172.16.10.11:ssh(22) 513 EST 61780 91229 505 505
5507b0df004b65d1 Active tcp 0800 2 1017 0 0  172.16.10.11:Unknown(54272) 172.16.20.11:Unknown(8443) 2911 FINWAIT2 1320 3002 11 9

You can see an active session from 192.168.110.10 to 172.16.10.11 on port 22 (SSH). Yay.

Filtered

What we achieved was a validation of this image.

4
Inside the Ringfence rules supplements with Address Sets to allow external HTTPS access to Web

 

Whilst you can be sure the filters are being applied through log and alerts these commands shown in this post help validate it further. I call these VCDX features – things used to demonstrate a masterful command of the technology whilst many may be obscure, unsupported, or deprecated. Whilst logging onto an ESX host is not always recommended or allowed (especially with lockdown mode) all I can say is watch this space! – #futures

4 thoughts on “Validating Distributed Firewall rulesets in NSX”

Leave a Reply

Your email address will not be published. Required fields are marked *


*