Enable MAC learning on OVS under OpenStack

I have access to a rather large OpenStack deployment. This environment I am a tenant and have standard rights assigned to me by my Administrator. For the purpose of the exercise I am going to show how to enable mac-learning for the Open vSwitch under NSX-MH. It will also demonstrate how one can nest workloads on an OpenStack powered environment. This environment runs OpenStack across KVM and vSphere. It is powered by NSX-MH.

Screen Shot 2015-05-12 at 8.35.06 am

There are three things that I will talk about in this post for ground floor switching success with Nested workloads on NSX-MH driven by OpenStack.


When creating and attaching an Instance to a Network, Neutron is responsible for creating the MAC address and assigning the IP address. Order of operations is Neutron generates MAC address and assigns it. Neutron subsequently creates and uses an IP address from the subnet ranged assigned to the network the Instance is being attached to. After all, Neutron is the source of truth when it comes to IP address and MAC creation. It takes precedence. During this process a chosen security-group is associated to the interface, lest default be assigned.


These are a set of rules that are applied to the OVS  logical port. Neutron Security groups maps to an NSX Security Policy which is assigned to a Logical Port on OVS. These have rule sets and permit and deny based on 5-tuple matching – SRC/DST IP, SRC/DST port, and protocol.


Oh, good old Port-Security. Port-security is equivalent to Spoofguard. When Port-Security is enabled it is not possible to change the IP address of an Instance. When an Instance has its IP address changed it will cease its ability to communicate on that network. You cannot remove Port-Security without first removing the Security-Group from a port.


Open vSwitch is has a learning ability. It can learn multiple MAC addresses assigned to a logical port. This allows capabilities such as nested virtualisation. This is what is used to utilise resources for lab environments. This has to be enabled on a per interface level. If your instance has been connected to multiple networks then you need to execute the command for each interface. MAC-Learning required for nesting environments. VMkernel interfaces have their own MAC’s as does nested VMs. Learning tables are enabled in OVS for the particular port. This allows the software switch to act and behave like a normal switch.

I’m just burning doing the Neutron dance

With what we know, I am going to prepare my Instances within my Project for MAC-Learning, disable security-groups and port-security while I am at it. First cab off the rank is my Storage appliance.

I am going to list all my workloads.

Screen Shot 2015-05-08 at 2.41.51 pm

I can expose just what I want with a simple grep.

Screen Shot 2015-05-08 at 2.41.37 pm

The output I have is as follows

  • Instance ID – a81c8bbe-ae5c-4227-bbba-d3c60f2684a2
  • Instance Name –  storage-a81c8bbe-ae5c-4227-bbba-d3c60f2684a2
  • Status – ACTIVE
  • Power State – Running
  • Network/IP = Backend_Network=

With this information I can further run commands to get the interface_ID. By using the Instance ID I can extract further information.

Screen Shot 2015-05-08 at 2.41.30 pm


The ID is what we are after for this instance. We can see here also the mac-address assigned by Neutron to the Instance.

Screen Shot 2015-05-08 at 3.16.40 pm

Port updated. To explain what is going on here – no-security-groups <id> is a Neutron command. Neutron processes the removal of the Security Group. Everything after the — such as –port-security_enabled=false –mac_learning_enabled=true are commands that are ignored by Neutron. They are received by NSX and processed accordingly.

The Result

The images stored within my Glance repository having static IP addresses there is a problem. There is a disconnect to what IP an Instance has to what Neutron programmed the OVS port. Passing the ability to disable Port-Security allows Neutron to learn what IP address an Instance has, irrespective of what was initially programmed by Neutron.

The subsequent command of enabling Mac-learning allows me to have a nested ESX host.

Rinse and Repeat

So Nick Bradford, my Solution Architect locally showed me a neat way to do this faster.

[email protected]:~$ export INST=9d66ed34-1076-4f71-9300-e98ac29a8f78
[email protected]:~$ for port in `neutron port-list --device-id=$INST | awk '{print $2}' | grep -v id`; do neutron port-update --no-security-groups $port -- --port-security_enabled=false --mac_learning_enabled=true; done
Updated port: 2f567a54-7522-4de7-9cc9-50ceb29dc8bd
Updated port: d860c3a2-ad09-45e1-8a17-360db3347da8
Updated port: eeb71acf-7f3c-4a13-9fea-b298b08c0228

This for loop allows me to take an Instance ID and loop it through all active interfaces for the instance. All I need to do is go through my required Instances to find out the relevant workloads to have the above for loop applied to. After I find the Instance ID, I update the INST environment variable and re-run the loop.


Purge all the firewall rules – Part 2

In my previous post I used an API call from a REST client that allowed me to purge all the rules within the Distributed Firewall function of NSX. I have also made this into a python script.

The snake script

Here is the script. This can be used at your own discretion. I am an awful coder. Feedback is welcomed. It is also found on my GitHub here.

## The big purge button for NSX for vSphere 6.1.x - purge-v01.py - 
## s: purge
## v: 0.1
## a: Anthony Burke
## e: [email protected]
## t: @pandom_
## w: networkinferno.net
## This purge script is designed to perform an emergency wipe of all rulesets applied to the global table of the distributed firewall.
## This can be used in the case vCenter access is blocked through a default deny all, excess rules in lab environments or to reset NSX pilots.
## NOTES - This needs an interactive prompt to ensure there is a chance to save yourself.
## History Log - v01 - Creation and initial commentary.

## Import libraries
import base64
import urllib2
import httplib
import xml.etree.ElementTree as ET
## Define NSX Manager IP address. Used by +nsx_ip+ as a variable.
## Define NSX Manager API port tcp/443
nsx_port = 443
## Credentials. I know this is not secure. Interactive maybe in future
username = "admin"
password = "VMware1!"
## Leveraging the library defined initially we pass the username and password through to create an authenticated session with NSX Manager.
creds= base64.urlsafe_b64encode(username + ':' + password)
## Headers we are using are Content-Type == application/XML. Authorization = basic password plus creds variable (which is actually defined as username + password!)
headers = {'Content-Type' : 'application/xml','Authorization' : 'Basic ' + creds }
## Create a variable of purge_firewall
def purge_firewall():
## Body == NONE due to this being a REST DELETE command.
    body = None
## Drawing on the httplib we use a HTTPS connection to the nsx_ip variable on the nsx_port variable
    conn = httplib.HTTPSConnection(nsx_ip, nsx_port)
## The actual secret sauce. Places the DELETE plus API call.hanks to Andrew Babakian for noting my URI mixup.
    conn.request('DELETE','https://'+nsx_ip+'/api/4.0/firewall/globalroot-0/config', None, headers)
    response = conn.getresponse()
## if Response not equal to 204 then spit out bad response. Otherwise happy days.
    if response.status != 204:
        print "error status code",  str(response.status) + " Firewall purge unsuccessful"
        print "Status code", str(response.status) + " Firewall purge successful"
## Main funciton that calls purge_firewall into action. 
def main():
    print "The deed has been."

In action

Here are some basic rulesets that I have created pre-purge.

Screen Shot 2015-04-16 at 10.51.51 pm

Time to execute the script. Don’t look too closely from where I am doing it. I did create the script in VI (+1 internet points).

Screen Shot 2015-04-16 at 10.34.11 pm

Woo! It seems it is successful. Time to look at the GUI to find out.

Screen Shot 2015-04-16 at 10.55.13 pm

Python purge

So what does all this mean? Well now I have a python script that allows the purging or reset of an environment. I am going to add an interactive prompt or maybe a question generator  to ensure a level of safety. This is a dangerous command as it can wipe out your security policies very fast. Luckily NSX does store a save on each commit.

I would like to thank Andrew Babakian (NSBU SE in Sydney. Rockstar!) for aiding me with some syntax and structure issues I had. I didn’t realise the URI was not passed in the rest DELETE command. I also did not specify how the libraries I imported would be used. Lessons learned by this grasshopper.