Python

I have access to a rather large OpenStack deployment. This environment I am a tenant and have standard rights assigned to me by my Administrator. For the purpose of the exercise I am going to show how to enable mac-learning for the Open vSwitch under NSX-MH. It will also demonstrate how one can nest workloads on an OpenStack powered environment. This environment runs OpenStack across KVM and vSphere. It is powered by NSX-MH.

There are three things that I will talk about in this post for ground floor switching success with Nested workloads on NSX-MH driven by OpenStack.

Neutron

When creating and attaching an Instance to a Network, Neutron is responsible for creating the MAC address and assigning the IP address. Order of operations is Neutron generates MAC address and assigns it. Neutron subsequently creates and uses an IP address from the subnet ranged assigned to the network the Instance is being attached to. After all, Neutron is the source of truth when it comes to IP address and MAC creation. It takes precedence. During this process a chosen security-group is associated to the interface, lest default be assigned.

Security-Groups

These are a set of rules that are applied to the OVS logical port. Neutron Security groups maps to an NSX Security Policy which is assigned to a Logical Port on OVS. These have rule sets and permit and deny based on 5-tuple matching – SRC/DST IP, SRC/DST port, and protocol.

Port-Security

Oh, good old Port-Security. Port-security is equivalent to Spoofguard. When Port-Security is enabled it is not possible to change the IP address of an Instance. When an Instance has its IP address changed it will cease its ability to communicate on that network. You cannot remove Port-Security without first removing the Security-Group from a port.

MAC-Learning

Open vSwitch is has a learning ability. It can learn multiple MAC addresses assigned to a logical port. This allows capabilities such as nested virtualisation. This is what is used to utilise resources for lab environments. This has to be enabled on a per interface level. If your instance has been connected to multiple networks then you need to execute the command for each interface. MAC-Learning required for nesting environments. VMkernel interfaces have their own MAC’s as does nested VMs. Learning tables are enabled in OVS for the particular port. This allows the software switch to act and behave like a normal switch.

I’m just burning doing the Neutron dance

With what we know, I am going to prepare my Instances within my Project for MAC-Learning, disable security-groups and port-security while I am at it. First cab off the rank is my Storage appliance.

I am going to list all my workloads.

I can expose just what I want with a simple grep.

The output I have is as follows

Instance ID – a81c8bbe-ae5c-4227-bbba-d3c60f2684a2
Instance Name – storage-a81c8bbe-ae5c-4227-bbba-d3c60f2684a2
Status – ACTIVE
Power State – Running
Network/IP = Backend_Network=10.0.1.3

With this information I can further run commands to get the interface_ID. By using the Instance ID I can extract further information.

The ID is what we are after for this instance. We can see here also the mac-address assigned by Neutron to the Instance.

Port updated. To explain what is going on here – no-security-groups <id> is a Neutron command. Neutron processes the removal of the Security Group. Everything after the — such as –port-security_enabled=false –mac_learning_enabled=true are commands that are ignored by Neutron. They are received by NSX and processed accordingly.

The Result

The images stored within my Glance repository having static IP addresses there is a problem. There is a disconnect to what IP an Instance has to what Neutron programmed the OVS port. Passing the ability to disable Port-Security allows Neutron to learn what IP address an Instance has, irrespective of what was initially programmed by Neutron.

The subsequent command of enabling Mac-learning allows me to have a nested ESX host.

Rinse and Repeat

So Nick Bradford, my Solution Architect locally showed me a neat way to do this faster.

[email protected]:~$ export INST=9d66ed34-1076-4f71-9300-e98ac29a8f78
[email protected]:~$ for port in `neutron port-list --device-id=$INST | awk '{print $2}' | grep -v id`; do neutron port-update --no-security-groups $port -- --port-security_enabled=false --mac_learning_enabled=true; done
Updated port: 2f567a54-7522-4de7-9cc9-50ceb29dc8bd
Updated port: d860c3a2-ad09-45e1-8a17-360db3347da8
Updated port: eeb71acf-7f3c-4a13-9fea-b298b08c0228

This for loop allows me to take an Instance ID and loop it through all active interfaces for the instance. All I need to do is go through my required Instances to find out the relevant workloads to have the above for loop applied to. After I find the Instance ID, I update the INST environment variable and re-run the loop.

The snake script

Here is the script. This can be used at your own discretion. I am an awful coder. Feedback is welcomed. It is also found on my GitHub here.

## The big purge button for NSX for vSphere 6.1.x - purge-v01.py - ## s: purge ## v: 0.1 ## a: Anthony Burke ## e: [email protected] ## t: @pandom_ ## w: networkinferno.net ## This purge script is designed to perform an emergency wipe of all rulesets applied to the global table of the distributed firewall. ## This can be used in the case vCenter access is blocked through a default deny all, excess rules in lab environments or to reset NSX pilots. ## NOTES - This needs an interactive prompt to ensure there is a chance to save yourself. ## History Log - v01 - Creation and initial commentary. ## Import libraries import base64 import urllib2 import httplib import xml.etree.ElementTree as ET ## Define NSX Manager IP address. Used by +nsx_ip+ as a variable. nsx_ip="192.168.110.42" ## Define NSX Manager API port tcp/443 nsx_port = 443 ## Credentials. I know this is not secure. Interactive maybe in future username = "admin" password = "VMware1!" ## Leveraging the library defined initially we pass the username and password through to create an authenticated session with NSX Manager. creds= base64.urlsafe_b64encode(username + ':' + password) ## Headers we are using are Content-Type == application/XML. Authorization = basic password plus creds variable (which is actually defined as username + password!) headers = {'Content-Type' : 'application/xml','Authorization' : 'Basic ' + creds } ## Create a variable of purge_firewall def purge_firewall(): ## Body == NONE due to this being a REST DELETE command. body = None ## Drawing on the httplib we use a HTTPS connection to the nsx_ip variable on the nsx_port variable conn = httplib.HTTPSConnection(nsx_ip, nsx_port) ## The actual secret sauce. Places the DELETE plus API call.hanks to Andrew Babakian for noting my URI mixup. conn.request('DELETE','https://'+nsx_ip+'/api/4.0/firewall/globalroot-0/config', None, headers) response = conn.getresponse() ## if Response not equal to 204 then spit out bad response. Otherwise happy days. if response.status != 204: print "error status code", str(response.status) + " Firewall purge unsuccessful" return else: print "Status code", str(response.status) + " Firewall purge successful" reset=True return ## Main funciton that calls purge_firewall into action. def main(): purge_firewall() print "The deed has been." main()py

Python purge

So what does all this mean? Well now I have a python script that allows the purging or reset of an environment. I am going to add an interactive prompt or maybe a question generator to ensure a level of safety. This is a dangerous command as it can wipe out your security policies very fast. Luckily NSX does store a save on each commit.

I would like to thank Andrew Babakian (NSBU SE in Sydney. Rockstar!) for aiding me with some syntax and structure issues I had. I didn’t realise the URI was not passed in the rest DELETE command. I also did not specify how the libraries I imported would be used. Lessons learned by this grasshopper.

Network Inferno

– Abstracting the homogeneous from the heterogeneous –

Enable MAC learning on OVS under OpenStack