Juniper Backdoors and vendor stone throwing

By now many in the network and security field will have heard about the announcement from Juniper. Juniper’s commentary about an internal code review identifying malicious code on their ScreenOS platform sparked a marked increase of hype on the Twittersphere. Everywhere ranging from the US Senate down to the local security administrator people have been commenting.

Anything from “Who put it there?”, “Why did they put it there?” and “How did it not get noticed?” have been asked. Was it China or the NSA? Do we know who? No we do not. Should we speculate? No we should not. Will we find out the truth? Not unless it is a controlled media statement. To be honest these things are currently above my pay grade and will stay that way.

So as a network administrator who used to manage (and has upgraded some old ones from my last work places) what should you do?

Lets review what is known to the public

  • Juniper revealed they noticed additional and unauthorised code in the ScreenOS source. This has been in it since 2012.
    • ScreenOS 6.2.0r15 through 6.2.0r18 and 6.3.0r12 through 6.3.0r20
  • VPN traffic passing through the device can be decrypted and subsequently inspected
  • System access can be gained via any named account active on the device with the password <<< %s(un=’%s’) = %u

Instead of running around like a headless chook calling for blood there are some course of action to take:

  1. Is this firewall internet facing or have internet access? If not, plan and validate the risk profile. Plan to update the version. See 2 and 3.
  2. Juniper have released updated versions of the ScreenOS software. Update as soon as possible.
  3. IDS or SNORT rules to protect against any login by a system account which will generate the idea alert.
# Signatures to detect successful abuse of the Juniper backdoor password over telnet.
# Additionally a signature for detecting world reachable ScreenOS devices over SSH.

alert tcp $HOME_NET 23 -> any any (msg:"FOX-SRT - Flowbit - Juniper ScreenOS telnet (noalert)"; flow:established,to_client; content:"Remote Management Console|0d0a|"; offset:0; depth:27; flowbits:set,fox.juniper.screenos; flowbits:noalert; reference:cve,2015-7755; reference:url,http://kb.juniper.net/JSA10713; classtype:policy-violation; sid:21001729; rev:2;)
alert tcp any any -> $HOME_NET 23 (msg:"FOX-SRT - Backdoor - Juniper ScreenOS telnet backdoor password attempt"; flow:established,to_server; flowbits:isset,fox.juniper.screenos; flowbits:set,fox.juniper.screenos.password; content:"|3c3c3c20257328756e3d2725732729203d202575|"; offset:0; fast_pattern; classtype:attempted-admin; reference:cve,2015-7755; reference:url,http://kb.juniper.net/JSA10713; sid:21001730; rev:2;)
alert tcp $HOME_NET 23 -> any any (msg:"FOX-SRT - Backdoor - Juniper ScreenOS successful logon"; flow:established,to_client; flowbits:isset,fox.juniper.screenos.password; content:"-> "; isdataat:!1,relative; reference:cve,2015-7755; reference:url,http://kb.juniper.net/JSA10713; classtype:successful-admin; sid:21001731; rev:1;)
alert tcp $HOME_NET 22 -> $EXTERNAL_NET any (msg:"FOX-SRT - Policy - Juniper ScreenOS SSH world reachable"; flow:to_client,established; content:"SSH-2.0-NetScreen"; offset:0; depth:17; reference:cve,2015-7755; reference:url,http://kb.juniper.net/JSA10713; classtype:policy-violation; priority:1; sid:21001728; rev:1;)

 

Does this put a nail in the coffin of the Security business unit of Juniper? Probably not. What isn’t kosha is the fact that vendors are out for blood and throwing stones. I work for a software company. Network companies are evolving into software companies. I put this tweet out the other day

Now more than ever we should be vigilant. We should work to better ourselves. United we stand and together we fall.

For some more reading this Wired article provides a great summary.

“..the first thing you do when you get NSX is .. do not deploy NSX”

I was recording a PacketPushers podcast this morning with Ethan Banks, Stephen Skinner, and Chris Wahl which was great. It was talking the tips and war stories associated with VMware NSX and deploying it. There was a quote that was fired off early in the podcast and it was as follows

“..the first thing you do when you get NSX is .. do not deploy NSX”.

This is something that I wholeheartedly agree with. VMware espouse that NSX is quick to install, easy to integrate, and can happily work in a brownfield environment. All of that is true. Deploy NSX Manager OVA to your management cluster, point it towards vCenter, and happy days I now have a new inventory item in vCenter called Networking and Security.

The problem lies when people don’t have a design. Like a bull in a china shop and clicking buttons, deploying VTEPs, next next next and random policy application is something that becomes unruly.

What are you building that you will throw over the fence to ops? What was the design? Did you have a design?

I’ve walked into a couple of environments that have seen customers deploy:
* mind-boggling complex security policy (taking the hardware mentality of security) and applying it to NSX not uses
* complex routing architectures uses NSX edges but not distributed routing
* crazy L2 bridging and routing scenarios
* Did I mention security rule crazy ness?

The end result is a customer deploying something very different to what they expected it to be. The end result is the customer not getting full benefits from something they own. The end result is no net benefit moving the function to the hypervisor because they’ve just pushed

Security Policy

Policy is quite the buzzword. It is almost the new ‘SDN’ in terms of hype. Policy this and policy that. Policy is very personal and very much a word that holds most meaning to the individual.

In my world I term policy in the context of security as ‘desired state’. I want to attest a set of rules upon my infrastructure and ensure that they are applied. I want to know that what I express against ‘Web Policy’ is enforced. This has been a dilemma in the past where rules have for a long time been based on 5-tuple matching – the old SRC/DST IP SRC/DST Port and protocol type. The firewall has no idea about the platform or what is running on it other than 192.168.103.0/24 can talk to 10.0.0.3 on port TCP 3389.

So where does one start thinking about security in the scope of design? With software like VMware NSX you can enforce security rules at the vNIC. This is great. Every VM has a firewall filter protecting it. Think that last statement through. If every virtual machine has a firewall then I need to move beyond 5-tuple matching.

There are three mindsets I discuss with people with regards to security and NSX. They are grouped into the following:

  • Networking – Rules based on IPSet & MACSet
  • Infrastructure – Rules based on vCenter objects (Datastore, VM Name, VM type, Network port (VXLAN/VLAN)
  • Application – Service orientated focusing on Security group membership via Security tags.

Looking at this as a pyramid with networking rule on the bottom this is where we find most of the thinking happens initially. The need to define subnets, enforce them against VM’s and the thought of that alone is overwhelming. Network rules work for ingress and egress for elements that are not within the domain of NSX or vCenter. This could indulge physical objects such as SCADA or end user desktops. If you take the mindset that the industry as awhile has used for other 20 years then you will end up with thousands of firewall rules, overlap, and potential holes in your firewall environment.

Infrastructure rules look to bring administrators and security architects up a level and start expressing their security attestation using relevant objects. Building rules based on networks a workload is connected to is a start. Web-VLAN to App-VLAN. App-VLAN to Shared services-Logical-Switch . Assign a Service Group of protocols. There is a rule that is based on the virtual infrastructure.

You may use the vCenter object ‘Cluster’ and state Deny Cluster-PCI to All other Clusters. This would stop communication to any components on PCI cluster to All other clusters. You may chose to exempt AD-VM and DNS-VM with a Service Group matching AD and DNS ports to PCI cluster. This is one way to allow explicit communication from Cluster-PCI to AD and DNS VM.

Evolving from this customers and architects eventually iterate their environment to the final, Application centric or focused, security model. This is going ‘all in’ on the tools available within NSX. Security Tags, Service Policy, and Security Groups. The anatomy of a service architecture is as follows.

Security Group SG.Internet.Proxy that matches membership based on the Security Tag ST.Internet.Proxy. Applied to SG.Internet.Proxy is a Security Policy named SP.Internet.Proxy. This Security Policy states ‘Source security group’ to destination Proxy VIP (Squid is load balanced!) is allowed on 8080.

This means if a workload is allowed internet access it is tagged ST.Internet.Proxy and gains the ability to connect to the internet via the Squid Proxy. A workload can be the member of numerous tags at any given time. This allows a service based architecture that ensures key-security policies are based on broader groups such as ‘Users’ ‘Admins’ ’HR’ and then smaller service policies can be used between these larger groups.

This way of thinking does take time. This is a fundamental shift in the way the industry as a whole, irrespective of product, is looking at security.

I have built policy out and explored grouping mechanisms here.

So coming back onto design. How do you foresee your ‘micro segmentation’ plan? How do you feel you will carve your applications up? A mix of infrastructure and network rules? What about a mix of infrastructure and application rules? Or swim in the deep (and very awesome end) of Application rules?

Design. Whiteboard. Draw it out. Talk it out. Revise.

Earlier it was discussed there was some horror stories. In each instance a more refined, scalable and automated policy was applied with a high weight – same source and destination but using a more efficient matching and object criteria. This was applied inline, on a per application basis which allowed the removal of the network centric rules.

War Stories

There have been horror stories but we are changing the way we approach many things. After all – you do not know what you don’t know. Luckily there are many was to skin a cat. It is just simply one way results in a fur coat and the other a bloody mess and Dim Sims.

Like any other networking or IT project or work there is strong value in understanding what your desired goal is, the outcomes that are tied to it, and what each milestone along the way means. Just because it is done in software and you can install it by simply clicking next next next doesn’t mean you will get a desirable outcome.

Remember – Prior preparation prevents piss poor performance!