I see a lot of the same errors made repeatedly as organizations stand up Security Operations. They not only result in lost time and money, but often result in breaches and malware outbreaks. I tweeted these out of frustration quite some time ago and I’ve since been repeatedly asked for a blog post condensing and elaborating on them. So, without further ado, here are Lesley’s Rules of SOC, in their unabridged form. Enjoy!
- You can’t secure anything if you don’t know what you’re securing.
Step one in designing and planning a SOC should be identifying high value targets in your organization, and who wants to steal or deface them. This basic risk and threat analysis shows you where to place sensors, what hours you should be staffed in what regions, what types of skill and talent you need on your team, and what your Incident Response plan might need to include,
- If you’re securing and monitoring one area really well and ignoring another, you’re really not securing anything.
An unfortunate flaw in we as an infosec community is that we often get distracted by the newest, coolest exploit. The vast majority of breaches and compromises don’t involve a cool exploit at all. They involve unpatched systems, untrained employees, and weak credentials. Unfortunately, I often see organizations spending immense time on their crown jewel systems like their domain controllers, and very little paid to their workstations or test systems. All an attacker needs to be in a network is a single vulnerable system from which he or she can move laterally to other devices (see the Target breach). I also see people following the letter of the law in PCI compliance, ignoring all the software and human practices beyond this insufficient box.
- You can buy the shiniest magic box, but if its not monitored, updated, and maintained with your input, you’re not doing security.
Security is a huge growth market, and vendors get better and better at selling solutions to executives with every newsworthy data breach. A lot of ‘cybersecurity’ solutions are now being sold as a product in a box – ‘install our appliances on your network and become secure’, This is simply not the case. Vendor solutions vary vastly in quality and upkeep. All of this is moot if the devices are placed in illogical places in the network, so that the devices can’t see inbound or outbound internet traffic, or host to host traffic. Even with a sales engineer providing product initial setup, a plan must be developed for the devices to be patched and updated. Who will troubleshoot the devices if they fail? And finally, their output must be monitored by somebody who understands the output. I’m constantly appalled by the poor documentations big vendors provide for the signatures produced by their product. Blocking alone is not adequate. Who is attacking and what is the attack?
- If your executives aren’t at the head of your InfoSec initiatives, they’re probably clicking on phishing emails.
I think this is pretty self explanatory. Security is not an initiative that can be ‘tacked on’ at a low level in an organization. To get the support and response needed to respond to incidents and prevent compromise, the SOC team must have a fast line to their organization’s executives in an emergency.
- Defense in Depth, mother##%er. Your firewall isn’t stopping phishing, zero days, or port 443.
I constantly hear organizations (and students, and engineers) bragging about their firewall configs. This is tone deaf and obsolete thinking. Firewalls, even next generation firewalls that operate at layer 7, can only do so much. As I’ve said previously, exploits from outside to inside networks are not the #1 way that major breaches are occurring. All it takes is one employee clicking yes to security prompts on a phishing message or compromised website to have malware resident on a host inside their network. The command and control traffic from that host can take nigh infinite forms, many of which won’t be caught by a firewall without specific threat intelligence. You can’t block port 80 or 443 at the firewall in most any environment, and that’s all that’s really needed for an attacker to remote control a system. So you have to add layers of detection that have more control and visibility. such as HIDS, internal IDS, and system level restrictions.
- There are a lot of things that log besides your firewall and antivirus.
I wrote a post on this a while back listing a bunch. The thing that horrifies me more than SOCs that don’t have a decent SIEM or log aggregation solution are the ones that only monitor their antivirus console and firewall. So many network devices and systems can provide security logs. Are you looking at authentication or change logs? DNS requests? Email?
- Good security analysts and responders are hard to find. Educate, motivate, and compensate yours.
Or you will lose them just as they are becoming experienced. Our field has almost a 0% unemployment rate.
- Make good connections everywhere in your organization. People will know who to report security incidents to, and you’ll know who to call when they do.
There’s often a personality and culture clash between infosec people and the rest of the business. This is really dangerous. We are ultimately just another agency supporting the business and business goals. All of our cases involve other units in or organization to some extent or another.
- If you don’t have some kind of Wiki or KB with processes, contact info, and lessons learned, you’re doing it wrong.
I can’t believe I have to say this because it’s true of almost any scientific or technical field. If you don’t write down what you did and how you did it, the next person who comes along will have to spend the time and effort to recreate your steps and potentially make the same mistakes. This also means everybody on your team needs to be able to make notes and comment on processes, not just one gatekeeper.
- You can’t do everything simultaneously. Identify and triage your security issues and tackle one project at a time.
Plenty of the horror stories I hear from security operations centers in their early stages involve taking on too much at once – especially without the guidance of a project manager. These teams drop everything because they can’t do it all simultaneously. We have the unfortunate tendency to be ideas people without organizing the projects and tasks we develop into structured projects.
- Threat Intelligence is not a buzzword and does not center around APTs. Have good feeds of new malware indicators.
Yes, there are predatory companies selling threat intelligence feeds with little or no value (or ones that consist entirely of otherwise free data). The peril in discounting threat intelligence is that signature based malware and threat detection is becoming less valuable every day. Every sample of the same malware campaign can look different due to polymorphism, and command and control mechanisms have gotten complex enough that traffic can change drastically. We are forced, at this point, to start looking in a more sophisticated way at who is attacking and how they operate to predict what they will do next. The includes things from identifying domains resolving to a set of IPs to sophisticated intelligence analysis. How far you take threat intelligence depends on time, funding, and industry, but every organization should be making it a part of their security plan.
- if your employees have to DM me for help with their basic SIEM / log aggregation, you’re failing at training.
Happens all the time, folks. I see a lot of good people at organizations with terrible training cultures. Make sure everybody has a level of basic knowledge from the start, and isn’t so intimidated in asking for help that he or she feels forced to go outside your organization.
- Team build, and don’t exclude. The SOC that plays well will respond well together and knows their members’ strengths and shortfalls.
Prototypical hacker culture, while an absolute blast, is not for everyone. I’ve seen people shamed out of infosec for the most bizarre reasons – the fact is that some people don’t drink alcohol, or want to go to cons, or think Cards Against Humanity is appropriate. Yes, we are generally intelligent people and we can be rather eccentric. That doesn’t mean that people who find these things unpleasant don’t have skills and knowledge to contribute. Accept that they don’t have the same interests and move on without badgering. It’s their personal choice. When you plan your teambuilding activities, try to make them inclusive – people with kids might not be able to hang out at the bar at midnight.
- If you seek hires do it in range of places. Grads, veterans, exploit researchers, and more all may have different stuff to offer.
I see a lot of organizations with a relationship with a infosec group or university that only recruit from that specific pool. As with lack of genetic diversity, this provides no advancement or innovation. There are tons of places to find interesting perspectives on infosec from well educated candidates. It’s important to bring fresh ideas and perspective into your team.
- if your ticketing system doesn’t work in a security context, get your own dang ticketing system and forward.
There are two main reasons that you shouldn’t be using the same ticketing system for security cases that your IT department uses for everyday help desk operations. The first is security – there is no reason that your IT contractors or non-IT staff in general should be able to see the details of sensitive cases, even by an error in permissions. This also includes their accounts, should they become compromised. The second is that these ticketing systems are not designed with security incidents in mind. A security incident case management platform should do Critical things like store malware samples safely, provide court admissible records of evidence hashes and case notes, and integrate with SIEM or log aggregation solutions. If your ticketing solution is not doing these basic functions, it’s high time to consider a separate platform.
- DO virtualize your malware analysis. DON’T virtualize your security applications unless the vendor says how to.
Virtualization software is critical for lots of reasons in infosec – from setting up malware analysis labs to CTFs to honeypots. It is not appropriate for all security applications and solutions. Most organizations are heavily pushing virtualization as a cost saving initiative, but be very cautious when presuming all resource intensive and highly specialized security tools will function alike when virtualized.