Developing Security with Metrics

It is a professional hazard in security to become stuck in a reactive stance, always running to put out the latest fire. Many security personnel find themselves in this mode and cannot seem to escape it. It is important, from time to time, or especially in the case that it has never happened, to stop and take stock of an organization as a whole. No matter how pressing the issues of the moment seem, it is critical to examine your organization from the top down in order to develop, and maintain, an effective information security program. While this sort of planning can seem like a waste of time when the very real threats are battering down the proverbial door of your defenses, it is critical to take a measured approach to your security response in order to be effective, especially with limited resources. The first step to achieving this goal is to gather effective intelligence, specifically having accurate monitoring systems and incident reports.

Setting up a monitoring system can be a painful, and dull experience. Intrusion Detection Systems (IDS) are never particularly sexy. They tend to produce a host of false positives, require a lot of time and care in order to tune properly, and generally suck up a lot of attention. It's much more fun to run a pen test or even to do some white box testing (a.k.a. code audits) because you're probing for vulnerabilities and generally hacking stuff. The urge to rush to the active phase of information security, though attractive, is misguided. Without understanding the threats your organization faces it isn't possible to do effective resource management.

Monitoring and logs are some are the foundation of a system that can be used to provide meaningful metrics for an information security organization. Once your organization can quantify their resources and the threats they face they can begin to accurately assess how they should invest their time and energy. Part of deploying a monitoring system involves identifying where to deploy it, and this phase naturally involves discovery that allows you to identify critical assets. Without identifying your critical infrastructure and assets any security plan is bound to be a shot in the dark. It pays huge dividends to take the time to identify where the critical resources in your organization exist. This type of inventory is a natural part of mission continuity practices but isn't always one that the average security analyst is familiar with. By asking yourself (or your organization) where their pain points exist it is easy to identify the most critical assets. Will the loss of a server cause business to grind to a halt? Will it take the loss of a whole server, or just a database? What about e-mail? What if nobody could SSH into a host, or if a file server was unavailable? The things that will cause your business to stop are the things that are your most critical assets. Some assets, although not proportionately high in terms of necessity to your business may be critical due to other factors as well. Would the loss of a specific data store trigger compliance with notification laws (and therefore legal and reputational expense)? These considerations should also be taken into account when performing any inventory. Once you have identified all your critical assets, then you can move on to developing a monitoring program.

Your monitoring should obviously focus on critical resources. It is much more important to know that your e-commerce database is crashing than that an outlying workstation might be contracting malware. While it may seem blasphemous for a security professional to admit that the spread of malware should rank low on the priority scale, having a botnet propagate off of an assistant's desktop is not nearly as important as protecting customer credit card data. This is one of the essential challenges of an information security program - malware is much sexier than database servers. Without careful allocation of resources it is easy to find yourself chasing down the exciting security threats rather than dealing with the mission critical ones.

Monitoring should include many factors so you can get a complete picture of the security stance of your resources. You should make sure your monitoring system is robust, and that it alerts you to threats to your critical resources. Once you have a good monitoring system in place then it's time to begin to try and crank out some measurable numbers. In addition to analyzing your incoming data it is also important to take stock of historical data. Incident reports are especially useful in this regard. By examining key incidents of the past you can include data about attacks that may not be common, but might have a wide ranging impact. For instance, if your web server is rarely ever attacked, but when it has been in the past it has been compromised and resulted in severe expense. This data can help you weigh the fact that you may not see many new web based attacks, adding priority to your assessment of web based threats despite active data supporting it.

Examining the landscape of your organizations over even a short period of time should reveal trends in the security threats you face. You may notice that you're experiencing a heavy load of web based attacks, while there are relatively few active scans of your servers. It is important when examining these sorts of reports that you ensure that your monitoring systems aren't skewed in their coverage because this could lead to a lopsided analysis. For instance, if you only have log monitors in place on your web servers and no network monitoring in place your reports will obviously show much heavier web attack patterns than network attacks. This could lead to a misguided decision to allocate resources to security web applications when in fact the greatest threat to your infrastructure is SSH brute force attacks.

Once you have identified your critical resources and the threats that face them then, and only then, should you begin to map out your security posture. Start with the most critical resources and devote resources to reducing the vulnerabilities they face. Limiting the exposure of critical resources will vastly reduce the likelihood that they will have problems. Look at the landscape from the top down and decide how you want to expend your time, energy, and perhaps budget. By taking a comprehensive look at your infrastructure and the risks you face you can make informed, and effective decisions that will affect real change. By devolving into a reactive stance you insure that you distract attention from your critical resources, leaving them open to attack, which in turn propagates more security incidents. This cycle can become self perpetuating and so it is important to pause and take stock of your organizations situation, no matter how overwhelmed you fell, lest you risk becoming trapped.