All Data is Security Data: A Shift in Thinking

Thursday, August 30, 2018

Chris Jordan


The most significant costs in security operations come from an unlikely source – missed opportunities caused by not collecting and organizing log data. In order to operate more efficiently, security operations need to collect more data to support not just detection but validation, impact analysis and response. Reviewing the data used in security operations shows that all data, not just security logs, are needed to operate efficiently.

In 2018, Ponemon’s Cost of Data Breach Study states that the average total cost of a breach ranges from $2.2 million to $6.9 million, depending on how many records are compromised.. And keep in mind, this is just the cost impact of a breach without regard for numerous other security issues. Efficient security operations save millions for the average mid to large sized company.

Back in 2005, in a conference room on the outskirts of Paris, Peiter Zatko, better known as Mudge, was explaining to me what he called the Physics of the Internet. It wasn’t physics, but the unwritten rules of how the flows of people and processes behave on the Internet. My wife leaned over and told me he was babbling incoherently. I leaned back and told her, “It is brilliant.” At that moment, I was listening to the first person to try and apply user behavior analysis to security. He was babbling— looking for any analogy to express his idea: and few understood the significance at the time.

I would follow Peiter on stage and speak about the scalability of detection analysis. I remember how awkward it was to quote Peiter while he sat in the audience.  He had said the year before, “You will never pay me to review audit logs.” That is, the smartest people in cybersecurity are not doing the tedious work, as they are far too engaged with more interesting projects.

Since then, Peiter has continued on from one engaging project to the next, and I continue to stew over the efficiency of security operations. The problem is that with limited talent, how do we review all the logs correctly? The answer has always been training more people and reducing log sizes.

Here lies the greatest failure of operational security: Less is not more— less really is less. Less data means less vision. This means less alerts, less validation data, less information with which to assess impact, and less data to respond. This begins to impact cost because events are missed and additional resources are spent obtaining data elsewhere when an organization must detect and respond without adequate data. Missed events and insufficient data for response far outweigh the costs of saving more.

We all know data is valuable. Facebook has built a business on that principle. At my company, we don’t consider ourselves trash collectors of log data, but the waste management team. We create value from all that trash.

All of us are literally surrounded by data in today’s world. And it’s useful in more ways than one would ever imagine. As we see companies collecting more and more data every day – and spending more money to understand it – we know the industry is finally coming to realize that all data is security data.

In the past, organizations, their IT departments, and especially their security vendors, cherrypicked the data they considered to be significant for security threat management. The 1990s and early 2000s saw the rise of security information and event management tools (SIEM’s), marketed as a tool to gather and provide access to the data causing vulnerabilities within an infrastructure. However, even though shifting perceptions fully will require more work, our modern understanding of what defines security data is completely different from when SIEM’s reigned supreme. Back then, truly massive data logs were quite rare. Not everything collected was deemed important enough to keep, let alone analyze.

In those days, most focused on prevention to address scaling response. But as our understanding of security data evolves, it’s become more apparent that allowance is the real issue.

When we enable full logging on a firewall, the logs mostly consist of allowed data flows—about 85%, given how firewalls handle UDP traffic. Blocked data are considered to be “completed,” or already addressed events. As previous security breaches have demonstrated, successful attacks will occur within allowed data flows, not blocked events. Organizations are beginning to understand this, and we see this reflected in the unraveling definition of security data.

Think about it – from the data an employee is accessing to their request to access that data can be considered security data… and that’s just the beginning. Mobile data, social media data, location data, geographical data, utilities data— you name it, and it can profoundly impact an organization’s security operations or even law enforcement.

In the days of handpicking data deemed important to security, the scope of relevant data was narrow and static. Now, as the internet expands into our countless devices, appliances and vehicles, we’re opening the door to an era where these standards don’t exist, and everything has its own name. This will need to change. Data created by developers’ apps and programs isn’t currently recognized as security data, yet it absolutely is. Even the data that your refrigerator or your phone or your camera wants to share with you – none of that data is labeled as security data, but it all is.

Look at cameras on the street. Everyone is walking around with a camera in their pocket. There are ATM cameras, gas station cameras, hotel hallway cameras and FedEx cameras. Regardless of the intent of the camera, they have all provide law enforcement with video information, GPS information, and phone call metadata, all of which is being used to solve crimes. When the device began generating data, it certainly never knew its eventual use.

Still, this paradigm shift isn’t happening quickly enough. Report after report indicates that organizations don’t trust their current security solutions to prevent attacks. As the number of successful cyberattacks increase, so do companies’ costs. Any idea why? It’s nearly always associated with the amount of data collected, and the use of outdated solutions that weren’t intended to store a day’s worth of modern data collection, let alone a year’s. Using a solution that’s not designed for today’s ongoing, huge scalability needs is akin to a choking problem that only gets worse and requires increasingly expensive doctor visits that just never seem to get you healthy. 

Business is about efficiency. Business competition is harsh, and it doesn’t care if you buy name brand solutions. Companies that collect and understand their data efficiently will perform fewer downstream actions and spend less money.

About the author: Chris Jordan is CEO of College Park, Maryland-based Fluency (, a pioneer in simplified log management and security analytics.

Possibly Related Articles:
Enterprise Security Security Awareness
Detection Data cybersecurity security operations
Post Rating I Like this!
The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.