Chapter 7. Disaster Recovery

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 7th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at sevans@oreilly.com.

The terms disaster recovery (DR) and business continuity planning (BCP) are often confused and treated as interchangeable. They are, however, two different, but related, terms.

Business Continuity pertains to the overall continuation of business via a number of contingencies and alternative plans. These plans can be executed based on the current situation and the tolerances of the business for outages and such. Disaster Recovery is the set of processes and procedures that are used in order to reach the objectives of the Business Continuity Plan.

BCP normally extends to the entire business, not just IT, including such areas as secondary offices and alternate banking systems, power, and utilities. DR is often more IT focused and looks at technologies such as backups and hot standbys.

Why are we talking about DR and BCP in a security book? The CIA triad (confidentiality, integrity, and availability) is considered key to nearly all aspects of Information Security, and BCP and DR are focused very heavily on Availability, while maintaining Confidentiality and Integrity. For this reason, Information Security departments are often very involved in the BCP and DR planning stages.

In this chapter we will discuss setting our objective criteria, strategies for achieving those objectives, and testing, recovery, and security considerations.

Setting Objectives

Objectives allow you to ensure that you are measurably meeting business requirements when creating a DR strategy and allow you to more easily make decisions regarding balancing time and budget considerations against uptime and recovery times.

Recovery Point Objective

The recovery point objective (RPO) is the point in time that you wish to recover to. That is, determining if you need to be able to recover data right up until seconds before the disaster strikes, or whether the night before is acceptable, or the week before, for example. This does not take into account of how long it takes to make this recovery, only the point in time from which you will be resuming once recovery has been made. There is a tendency to jump straight to seconds before the incident; however, the shorter the RPO, the more the costs and complexity will invariably move upwards.

Recovery Time Objective

The recovery time objective (RTO) is how long it takes to recover, taken irrespective of the RPO. That is, after the disaster, how long until you have recovered to the point determined by the RPO.

To illustrate with an example, if you operate a server that hosts your brochureware website, the primary goal is probably going to be rapidly returning the server to operational use. If the content is a day old, it is probably not as much of a problem as if the system held financial transactions for which the availability of recent transactions is important. In this case an outage of an hour may be tolerable, with data no older than one day once recovered.

In this example, the RPO would be one day and the RTO would be one hour.

There is often a temptation for someone from a technology department to set these times; however, it should be driven by the business owners of systems. This is for multiple reasons:

It is often hard to justify the cost of DR solutions. Allowing the business to set requirements, and potentially reset requirements if costs are too high, not only enables informed decisions regarding targets, but also reduces the chances of unrealistic expectations on recovery times.

IT people may understand the technologies involved, but do not always have the correct perspective to make a determination as to what the business’s priorities are in such a situation.

The involvement of the business in the DR and BCP plans eases the process of discussing budget and expectations for these solutions.

Recovery Strategies

A number of different strategies can be deployed in order meet your organization’s DR needs. Which is most appropriate will depend on the defined RTO, RPO, and as ever, by cost.

Backups

The most obvious strategy for recovering from a disaster is to take regular backups of all systems and to restore those backups to new equipment. The new equipment should be held at a dedicated disaster recovery facility or secondary office, located somewhere where the appropriate connectivity is available and the servers can begin operating right away.

Historically, backups were often to a tape-based medium such as DLT drives, which were physically shipped to another location. However, in recent times the cost of storage and network connectivity has come down, so backups can often be made to a more readily available and reliable media, such as an archive file on a remote hard disk.

Backups will generally have a longer RPO than other strategies, and backups tend not to be continuous but rather a batch job run overnight—and not necessarily every night. The RPO will be, at best, the time of the most recent backup. Additionally, backups frequently fail and so the RPO is in reality the time of your most recent working backup.

The RTO will vary depending on the speed of the backup media and the location of the backup media in relation to backup equipment. For example, if the backup media needs to be physically shipped to a location or if restoring would take days due to network and internet speeds, these must be factored in.

Warm Standby

A warm standby is a secondary infrastructure, ideally identical to the primary, which is kept in approximate synchronization with primary infrastructure. This infrastructure should be kept a reasonable geographic distance from the primary in case of events such as earthquakes and flooding. In the event of a disaster, services would be manually “cut over” to the secondary infrastructure. The method of doing so varies, but is often repointing DNS entries from primary to secondary or altering routing tables to send traffic to the secondary infrastructure.

The secondary infrastructure is kept in sync via a combination of ensuring that configuration changes and patches are applied to both primary and secondary, and automated processes to keep files synchronized. Ideally the configuration and patching would happen in an automated fashion using management software, however this is often not the case and can cause problems in the event that there are differences.

The RPO is fairly short on a warm standby, typically whatever the frequency of filesystem synchronization processes are.

The RTO is however long the cut-over mechanism takes. For example, with a DNS change this is the amount of time to make the change, and for old records to expire in caches so that hosts use the new system. With a routing change, the RTO is at least however long the routing change takes to make and, if using dynamic routing protocols, routing table convergence to occur.

However, this system does rely on having an entire second infrastructure that is effectively doing nothing until such time as there is a disaster.

High Availability

A high-availability system is typically a model like a distributed cluster. That is, multiple devices in distributed locations, which share the load during normal production periods. During a disaster, one or more devices will be dropped from the pool and the remaining devices will continue operation as per normal. In addition, they will continue to process their share of the additional load from the device that is no longer operational.

Due to the nature of high availability, it is typical that all devices in the cluster will be fully synchronized, or very close to it, and for this reason the RPO will be very short.

Many clustering technologies allow for devices to drop out of the cluster and the other devices will automatically adjust and compensate. For this reason, the RTO can also be lower than many other solutions.

Although the RPO and RTO are both advantageous when using a high-availability system, it is not without cost. The cluster needs to have enough capacity to compensate handling the additional load per remaining node. In the event of a disaster, this means running hardware that is not fully utilized in order to have spare capacity. Also, additional investment in areas such as intersite bandwidth will be required. Keeping all devices synchronized to run a clustered solution requires sufficient bandwidth at a low enough latency, which places additional requirements on the infrastructure.

Alternate System

In some cases, using an alternate system is preferential to running a backup or secondary system in the traditional sense. For example, in the case that an internal Voice over IP solution is rendered unavailable due to a disaster, the plan may not be to try to re-instantiate the VoIP infrastructure, but to simply change to using cellular phones until such time as the disaster is over.

This strategy does not always have an RPO per se, as recovery of the existing system is not part of the plan. This is why this type of approach is typically only taken with systems that do not hold data, but provide a service, such as telephones. There is, however, a measurable RTO in terms of the amount of time taken to switch over to using an alternate system.

System Function Reassignment

An approach that can prove to be cost effective is System Function Reassignment, which is a hybrid of other solutions. This is the repurposing of noncritical systems to replace critical systems in the event of a disaster situation. It is not applicable to all environments, and so should be considered carefully before being used as a strategy.

For example, if you already run two datacenters, structure your environments so that for any production environment housed in one datacenter, its test, pre-production, or QA environment is housed in the other datacenter. In this scenario you can have a near production site ready, but not idle, at all times. In the event of a disaster the environment in question will cease to operate as, for example, pre-production, and be promoted to a production environment.

This approach requires that the two environments be separated enough that a disaster affecting one will not affect the other. The state of the other environments should be tightly controlled so that any differences from production are known and easily changed to match that of production state prior to going live.

Dependencies

An important part of developing a strategy for DR and BCP is to understand the dependencies of all of the systems. For example, if you can successfully bring up a fileserver in another location, it does not matter if the staff can’t connect to it. Servers typically need a network connection, the associated routing, DNS entries, and access to authentication services such as Active Directory or LDAP. Failure to determine the dependencies required for any particular system may lead to missing the RTO for that service.

For example, if you have an email server with an RTO of 1 hour, and yet the network on which it depends has an RTO of 3 hours, irrespective of how quickly the email server is up and running, it may not be resuming operation in any meaningful sense until 3 hours have elapsed.

By mapping out dependencies such as this, it is much easier to identify unrealistic RTOs, or RTOs of other systems or services that need to be improved to meet these targets. Walking through tabletops and drills as mentioned in Chapter 1 will assist in discovering these dependencies.

Scenarios

When developing potential disaster plans, it is often useful to walk through a few high-level scenarios and understand how they impact your proposed plan. This exercise normally works most effectively with representatives from other IT teams who can assist with discussing the implications and dependencies of various decisions.

A few broad categories of scenarios are useful to consider, although which ones you choose to use will probably be dependent upon your own circumstances:

Hardware failure of mission-critical platform

Something that is isolated to a single platform, but that is significant enough to cause a DR incident—for example, the failure of server hardware for the production environment of a key system.

Loss of a datacenter

Potentially temporarily such as during a power outage, or perhaps for more prolonged periods such as a fire or earthquake.

Pandemic

In the event of a pandemic, services may remain available, but physical access may not be possible, which in turn could prevent certain processes from taking place, such as physically changing backup tapes, users working from home causing extra load up VPN, or other remote access services.

Invoking a Fail Over...and Back

It is all very well having a set of contingency plans in place and having target times by which to achieve them. If you do not know when you are in a disaster situation there is little point to the plans. There should be a process in place to determine what is and is not a disaster, and when to invoke the plan.

There may be a few key, high-level scenarios in which the plan would obviously be put into action. For example, the event of the datacenter being on fire is typically enough to invoke failing over to backup systems. However, care should be taken not to be too prescriptive or else the risk of minor deviations from the situations outlined may cause a failure to invoke the plan. Similarly, not being descriptive enough could cause an inexperienced administrator to invoke a DR plan needlessly. In this case, how do you determine when to invoke the plan? One of the most effective routes is to have a list of named individuals or roles who are authorized to determine when the organization is in a disaster situation and that the plan needs to be executed. The process for anyone who is not authorized to make this determination is to escalate to someone who can, who in turn will make the decision. This way the alarm can be raised by anyone, but the ultimate decision to execute is left to someone suitably senior and responsible.

One often overlooked area of DR and BCP is that as well as failing over to contingency systems, there will need to be a process of switching back again after the disaster has ended. Unlike the initial failover procedure, there is the advantage of being able to schedule the switch and take an appropriate amount of time to do so. Nevertheless, this should be a carefully planned and executed process that is invoked once again by an authorized person. Always remember to include the proper communication during the potential outages as it can be a high-stress time. A downtime is never too big for proper communication to happen.

Testing

Disaster Recovery can be extremely complex, with many of the complexities and interdependencies not being entirely obvious until you are in a disaster situation. Sometimes you’ll find that in order to complete the task, a file is required from a server that is currently under several feet of water. For this reason it is advisable—and under some compliance regimes mandatory—that regular DR tests be carried out. Of course, no one suggests that the datacenter be set on fire and attempted to be recovered. Choose a scenario and have the replacement systems brought up within the allotted RTO and RPO. This should be completed without access to any systems or services located on infrastructure affected by the scenario you have chosen.

The test should be observed and notes taken on what worked well and what did not. Holding a post-test debrief with the key people involved, even if the test met all targets, is a valuable process that can yield very useful results insofar as learning what can be improved in preparation for next time. Findings from the debrief should be minuted with clear action items for individuals in order to improve plans and work toward a more efficient and seamless process. A more in-depth look at this topic is covered in Chapter 1.

Security Considerations

As with any process, there are security considerations involved with most plans. These can be summarized into a few key categories:

Data at rest

Many contingency plans require that data from production systems be duplicated and retained at another site. This is true of both warm standbys and traditional backups, for example. It should always be remembered that this data will have controls placed on it in production in line with its value and sensitivity to the organization. For example, it may be encrypted, require two-factor authentication to access, or be restricted to a small group of people. If equal restrictions are not placed on the contingency systems, the original access controls are largely useless. After all, why would an attacker bother trying to defeat two-factor authentication or encryption on a production system when they can simply access a relatively unprotected copy of the same data from a backup system?

Data in transit

In order to replicate data to a secondary system, it will probably have to be transmitted over a network. Data transmitted for the purposes of recovering from or preparing for a disaster should be treated as carefully as any other time the data is transmitted. The appropriate authentication and encryption of data on the network should still be applied.

Patching and configuration management

It is often easy to fall into the trap of backup systems not being maintained in line with the production environment. This runs the risk of leaving poorly patched equipment or vulnerabilities in your environment for an attacker to leverage. In the event of a disaster, these vulnerabilities could be present on what has become your production system. Aside from the security issues, you cannot be sure that systems with differing configuration or patch levels will operate in the same way as their production counterparts.

User access

During a disaster situation there is often a sense of “all hands to the pumps” in order to ensure that production environments are operationally capable as soon as possible. It should be considered that not all data can be accessed by just anybody, particularly if the data is subject to a regulatory compliance regime such as those that protect personally identifiable healthcare or financial data. Any plans should include the continued handling of this type of data in line with established processes and procedures.

Physical security

Often the secondary site may not be physically identical to the primary site. Take, for example, a company for which the primary production environment is housed in a secure third-party facility’s managed datacenter, and the disaster location makes use of unused office space in the headquarters. A lower standard of physical access control could place data or systems at risk should an attacker be willing to attempt to physically enter a building by force, subterfuge, or stealth.

Conclusion

There is no one-size-fits-all solution to DR, although there are several well-trodden routes that can be reused where appropriate. One of the most important aspects of DR planning is to work with the business to understand what their requirements are for your DR solution. By aligning your solution with their expectations, it is easy to measure the success or failure of the system.

Defensive Security Handbook

Best Practices for Securing Infrastructure

Second Edition

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

Lee Brotherston and Amanda Berlin

Defensive Security Handbook

by Lee Brotherston, Amanda Berlin, and William Reyor

Copyright © 2024 Amanda Berlin, Lee Brotherston and William F. Reyor III. All rights reserved.

Printed in the United States of America.

Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.

O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com.

Acquisitions Editor: Jennifer Pollock

Development Editor: Shira Evans

Production Editor: Clare Laylock

Interior Designer: David Futato

Cover Designer: Karen Montgomery

Illustrator: Kate Dullea

December 2023: Second Edition

Revision History for the First Edition

2022-04-21: First Release

2022-07-05: Second Release

2022-10-14: Third Release

2022-11-22: Fourth Release

2023-02-09: Fifth Release

2023-03-16: Sixth Release

2023-05-03: Seventh Release

2023-06-28: Eighth Release

See http://oreilly.com/catalog/errata.csp?isbn=9781098127183 for release details.

The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Defensive Security Handbook, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc.

While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

978-1-098-12718-3

[LSI]

Chapter 1. Creating a Security Program

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 1st chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at sevans@oreilly.com.

Creating or improving upon a security program can be a daunting task. With so many facets to consider, the more initial thought and planning that is put into the creation of this program, the easier it will be to manage in the long run. In this chapter, we will cover the skeleton of a security program and initial administrative steps.

Do not fall into the habit of performing tasks, going through routines, or completing configuration with the mindset of, “This is how we’ve always done it.” That type of thinking will only hinder progress and decrease security posture as time goes on.

Humans are allergic to change. They love to say, “We’ve always done it this way.” I try to fight that. That’s why I have a clock on my wall that runs counter-clockwise.”

Grace Hopper, “The Wit and Wisdom of Grace Hopper” (1987)

We recommend that when creating the program, you follow this chapter in order. While we attempted to group the remaining chapters accordingly, they can be followed as best fits a company.

Lay the Groundwork

It is not necessary to reinvent the wheel in order to lay out the initial groundwork for an information security program. There are a few standards that can be of great use that we will cover in Chapter 8. The National Institute of Standards & Technology (NIST) has a risk-based cybersecurity framework that covers many aspects of a program. The NIST Framework Core consists of five concurrent and continuous functions-Identify, Protect, Detect, Respond, and Recover. When considered together, these functions provide a high-level, strategic view of the lifecycle of an organization’s management of cybersecurity risk. Not only will a framework be a possible asset, so will compliance standards. Although poorly implemented compliance standards can hinder the overall security of an organization, they can also prove to be a great starting point for a new program. We will cover compliance standards in more depth in Chapter 8. While resources like these can be a phenomenal value add, you must always keep in mind that every organization is different, and some aspects covered may not be relevant (there are continuous recurring reminders of this throughout the book).

Establish Teams

As with many other departments, there are virtues in having the correct staff on the correct teams in regards to security. Open cross-team communication should be a primary goal, as without it the security posture is severely weakened. While smaller organizations may combine several of the following teams, or have a lack of them all together, this remains a good goal to populate a security department.

Executive team

A chief information officer (CIO) or chief information security officer (CISO) will provide the leverage and authority needed for business wide decisions and changes. An executive team will also be able to provide a long-term vision, communicate corporate risks, establish objectives, provide funding, and suggest milestones.

Risk team

Many organizations already have a risk assessment team, and this may be a subset of that team. In the majority of organizations, security is not going to be the number-one priority. This team will calculate risks surrounding many other areas of the business, from sales to marketing and financials. Security may not be something they are extremely familiar with. In this case they can either be taught security basics case by case, or a security risk analyst could be added to the team. A risk framework such as NIST’s Risk Management Framework (RMF) or the Operationally Critical Threat, Asset, and Vulnerability Evaluation (OCTAVE) Framework can assist with this.

Security team

The security team will perform tasks to assess and strengthen the environment. The majority of this book is focused toward this and the executive team. They are responsible for daily security operations, including managing assets, assessing threats and vulnerabilities, monitoring the environment for attacks and threats, managing risks, and providing training. In a large enough environment, this team can be broken up into a variety of subteams such as network security, security operations, security engineering, application security, and offensive security.

Auditing team

It is always a good idea to have a system of checks and balances. This is not only to look for gaps in the security processes and controls, but also to ensure the correct tasks and milestones are being covered. As with the Risk Team, this may be a subset of a larger group.

However it is entirely possible that a Small to Medium Business (SMB) may combine one or (unfortunately due to things like budget constraints/etc) all of these roles into one. In those cases we definitely commiserate with you, as it happens all too often. As the company grows, and hopefully the security program also grows, these separate roles can then be planed and adequately filled.

Baseline Security Posture

The unknowns in any environment are going to be scary, but that shouldn’t stop you from diving in. How will you know what level of success the program has had without knowing where it started? At the beginning of any new security program or any deep dive into an existing one, a baseline and discovery phase should be one of the first and foremost tasks at hand for all teams. Throughout this book we will cover asset management several times in different ways. The baseline of the security of the organization is just another step in that management. Items that should be gathered include:

Policies, procedures, and incident response playbooks

Endpoints-desktops and servers, including implementation date and software version

Licensing and software renewals, as well as SSL certificate expiration dates

Internet footprint-domains, mail servers, DMZ devices, cloud architecture

Networking devices-routers, switches, APs, IDS/IPS, and Network Traffic

Logging and monitoring

Ingress/egress points-ISP contacts, account numbers, and IP addresses

External vendors, with or without remote access, and primary contacts

Applications - any primary software applications either maintained by your company, or used in any aspect as primary business functions

Assess Threats and Risks

Assessing threats and risks will be incredibly different for each and every organization. Each internal and external footprint is unique when combined with the individual infrastructure involved. Assessing these includes both a high-level overview, as well as in-depth knowledge of assets. Without the knowledge of the threats and risks your organization faces, it is more difficult to custom fit technologies and recommendations to provide a suitable defense. As mentioned previously, a risk team or role is an essential part of creating an information security team. Much more detailed information can be found when researching Governance, Risk, and Compliance (GRC).

As we can’t cover the entirety of GRC in this book we’ll go over a general risk framework. There are a handful of risk management frameworks, but can be summarized in 4 steps: identify, assess, mitigate, and monitor.

Identify Scope, Assets, & Threats

Organizations should be concerned with a large amount of threats and risks that will cross industry verticals. Focusing on industry trends and specific threats will allow the security program to be customized and prioritized to become more efficient. Many organizations have put very little thought into what threats and risks they face on a day-to-day basis, and will continue to do so until they fall victim to them. Invaluable resources in this case are available through Information Sharing and Analysis Centers (ISACs), which are brought together by the National Council of ISACs to share sector-specific Information Security. “ISACs collect, analyze and disseminate actionable threat information to their members and provide members with tools to mitigate risks and enhance resiliency.”¹

Not only should industry-specific threats be identified, but also overall trending threats such as malware, ransomware, phishing, and remote exploits. Three very important places to make note of are the OWASP top 10, Center for Internet Security (CIS) 18, and the standards outlined by the Cloud Security Alliance. The majority of the items on these lists will be covered in more depth in this book, but keeping up-to-date with them year to year should be a key part of any strategic plan.

Assess Risk & Impact

After the potential risks have been identified, assess these risks to determine if they apply to the particular environment. Tasks such as internal and external vulnerability scans, firewall rule audits, authentication/user permission assessments, and asset management and discovery will lend a larger picture to the type of overall risk exposure.

During the assessment step, you’ll want to analyze each identified risk and determine the likelihood of it negatively impacting the organization, how severe that impact might be, and what it would look like when executed. For example:

Threat

An attacker exploits a new vulnerability on an _______.

Vulnerability

Unpatched

Asset

Mail server

Consequence

To use a Remote Code Exploit (RCE) to access and pivot to internal systems

Mitigate

Mitigation of risks is the meat and bones of why we’re all here; it’s also the purpose of the majority of this book. Options include avoiding, remediating, transferring, or accepting the risk. Some examples:

Risk avoidance

Dave decides that storing Social Security numbers for customers is an unneeded process and discontinues the practice.

Risk remediation

Alex starts turning off open ports, implementing stricter firewall rules, and patching endpoints.

Transferring of risk

Ian outsources credit card processing to a third-party as opposed to storing data on site.

Accepting risk

Kate knows that a certain endpoint has no access to other endpoints and runs a third-party application. This application has a low-risk vulnerability that is required for it to function. While nothing at that point can be changed or remediated with that vulnerability, the risk is low enough to accept.

Note

You should only accept risk as a last resort. If a risk ever makes it to this point, request full documentation from third-party vendors and the executive team, as well as documentation of processes that have been attempted prior to making this decision. Add at least an annual review of any accepted risks to ensure they are revisited accordingly.

Monitor

Keep track of the risk over time with scheduled quarterly or yearly meetings. Throughout the year, many changes will have taken place that affect the amount and type of risk that you should consider. As a part of any change monitoring or change control, determine if the change is affecting risk in any way. One way of tracking ongoing risk status is by using a risk register to document different scenarios, controls, and treatment plans, which also can be combined in any vulnerability management program.

Prioritize

Once threats and risks have been identified and assessed, they must also be prioritized from highest to lowest risk percentage for remediation, with a concentration on ongoing protection. This doesn’t always have to be an expensive venture, however. A large amount of defensive mitigations can be performed at little or no cost to an organization. This enables many opportunities to start a security program without having a budget to do so. Performing the due diligence required to get the program off the ground for free should speak volumes to an executive team.

Note

Do not always take application, vendor, or third-party advice for prioritization. Every environment is different and should be treated as such. Prioritize tasks based on the bigger picture when all of the information has been collected.

This book wasn’t written to be a sequential list of security tasks to complete. Prioritization can differ greatly from environment to environment. Just remember, if the environment is already on fire and under attack, don’t start by creating policies or reversing malware. As a Fire Marshall, you shouldn’t be worried about looking for the arsonist and point of origin when you haven’t even put out the fire yet.

To determine the priority of some risks, you can use a risk matrix, where the overall risk level is calculated by taking “Likelihood times Impact”

Figure 1-1. https://www.techtarget.com/searchsecurity/tip/How-to-perform-a-cybersecurity-risk-assessment-step-by-step

Create Milestones

Milestones will take you from where you are to where you want to be. They will be a general progression on the road to a secure environment. This is heading a little into project manager (PM) duties, but in many cases companies do not have dedicated PMs. Milestones can be broken up loosely into four lengths or tiers:

Tier 1: Quick wins

The earliest milestones to meet should be quick wins that can be accomplished in hours or days-high vulnerabilities such as one-off unused endpoints that can be eliminated, legacy devices that can be moved to a more secure network, and third-party patches all could fall under this category. We will mention many free solutions as the sales process in some organizations can take a significant time to complete.

Tier 2: This year

Higher vulnerabilities that may need to go through a change management process, create a change in process, or be communicated to a significant amount of people might not end up in Tier 1. Major network routing changes, user education implementation, and decommissioning shared accounts, services, and devices are all improvements that also require little-to-no-budget to accomplish, but can take a little more time due to the need for planning and communication.

Tier 3: Next year

Vulnerabilities and changes that require a significant amount of planning or that rely on other fixes to be applied first fall into this tier. Transitioning entire business functions to a cloud service, domain upgrades, server and major infrastructure device replacements, monitoring, and authentication changes are all good examples.

Tier 4: Long-term

Many times a milestone may take several years to accomplish, due to the length of a project, lack of budget, contract renewals, or difficulty of change. This could include items such as a network restructure, primary software replacement, or new datacenter builds.

It is helpful to tie milestones to critical controls and risks that have already been identified. Although starting with the higher risks and vulnerabilities is a good idea, they may not be easy fixes. In many cases, not only will these items take a significant amount of time and design, but they may require budget that is not available. All aspects need to be taken into account when creating each tier.

Use Cases, Tabletops, and Drills

Use cases are important for showcasing situations that may put critical infrastructure, sensitive data, or other assets at risk. Brainstorm with data owners and leaders to plan ahead for malicious attacks. It is best to come up with around three different use cases to focus on in the beginning and plan on building security mitigations and monitoring around them. Items such as ransomware, DDoS (Distributed Denial of Service), disgruntled employee, insider threat, and data exfiltration are all good examples of possible use cases. After several use cases have been chosen they can be broken down, analyzed, and correlated to each step of any one of the security frameworks that are covered in this book, or additionally others that may end up being created after we’re done writing. One example of a common framework used to map use cases is Lockheed Martin’s Intrusion Kill Chain.

The Intrusion Kill Chain, sometimes called the Cyber Kill Chain, is “a model for actionable intelligence when defenders align enterprise defensive capabilities to the specific processes an adversary undertakes to target that enterprise.” It is composed of seven steps as described in the Lockheed Martin whitepaper:

1. Reconnaissance

Research, identification, and selection of targets, often represented as crawling internet websites such as conference proceedings and mailing lists for email addresses, social relationships, or information on specific technologies.

2. Weaponization

Coupling a remote access trojan with an exploit into a deliverable payload, typically by means of an automated tool (weaponizer). Increasingly, client application data files such as Adobe Portable Document Format (PDF) or Microsoft Office documents serve as the weaponized deliverable.

3. Delivery

Transmission of the weapon to the targeted environment. The three most prevalent delivery vectors for weaponized payload are email attachments, websites, and USB removable media.

4. Exploitation

After the weapon is delivered to victim host, exploitation triggers intruders’ code. Most often, exploitation targets an application or operating system vulnerability, but it could also more simply exploit the users themselves or leverage an operating system feature that  auto-executes code.

5. Installation

Installation of a remote access trojan or backdoor on the victim system allows the adversary to maintain persistence inside the environment.

6. Command and Control (C2)

Typically, compromised hosts must beacon outbound to an internet controller server to establish a C2 channel. APT malware especially requires manual interaction rather than conduct activity automatically. Once the C2 channel establishes, intruders have “hands on the keyboard” access inside the target environment.

7. Actions on Objectives

Only now, after progressing through the first six phases, can intruders take actions to achieve their original objectives. Typically, this objective is data exfiltration, which involves collecting, encrypting and extracting information from the victim environment; violations of data integrity or availability are potential objectives as well. Alternatively, the intruders may only desire access to the initial victim box for use as a hop point to compromise additional systems and move laterally inside the network.

This whitepaper has a good amount of information that can be used for creating use cases as well.

Table 1-1 is an example of a step-by-step kill chain use case we’ve created for a ransomware attack.

Table 1-1. Ransomware use case

Kill chain step
Malicious action
Defensive mitigation
Potential monitoring

Reconnaissance
Attacker obtains email addresses, technologies used, and creates an organizational profile based on that information.

Create policies around sharing internal information on sites such as LinkedIn or using corporate email addresses for nonbusiness use.

After a major breach has been seen on the news run a password reset. Even though they shouldn’t, employees will reuse passwords for other services and sites.

Have corporate emails been seen in breaches elsewhere? How many emails are found with OSINT?

Weaponization

Attacker creates a malicious exploit to send to the victim, or uses a current exploit.

Knowledge and awareness of threats currently being used by attackers will allow for better constructed and tuned mitigation steps.

n/a

Delivery

A user receives a phishing email.

Assess which attachment types are needed in the organization. File types such as .js can be extremely harmful and are rarely exchanged from external sources.

Implement mailing blacklists and greylists such as Spamhaus and dnsbl to block known malicious mail servers.

Instill the idea of “trust but verify” to your users.

Implement Ad-Blocking

Filetypes of a certain size known to be malicious and associated with ransomware. (Flag .scr files over 22 MB and .js over 15 MB.)

Exploitation

Endpoint downloads a JavaScript file or Word document with malicious macro.

Disable macros and malicious filetypes via group policy.

Ensure any endpoint protection is up-to-date and installed.

Monitor proxy logs for unexpected file retrievals (e.g., JavaScript is the first file fetched from that host, host is on a threat intel list, etc.)

Use proxies or IDS (if cleartext) to monitor for known deobfuscation strings.

Installation

The payload is executed on the end user’s device. (Lucky, Cerber, and CryptoWall use the built-in Windows Crypto API to handle the encryption.)

Keep backups (that are not permanently attached) so that encrypted files can be restored easily.

Depending on OS, you can use “filesystem firewalls” such as as Little Flocker to permit access to files on per-process basis. That means that you can permit read access to MS Word, but not IE, for example.

There are experimental techniques that can be used to block crypto-based ransomware (e.g., Decryptonite)

High increase in Windows Crypto API over short amount of time.

Excessive numbers in a domain or low % of meaningful strings in domain.

Command & Control (C&C)

The ransomware contacts a C&C server on the internet to transmit the decryption key.

Implement sinkhole DNS and autoblock outbound connections to known malicious IP addresses using Dynamic Block Lists (DBL).

Connection to known C&C servers.

Actions & Objectives

The malware starts encrypting the files on the hard disk, mapped network drives, and USB devices. Once completed, a splash screen, desktop image, website, or text file appear with instructions for the ransom.

Implement Honey Directories-the ransomware goes into C:\$$ it sees another $$ directory, when it goes into C:\$$\$$ it sees another $$ directory, and so on.

Advanced file auditing can be enabled for alerting on an extreme increase in filesystem changes.

Many different defensive mitigations and specific detections can be added at each step of the kill chain for an overall decrease in risk at each layer.

Following the creation and implementation of security controls around use cases, and employing tabletop exercises and drills can serve as a proof of concept as well as assist in an ever growing collection of playbooks. A tabletop exercise is a meeting of key stakeholders and staff who walk step by step through the mitigation of some type of disaster, malfunction, attack, or other emergency in a low stress situation. A drill is when staff carries out as many of the processes, procedures, and mitigations that would be performed during one of the emergencies as possible.

While drills are limited in scope, they can be very useful to test specific controls for gaps and possible improvements. A disaster recovery plan can be carried out to some length, backups can be tested with the restoration of files, and services can be failed over to secondary cluster members.

Tabletop exercises are composed of several key groups or members.

During a tabletop exercise there should be a moderator or facilitator who will deliver the scenario to be played out. This moderator can answer “what if” questions about the imaginary emergency, as well as lead discussion, pull in additional resources, and control the pace of the exercise.

Inform the participants that it is perfectly acceptable to not have answers to questions during this exercise. The entire purpose of tabletops is to find the weaknesses in current processes to mitigate them prior to an actual incident.

A member of the exercise should also evaluate the overall performance of the exercise, as well as create an after-action report. This evaluator should take meticulous notes and follow along any runbook or playbook to ensure accuracy. While the evaluator will be the main notetaker, other groups and individuals may have specific knowledge and understanding of situations. In this case, having each member provide the evaluator with their own notes at the conclusion of the tabletop is a good step.

Participants make up the majority of this exercise. Included should be groups such as finance, HR, legal, security (both physical and information), management, marketing, and any other key department that may be required. Participants should be willing to engage in the conversation, challenge themselves and others politely, and work within the parameters of the exercise.

What to include in the tabletop:

A handout to participants with the scenario and room for notes.

Current runbook of how security situations are handled.

Any policy and procedure manuals.

List of tools and external services.

Post-exercise actions and questions:

What went well?

What could have gone better?

Are any services or processes missing that would have improved resolution time or accuracy?

Are any steps unneeded or irrelevant?

Identify and document issues for corrective action.

Change the plan appropriately for next time.

Tabletop Templates

IncidentResponse.com

Microsoft’s Incident Response Playbooks

The Federal Emergency Management Agency (FEMA) has a collection of scenarios, presentations,  and tabletops that can be used as templates. 

We also recommend reviewing the case studies provided within The CERT Guide to Insider Threats (Cappelli et al., 2012). Which coveres in depth the Merit model of insider it sabatoge

Expanding Your Team and Skillsets

Finding a dedicated, passionate, and intelligent team can be one of the most difficult aspects of any professional’s life. 

What can you and your team do to expand knowledge and skillsets?

Encourage staff to either set up a home lab or provide a lab for them.

Labs can be used for testing out real-world scenarios, as well as practicing skills and learning new ones. Labs can be created at a relatively low cost by buying secondhand equipment. The best way to learn for the majority of people is hands-on, and with a lab there is no risk introduced into a production environment.

Compete in or create Capture the Flag competitions (CTFs).

CTFs are challenging, and they can provide cross training and team building, as well as increase communication skills. Most information security conferences have CTFs. If you are looking to expand a team, CTFs are a wonderful place to find new talent. Not only will participants be showing their level of knowledge, but also communication skills, how well they work with others in a team, and their willingness to help and teach others.

Find or create a project.

Automate something in the enterprise, find a need and fill it. It doesn’t matter what the skillset, there will be a project out there that needs help. Documentation is needed on 99% or more of the open source projects out there.

Attend, organize, volunteer, speak, sponsor, or train at an industry conference or local meetup.

There are hundreds of them across the US and they almost always need volunteers. Just attending a conference has its benefits, but truly immersing yourself will push you further to learn and experience more. Many careers have been started by having a simple conversation about a passion over lunch or a beer. Networking is a game changer in our industry, but it’s not the silver bullet for everyone. You can network all you want, but unless you are a desirable candidate it won’t matter. Having a willingness and desire to learn, listen, collaborate, and the ability to think for yourself are all ideal traits in such a fast-paced industry.

Participate in mentoring.

Whether as a mentor or mentee, structured or unstructured, mentoring can be a valuable learning process both on and off the job.

Conclusion

Creating an information security program is no easy task. Many programs are broken or nonexistent, adding to the overall lack of security in the enterprise environment today. Use this book as a guide to work through the different areas and to suit them to a custom-tailored plan. Organizational skills, a good knowledgeable, hard-working team, strong leadership, and an understanding of the specific environment will all be crucial to an effective program. 

¹ https://www.nationalisacs.org/about-isacs

Chapter 2. Asset Management and Documentation

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 2nd chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at sevans@oreilly.com.

As a whole, asset management is not an information security function. However, there are definitely components to it that assist in strengthening the overall security posture. It is one of the most difficult verticals to cover. Without proper asset management, an environment cannot be protected to its full potential. It is impossible to protect assets that are unknown, and they can be a significant disadvantage in any troubleshooting or incident response. In larger and older networks, it is next to impossible to completely be aware of each and every device that is connected or every piece of software the users may have installed. However, with the correct organization and security controls in place, it becomes much easier.

The majority of this chapter will cover how best to find assets, tie all of the information together, and document it for ease of use and troubleshooting. Above all else, the two most important things to remember about asset management are to ensure there is one source of truth, and that it is a process, not a project.

Additionally, each asset or group of assets must be assigned an owner and/or a custodian. An asset owner serves as a point of contact for the assigned asset, whereas a custodian has responsibility for the stored information. The assets are then categorized into different levels of importance based on the value of the information contained in them and the cost to the company if an asset is compromised.

Information Classification

The need for information classification has risen as the amount of data on digital storage has grown. Attackers use confidential data for their profit by selling it on the black market, to expose or cripple a specific organization, to commit fraud, or to aid in identity theft. While many industry compliance standards such as HIPAA and PCI DSS attempt to dictate the type of information that should be specifically guarded and segregated, that may not be the only data that is classified as confidential in an organization. There may also be contracts and other legal measures that must be consulted for classification and protection of certain data. Steps to correctly classify data can be described as follows:

1. Identify data sources to be protected.

Completion of this step should produce a high-level description of data sources, where they reside, existing protection measures, data owners and custodians, and the type of resource. Obtaining this information can be difficult, but can be an added part of the documentation process as data owners and custodians are assigned and documented.

2. Identify information classes.

Information class labels should convey the protection goals being addressed. Classification labels like Critical and Sensitive have different meanings to different people, so it is important that high-level class descriptions and associated protection measures are meaningful and well-defined to the individuals who will be classifying the information, as well as those who will be protecting it.

3. Map protections to set information classification levels.

Security controls such as differing levels and methods of authentication, air-gapped networks, firewalls/ACLs, and encryption are some of the protections involved in this mapping.

4. Classify and protect information.

Now the hands on work comes in! All information that has been identified in step 1 should now be classified as dictated in step 2, and protected as in step 3.

5. Repeat as a necessary part of a yearly audit.

Data footprints are ever expanding. As new software is installed or upgraded with add-ons, data grows or changes in scope. A yearly audit of the current data footprint in the enterprise will be required to ensure data continues to be protected as documented.

Asset Management Implementation Steps

The asset management process can be separated out into four distinct steps: defining the lifecycle, information gathering, change tracking, and monitoring and reporting. Assets can be added to the environment at an alarming rate via scripted virtual machine roll outs or business acquisitions, refreshed to new hardware or software versions, or removed altogether. There are several enterprise-level tools that assist in identifying data on systems. A solution should be implemented that will track an asset from as early as possible until its eventual decommissioning.¹

Defining the Lifecycle

There are many lifecycle stages in between delivery and decommissioning: an asset may be moved; the person it’s assigned to may no longer be employed; it may require repair or replacement; or it may go inactive while its assigned user is on leave of absence. Define lifecycle events and document them. Each department or person involved in each step should understand when and how assets are tracked at every point of their lifecycles. This assists with ensuring that any unplanned deviation from documented processes is caught. Following is a map of a very basic asset management lifecycle:

Procure

This is the procurement step of the lifecycle where assets are initially added to be tracked. At this point, the initial device information, such as serial number, PO, asset owner, criticality, and model name and number, can be added to the tracking system. Be aware that many organizations procure equipment in more than one area.

Deploy

When an asset is deployed by a sys admin, net admin, helpdesk member, or other employee, the location of the device can now be updated and any automated software data population can be tested. Remember: prior to deploying assets, they should be scanned for viruses and vulnerabilities or built with a custom secure image (if applicable) before being attached to the network. Too many times, do assets arrive from vendor’s that are already pre-infected, or shipped with outdated software that have security flaws.

Manage

The management lifecycle step can contain many subsections depending on the level of documentation and tracking that is decided upon. Items can be moved to storage, upgraded, replaced, or returned, or may change users, locations, or departments.

Decommission

Decommissioning assets is one of the most important steps of the lifecycle due to the inherent security risks regarding the disposal of potentially confidential data. When deciding on disposal options, different classifications of data can be tied to varying levels. There are many different ways to destroy data, and these have varying levels of security and cost. For example, for a physical spinning HDD options would include:

Staging for disposal

A single pass wipe: drives can be reused and provide a residual value return.

Multiple wipes: increases the level of security; this still protects the residual value but adds cost.

Degaussing: removes the ability to resell the drive and has no visual indicator that it has worked, but is a cheaper alternative to shredding and is often more practical.

Full disk encryption: drives can be reused; this increases the level of security.

Physical disposal

Crushing/drilling/pinning: these are other low-cost options, all of which show physical evidence that they have been completed and deter ordinary criminals. These methods destroy the value of the unit and the data is still present on the platters despite them not being able to spin up.

Shredding: the most secure form of data destruction and typically used for the highest levels of secure data, but it is expensive and destroys the ability for resale.

Removed as asset from inventory

Information Gathering

Information gathering contains the most hurdles and has a huge amount of complexity from environment to environment. As stated previously, obtaining a good software package coupled with well thought-out processes will lead to gathering as much information as possible on network-connected assets. If you do not have the budget for a full blown asset management and discovery platform, there are a handful of good free, cheap, or open source projects available such as Rumble or Netdisco.  

Figure 2-1 shows how Rumble can be used for asset inventory, and it is currently free for smaller organizations.

Figure 2-1. Rumble web interface

Figure 2-2 shows how Netdisco displays an example router and what ports are configured.

Netdisco is an SNMP-based L2/L3 network management tool designed for moderate to large networks. Routers and switches are polled to log IP and MAC addresses and map them to switch ports. Automatic L2 network topology discovery, display, and inventory.²

Figure 2-2. Netdisco web interface

There are also manual steps and several methods that can aid initial collection of information. 

Address Resolution Protocol (ARP) cache

An ARP cache is a collection of entries that are created when an IP address is resolved to a MAC address. Pulling the ARP cachefrom routers and switches will provide a list of IP and MAC addresses connected to the network.

Dynamic Host Configuration Protocol (DHCP)

Whatever device or software platform that has the role of handing out DHCP addresses will contain all IP address reservationsand possibly hostnames.

Nmap

"Nmap (“Network Mapper”) is a free and open source (license) utility for network discovery and security auditing. Many systems and network administrators also find it useful for tasks such as network inventory, managing service upgrade schedules, and monitoring host or service uptime.”

Running nmap against the entire range of networks can provide an amazing amount of information as it is a very comprehensive scanning tool. A simple scan to get started is this:

nmap -v -Pn -O 192.168.0.0/16 -oG output.txt

This command provides a verbose output (-v, or -vv for even more verbosity), and assumes all hosts are online, skipping discovery (-Pn) with operating system detection (-O) on 192.168.0.0 through 192.168.255.255, and outputs the results in a grepable (searchable) format (-oG) to output.txt. 

PowerShell

PowerShell is a very versatile and powerful tool in Windows environments. Starting with Windows 2008R2, the Active Directory Modulewas introduced. Prior to this, dsquery, adsi, and ldap commands were used to obtain information from AD.

Get-ADUser -filter *

can be used to return an entire list of users within an AD domain. 

There are many filters that can be added to return subsets of this list as well. To retrieve all domain computer accounts, run 

Get-ADComputer-Filter 'ObjectClass -eq "Computer"' | Select -Expand DNSHostName.

Simple Network Management Protocol (SNMP)

SNMP can provide a great amount of information on networked devices. By default, most SNMP version 2 read and write strings (the passwords used to query devices) are set to “public” and “private.” SNMP settings should be changed to custom strings, and if possible switched to SNMP version 3, which supports username and password authentication. Many software packages, such as NetDisco, which we mentioned earlier in this chapter, use SNMP to gather data.

Vulnerability management software

Data from a vulnerability scanner can be added into the asset management system. This assists in tracking risks and adding to the overall amount of useful information about assets.

Windows Management Interface (WMI)

WMI can be used to pull almost all important information from a Microsoft Windows device. Specific information can be gathered on components such as the CPU, memory, and disk, as well as system information such as OS, processes, and services. The command WMIC can be performed on a system locally, or the WMI API can be used with most programming languages—for example, remotely connecting with PowerShell uses the Get-WmiObject cmdlet.

Change Tracking

Keeping track of changes in hardware, software, and performance is a necessary step to having an up-to-date inventory. Certain modifications can significantly change the security risk for a device. Below are two different examples of how changes can introduce new security vulnerabilities to an environment without the proper tracking.

Steven runs a section of the company that has just spun up a new service line. They have been working with marketing and they give them the okay to buy a domain and spin up a Wordpress site. Before configuring the site for the public, they installs Wordpress on their own machine and begins to install add-ons. First off, Wordpress itself has had a history of vulnerabilities, but add-ons can be written by anyone and can introduce vulnerabilities from privilege escalation to XSS (Cross-Site Scripting). An endpoint change-tracking client could trigger an alert on the addition of unapproved or unlicensed software.

Mark decides he’s had enough with this company and quits. Before leaving, they remove an expensive piece of software from their laptop to keep the license key for their own personal use. When the equipment is repurposed, the replacement will more than likely need to use that same software. The asset management software should have the list of software that was present, as well as any corresponding licensing keys, the vendor contact information, and software renewal dates.

Monitoring and Reporting

Monitoring and reporting on assets provides notifications of upcoming software licensing renewals and hardware warranty expirations. Trends can be discovered with the right amount of information, taking guesswork out of creating yearly budgets and equipment procurement plans. This information can also be used to assist in any equipment refresh processes.

A helpful security measure to implement is the monitoring and alerting of any unapproved devices. In a perfect world, an alert would fire when a device MAC shows up that isn’t located in the asset management tracking program. As mentioned at the beginning of this chapter, this level of control for assets can be extremely difficult in larger, more complex, and older environments. 

Alerts may also be created for lack of certain software or system settings if, for example, an endpoint has no antivirus or management software installed, isn’t encrypted, or has unauthorized software. More than likely this will be done with some sort of endpoint monitoring software, but can also be accomplished in a more manual route with some software. Microsoft Endpoint Configuration Manager (formerly SCCM) has the ability to report on installed software as well.

Asset Management Guidelines

In addition to the steps involved in implementing asset management, there is also a solid set of guidelines to keep in mind during the implementation process.

Automation

To accomplish these steps effectively, attempt to automate as many of them as possible. If any person along the chain of custody of an asset finds they are repeating a manual process, the question, “Can this be automated?” should be asked. The process should pull authoritative information from trustworthy locations on as many assets as possible. DNS can pull in hostnames and IP addresses; DHCP can tie MAC addresses to those IP addresses; and a vulnerability scanner may find entire networks that were previously unknown. Adding barcodes early on in the lifecycle can greatly assist with automation as well. Everything that can be automated leads to a more efficient process.

One Source of Truth

As there are many different ways to gather information about devices, such as DNS, DHCP, wireless connections, MAC address tables, software licenses, nmap scans, etc., it is important to select a software that will easily integrate with the technologies already present. Having conflicting information in several different locations like spreadsheets and SharePoint is not conducive to a complete picture regarding current assets. When choosing a software or method, it should be well communicated that it alone is the one source of truth regarding assets, and any deviation should be dealt with accordingly (author’s note: As much as I didn’t want this to sound menacing, it totally does).

Organize a Company-Wide Team

Assets will enter the company from a variety of different areas. The purchasing department is the obvious first choice; third-party vendors may bring their own equipment; or there may be a BYOD (bring your own device) policy, which is a whole other can of worms. Types of departments that would benefit from being on an asset-management team include purchasing, receiving, helpdesk, communications, maintenance, and system administrators.

As with most other processes and procedures, it is close to impossible to plan for every possibility. Plan for the unplanned. If a member of the helpdesk team or another group that has access to the asset management software happens upon a device that has not been documented, there should be a process to deal with this. Not only should the asset then be added to the software, but the cause should also be investigated. Are there assets entering the organization in a different department or by different means that have yet to be added to the asset management process?

Executive Champions

The organizational team should also contain one or more members of the executive staff as a champion to assist in process and procedure changes that will cross through several departments. Larger organizations normally have difficulty with communicating changes and additions to procedures to the correct individuals, while smaller companies seem to resist change. Creating a well thought-out and communicated directive from someone other than security or IT staff will greatly increase the success. This executive member will also be able to see the effects of proper asset management in the form of cost savings and avoidances.

Software Licensing

When it comes to software license management, knowing what you are entitled to have deployed is often more important that what you actually have deployed. More often than not, organizations fail software audits for over-deployment because they can’t prove exactly what they have the right to have deployed. Ending up behind on software licensing can be a very expensive mistake. Not only will an updated list of currently installed software eliminate the risk of paying for software that isn’t being used, but it also ensures that there are no licensing overage fees or fines.

Define Assets

Define criteria for what constitutes a critical asset—many times they may be the device where critical data lives, as well. It may be a specific type of hardware or appliance such as a head end firewall or fiber switches or certain custom software packages. Discovery and inventory will produce a large asset list. Some assets will require more oversight or management than others. 

Documentation

Proper and complete documentation is an integral part of asset management. Creating and maintaining it should be a continual process from day one. Documentation is used to set clear directions and goals, as well as offering a continual reference as needed.

Spend sufficient time creating documentation and provide detailed descriptions of all security projects, including charts and statistics. These documents can be a major advantage when showing management where the security budget went. Another benefit of documentation is the knowledge that will be gained by all parties while creating it. Potential security holes or weaknesses in the program may also become apparent during this process. Every mistake is a learning opportunity; document problems or mistakes so they are not repeated.

What should be documented? There are several levels of documentation depending on the size and scope of an environment. The following sections present a starting point as to what documentation will be beneficial.

Networking Equipment

Many automated scanning tools can provide a detailed overview of networking equipment, examples of which are outlined in the following lists:

Hostname

Licensing information

Location

Management IP

Software, hardware, and firmware versions

Warranty information

Network

Ingress/Egress point public IP addresses for all sites

ISP Account Information and Contacts

Performance baselines of network traffic over a day/week/month period

Default Gateways

Servers

Applicationsand Roles

Departmentor group that manages/supports

Hostname

iLO address

IP address(es)

Is remote access allowed?

Is there PII or other sensitive data?

OS version

Open ports

Performance baselines of CPU, Memory, & Disk

Warranty Information

Desktops

Hostname

Department

Patch level

Users

Not only should the individual accountsbe documented, but also what users have access to each. Many times this can be tracked in an Identity and Access Management (IAM) solution or in a more manual process:

Database Administrator Accounts

Domain/Enterprise/Schema Admins (and other admin level accounts)

Root and Administrator Accounts

Service Accounts

Applications

Administrative Users

Licensing

Servers and Appliances involved

Type of Authentication

Workflow of data (explain further)

Cloud Assets

Many cloud providers have their own automated asset management and tracking systems built into their solution by default. However if you have assets spread over multiple providers or less advanced providers where you have to track your own items you’ll want to remember the following:

Resources

Compute Engine virtual machines (VMs)

Cloud Storage buckets

App Engine instances

Policies

IAM policy

Access Context Manager policy

Other

Certificates and Expiration dates

Domains and Expiration dates

Just as important as the documentation itself is the consistency and organization of the information. Naming in a consistent fashion assists in locating and understanding the roles of assets. For example:

ORG1-DC1-R2B-RTR3 = Organization 1, Datacenter 1, Row 2, Rack B, Router 3

SVC_ORG2-SQL10-SNOW = The service account for Organization 2, SQL Server 10, the Snow Application

ORG3-FL1-JC-AP3 = Organization 3, Floor 1, JC Hall, Wireless Access Point 3

Conclusion

Classify, organize, automate, define, gather, track, monitor, report, document, rinse, lather, repeat. The messy world of asset management can be a daunting task without a solid plan and the understanding that it’s not a one-time project. Having as much information in one place as possible about endpoint and infrastructure devices will not only assist in short-term troubleshooting, but also long-term design planning and purchasing decisions.

¹ “Information Classification—Who, Why, and How”, SANS Institute InfoSec Reading Room.

² https://sourceforge.net/projects/netdisco/

Chapter 3. Policies

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 3rd chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at sevans@oreilly.com.

Policies are one of the less glamorous areas of information security. They are, however, very useful and can be used to form the cornerstone of security improvement work in your organization. In this chapter we will discuss why writing policies is a good idea, what they should contain, and the choice of language to use.

Why are policies so important? There are a range of reasons:

Consistency

Concerns about inconsistent approaches from day to day or between members of staff should be vastly reduced in the wake of decent policies. A written set of policies reduces the need to make a judgment call, which can lead to inconsistent application of rules.

Distribution of knowledge

It is all well and good for you to know what the policy is with regards to not sharing passwords with others, but if the entire organization is unaware, then it is not providing you much benefit. Policy documents disseminate information for others to consume.

Setting expectations

Policies set rules and boundaries; by having clearly defined rules, it becomes equally clear when someone breaks those rules. This enables appropriate action to be taken. Departments like human resources find it difficult to reprimand someone because it “feels like” they may have done something wrong. A clear contravention of a rule is easier to enforce.

Regulatory compliance and audit

Many industries are regulated or pseudo-regulated, and many have auditors. A criteria common amongst nearly every regulatory compliance or auditing scheme is the existence of policies. By having a set of policies, you have already ticked a box on the regulatory compliance or audit checklist.

Sets the tone

The policy set can be used to set the overall tone of a company’s security posture. Even if not explicitly laid out, the policy set gives an overall feel as an organization’s approach to security.

Management endorsement

A management-endorsed policy, published within an organization’s official document library, lends credibility to the policy set itself and by extension to the security team as well.

Policies are living documents—they should grow with an organization and reflect its current state. Making changes to policy should not be frowned upon; evolution of both the policies themselves and the associated documentation is a positive change. A scheduled annual review and approval process of policies will allow you to ensure that they remain aligned with business objectives and the current environment.

Language

Policies should lay out what you, as an organization, wish to achieve in a series of policy statements. Detail as to specifically how this is achieved is outlined in procedure and standards documentation. For this reason there is no need to get caught up with complexity and detail. Policy statements should be fairly simple, clear, and use words like “do,” “will,” “must,” and “shall.” They should not be ambiguous or use words and phrases such as “should,” “try,” and “mostly.”

For example, a good policy will use statements such as:

A unique User ID shall be assigned to every user.

As opposed to

A unique User ID should be assigned to a user.

The use of “should” as opposed to “shall” gives the impression that this is a “nice to have,” not a rule. If there are times when a policy can be overridden, then this should be stated as part of the policy statement. This is often achieved by using phrases such as “unless authorized by a manager.” Care should be taken not to introduce ambiguity with such statements, however; for example, it must be clear what constitutes “a manager” in this case.

Documents should be designed to be read. There is no need to fill documents with excessively wordy statements or some kind of confusing legalese. Each policy statement can be only a few sentences, often only one, in a bullet point format.

Document Contents

Policy documents should contain a few key features:

Revision control

At the very least, this should include a version number and an effective date for the document. This allows a user in possession of two versions of a document to quickly establish which is the current version and which is out of date and no longer applicable.

Revision detail

A brief summary of what has changed since the last revision allows approvers and those already familiar with the policy to quickly understand changes and the new content.

Owner/approver

Being clear as to who owns and approves any particular document is useful not only for demonstrating that it has been accepted and agreed upon by the appropriate level of management, but it also serves to facilitate feedback and suggestions for updates in future revisions.

Roles and responsibilities

Defining whose responsibility it is to implement, monitor, abide by, and update policies ensures that there is little room for ambiguity with regard to roles.

Executive sign-off

By ensuring that executive sign-off is clearly marked on each document it is clear to the reader that it is endorsed at the highest level and approved for immediate use.

Purpose/overview

This provides a brief overview as to what the policy document covers. This is typically only a paragraph and is intended to allow the readers to gauge if they are looking at the correct policy document before they get to the point of reading every policy statement.

Scope

In all likelihood, the scope section will only be a couple of sentences and will be the same for most policy documents. This explains who the policy document applies to; for example, “This policy applies to all <Company Name> full-time employees, part-time employees, contractors, agents, and affiliates.” Of course, there could be policies that only apply to a particular subset of readers for some reason, and the scope can be adjusted accordingly.

Policy statements

As discussed earlier, these are the guts of the document—they are the policies themselves.

Consistent naming convention

Consistent naming conventions not only for the documents themselves, but also for artifacts they reference, ensure that they are easy to understand and can be applied consistently across the organization.

Related documents

Cross references to other relevant documents such as standards, policies, and processes allow the reader to quickly locate related information.

For ease of reference during an audit, it is prudent to also include references to sections of any relevant regulatory compliance, standards, and legal requirements.

Topics

For ease of reading, updating, and overall management it is probably easier to produce a set of policy documents rather than a single monolithic document.

Selecting how the policies are broken up is, of course, a matter of determining what is most appropriate for your organization. You may have a favorite security framework, such as ISO 27002, for example, from which you can draw inspiration. Similarly, aligning policy topics with a particular regulatory compliance regime may be more aligned with your organization’s objectives. In reality, there are many high-level similarities between many of the frameworks.

SANS, for example, publishes a list of template policies that you can edit for your own needs. At the time of writing, its list of topics are:

Acceptable Encryption Policy

Acceptable Use Policy

Acquisition Assessment Policy

Analog/ISDN Line Security Policy

Anti-Virus Guidelines

Automatically Forwarded Email Policy

Bluetooth Baseline Requirements Policy

Clean Desk Policy

Communications Equipment Policy

Data Breach Response Policy

Database Credentials Policy

Dial In Access Policy

Digital Signature Acceptance Policy

Disaster Recovery Plan Policy

DMZ Lab Security Policy

Email Policy

Email Retention Policy

Employee Internet Use Monitoring and Filtering Policy

End User Encryption Key Protection Plan

Ethics Policy

Extranet Policy

Incident Handling - Chain Of Custody Form

Incident Handling Forms

Incident Communication Log

Incident Contacts List

Incident Containment

Incident Eradication

Incident Identification

Incident Survey

Information Logging Standard

Intellectual Property Incident Handling Forms

Incident Communication Log

Incident Contacts

Incident Containment

Incident Eradication

Incident Form Checklist

Incident Identification

Internet DMZ Equipment Policy

Internet Usage Policy

Lab Anti Virus Policy

Lab Security Policy

Mobile Device Encryption Policy

Mobile Employee Endpoint Responsibility Policy

Pandemic Response Planning Policy

Password Construction Guidelines

Password Protection Policy

Personal Communication Devices and Voicemail Policy

Remote Access Mobile Computing Storage

Remote Access Policy

Remote Access Tools Policy

Removable Media Policy

Risk Assessment Policy

Router and Switch Security Policy

Security Response Plan Policy

Server Audit Policy

Server Malware Protection Policy

Server Security Policy

Social Engineering Awareness Policy

Software Installation Policy

Technology Equipment Disposal Policy

Virtual Private Network Policy

Web Application Security Policy

Wireless Communication Policy

Wireless Communication Standard

Workstation Security (For HIPAA) Policy

This is not an atypical list; however, many of the policies listed will not apply to your organization. This is completely fine.

Storage and Communication

The nature of policies and procedures is meant to lend as much standard communication as possible to the organization as a whole. To do this, policies must be easily accessible. There are many software packages that can not only provide a web interface for policies, but also have built-in review, revision control, and approval processes. Software with these features makes it much easier when there are a multitude of people and departments creating, editing, and approving policies.

Another good rule of thumb is to, at least once per reviewal process, have two copies of all policies printed out. As the majority of them will be used in digital format, there will be many policies that refer to and are in direct relation to downtime or disaster recovery procedures. In cases such as these, they may not be accessible via digital media so having a backup in physical form is best.

Conclusion

Policies are important tools used to express the direction of an organization from a security perspective, clearly articulating expectations and providing a level of consistency. They can also be used to explicitly state and enforce rules that have previously been ambiguous or inferred.

Policies are not set in stone forever—they are living documents that can grow and change in line with your organization.

Chapter 4. Standards and Procedures

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 4th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at sevans@oreilly.com.

Standards and procedures are two sets of documentation that support the policies and bring them to life. In this chapter we will learn what standards and procedures are, how they relate to policies, and what they should contain.

If we consider the policies of an organization to be the “what” we are trying to achieve, standards and procedures form the “how.” As with policies, standards and procedures bring with them many advantages:

Consistency

Worries about the nuances of the implementation of policies at a technology level are removed when consistency is applied. By having a written set of standards and procedures, the rules outlined in the policies can be applied equally across the organization.

Distribution of knowledge

As with policies, it is all well and good for you to know how to implement a policy in practice; however, if those elsewhere in the organization are unaware, then it is not providing much benefit. Standards and procedures disseminate this information for others to consume. Many times organizations fall into the mistakes of relying on institutional or legacy knowledge, which overtime can become almost impossible to distribute amongst current employees. Original decision makers may have moved on from the company, switched roles, or maybe don’t have a photographic memory and all the time in the world.

Setting expectations

Policies have already set the rules and boundaries. However, they do not provide the detail required to consistently achieve those goals in the same way every time. Standards and procedures do this by including a purpose and overview. We’ll also cover in this chapter other pieces and parts of these documents.

Regulatory compliance

Many industries are regulated or “pseudo” regulated. A criterion common amongst nearly every regulatory compliance scheme is the existence of standards and procedures to accompany the policies. By having a set of standards and procedures, you have now ticked a second box on the regulatory compliance checklist! (As long as you’ve also read Chapter 3)

Management endorsement

A management-endorsed set of standards and procedures, as with policies, lends credibility to both the documentation set itself, and by extension the security team.

Standards

Standards provide the “how” portion to a policy at a technology viewpoint without specific procedural detail. For example, many policy statements include the requirement that access be authenticated by use of a password.

A standard that provides more detail as to what constitutes a password should accompany this policy statement. For example, it will most likely cover topics such as complexity requirements, the process for changing a password, storage requirements, ability to reuse passwords or otherwise, or any other related detail.

Separating this into two documents—three once we talk about procedures—provides several advantages:

Documents are easier to consume

A lightweight policy document is easier to navigate and less daunting to read than an all-encompassing policy document the size of a telephone directory. By having the separation of all three types, individuals can have a better idea of which one to use for a specific task.

Lack of repetition

Using the password example mentioned earlier, having to repeat the need to use complex passwords on each mention of password authentication in ancillary policies will become repetitive and leaves plenty of scope to introduce errors. This way the high-level policies can be read easily, and if further clarification is needed the reader can refer to the appropriate accompanying standard.

Ease of maintenance

Lack of repetition means that a change in the standard need only be applied in one place for consistent application across the organization. If standards were rolled into the policy documentation, changes would need to take into account all instances of the affected statements. Missing one of these statements could be catastrophic.

Language

As with policies, the language used within standards documentation should be fairly simple, clear, and use words like “do,” “will,” “must,” and “shall.” They should not be ambiguous or use words and phrases such as “should,” “try,” or “mostly.”

Unlike policies, however, standards can be more specific and detailed in their guidance. Although being specific about technologies in use, standards remain free of specific procedural detail such as commands. This will be explained when we get to procedures.

For example, in Chapter 3 we used the example:

A unique User ID shall be assigned to a user.

The accompanying standards documentation would typically include statements such as:

A User ID is created in the format <first 6 chars of surname><first 2 chars of firstname>, unless this User ID is already in use, in which case...

A User ID shall only be created after HR approval

A User ID shall only be created after Line Manager approval

HR must review and approve user access rights, ensuring that they align with the user’s role prior to the User ID being provisioned.

A record of User ID creation and associated sign-off will be kept in...

A one-way hash function shall be used for the storage of user passwords. Acceptable hashing algorithms are...

These statements enhance, support, and provide more detail to the associated policy statement.

Documents should be designed to be read. There is no need to fill documents with excessively wordy statements. Each policy statement can be only a few sentences, often only one, in a bullet point format.

Procedures

Procedures take the step made from policies to standards and makes another similarly sized step further along the same trajectory. 

Procedures take the detail from standards, which in turn offer guidance based on policies, and provide specific steps in order to achieve those standards at a technology level. This time, the documentation is not intended to describe what we, as an organization, are trying to achieve, but how this is to be implemented at a technology-specific level.

Language

Language is important once more, as ensuring that the desired configuration changes are applied consistently is the ultimate goal. Unlike policies and standards, however, the level of detail will probably depend on corporate culture. For example, it is more appropriate in some organizations to provide an almost keypress-by-keypress level of detail. In others, prescribing which configuration options to make is more appropriate and the administrators are trusted to make a judgment call on which editor they use to make such changes. In most environments the latter is typically sufficient.

Let’s revisit the last statement from the standards example, which was:

A one-way hash function shall be used for the storage of user passwords. Acceptable hashing algorithms are...

The procedure documentation should explain how this is achieved on a specific platform. Because technology platforms differ and procedures are technology-specific, it is entirely likely that there will need to be platform-specific documents created to account for differences between technologies.

For example, to implement platform-specific documentation about FreeBSD on a FreeBSD system, the procedures statement could be something like:

In order to configure system passwords to use the SHA512 hashing algorithm, edit /etc/login.conf, and amend the passwd_format field to read:

:passwd_format=sha512:\

Whereas on a Linux platform the guidance would be:

In order to configure system passwords to use the SHA512 hashing algorithm, execute the following command with root privileges:

authconfig --passalgo=sha512 --update

Both are systems that have a Unix heritage, and both routes ultimately achieve the same goal. However, the precise method at which the goal is reached is clearly articulated to ensure consistency of application across platforms and teams.

Document Contents

As with policies, documentation for standards and procedures should contain a few key features:

Revision control

At the very least, this should include a version number and an effective date for the document. This allows a user in possession of two versions of a document to quickly establish which version is current and which is out of date and no longer applicable.

Owner/Approver

Being clear as to who owns and approves any particular document is useful not only for demonstrating that it has been accepted and agreed upon by the appropriate level of management, but it also serves to facilitate feedback and suggestions for updates in future revisions.

Purpose/Overview

This provides a brief overview as to what the policy document covers. This is typically only a paragraph and is intended to allow the readers to gauge if they are looking at the correct policy document before they get to the point of reading every policy statement.

Scope

In all likelihood, the scope section will only be a couple of sentences and will be the same for most policy documents. This explains who the policy document applies to; for example, “This policy applies to all <Company Name> employees and affiliates.” Of course there could be policies that only apply to a particular subset of readers for some reason. The scope could be anywhere from the previous example of the entire company, to a certain level of management or engineering, or specific users of an application.

Procedure statements

As discussed earlier, these are the guts of the document—the procedures themselves. Ensure these statements are clear and concise, and reviewed on a regular basis for accuracy. The technology in our environments are constantly changing, and so should the directions on how to manage them. Technical writing at this level can be challenging as it can have a variety of audiences, so having the individuals that will be using these documents be a part of the review and approval process will be an integral part in making each one a success. Some good points to follow for procedure writing include:

Write actions out in the order in which they happen.

Use the active voice.

Use good grammar.

Explain your assumptions, and make sure your assumptions are valid.

Write concisely. In technical writing, clarity and brevity is your goal.

Use jargon and slang carefully, or not at all.

Consistent naming convention

Consistent naming conventions not only for the documents themselves, but also for artifacts they reference, ensure that they are easy to understand and can be applied consistently across the organization.

Related documents

Cross references to other relevant documents such as standards, policies, and processes allow the reader to quickly locate related information.

For ease of reference during an audit, it is prudent to also include references to sections of any relevant regulatory compliance, standards, and legal requirements.

Conclusion

As a whole, policies, standards, and procedures offer a high-level administrative overview down to the specific technology and step-by-step implementation. While each has its own functions, they all must be written with the skill level of the reader in mind. A clear and concise set of documentation makes a huge difference in creating a standardized and well-understood environment.

Chapter 5. User Education

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 5th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at sevans@oreilly.com.

User education and security awareness, for the most part, is broken in its current state in the majority of implementations. For a successful user education program, it is best to find a way to demonstrate with the right type of metrics that you are successfully implementing change and producing a more secure line of defense. A large portion of the information security industry is focused on perimeter security, VPNs, WAFs, email filtering, etc. However, we are beginning to see a shift from strictly data-level protection to an increase in user-level security and reporting. The security as a process and defense-in-depth mentality must be filtered down and implemented into our user training.

Before you spend money on threat intel that may tell you how better to defend your specific sector, it would be best to start where everyone is being attacked. One of the largest threats today is the targeting of our weakest link: people. According to the 2015 Verizon Data Breach Investigations Report (DBIR):

Phishing remains popular with attackers. Campaigns have evolved to include the installation of malware as the second stage of an attack. Our data suggests that such attacks could be becoming more effective, with 23% of recipients now opening phishing messages and 11% clicking on attachments. It gets worse. On average, it’s just 82 seconds before a phishing campaign gets its first click.

And again with the updated copy of this publication, there is no surprise that the 2021 Verizon DBIR had this to say:

Even in a year as unexpected as 2020, there are some things we can trust to stay the same. Phishing remains one of the top Action varieties in breaches and has done so for the past two years.

In this chapter we will demonstrate how to provide more value than the basic training offered in the majority of organizations.

Broken Processes

The reason that most Security Awareness Training programs fail is because they are trainings…not education.¹

Experience and time in the industry shows that the Computer Based Trainings (CBTs) that organizations require their employees to complete annually (or sometimes more often) are comparable to a compliance check box. It is a broken process. The employee is required to complete and pass this training for continued employment. Once the process is complete, the knowledge is either forgotten or greatly reduced. One of the largest proven gaps occurs when end users do not bring the information forward into their day-to-day working lives like they should. This is a large disconnect where it means the most, and it is known as the Ebbinghaus forgetting curve (Figure 5-1). Repetition based on active recall has been demonstrated as effective in other areas for avoiding the curve and, therefore, is the foundational design such awareness programs should be based on. From “The Ebbinghuas forgetting curve”:

...basic training in mnemonic techniques can help overcome those differences in part. He asserted that the best methods for increasing the strength of memory are:

Better memory representation (e.g., with mnemonic techniques)

Repetition based on active recall (especially spaced repetition)

Figure 5-1. Graph representing the Ebbinghaus Curve

There are significant additions and changes that can be made to transform user training into an effective and long-lasting security education.

Bridging the Gap

Repetition and real-life applicability is a proven, successful way to bridge the gap of compliance, teaching our users real-life skills and helping secure the infrastructure that we are responsible for protecting. This is best implemented with a comprehensive hands-on security phishing and awareness rewards program. A full program design will provide a maturity that the CBTs do not. While CBTs are a good value-add and can be used to reinforce real-life scenarios, relying on them as a primary means of security awareness training will not provide the value or insight to the enterprise’s sometimes most overlooked first line of defense, the endusers themselves. By consistently reinforcing the CBTs with a custom-built awareness program, you increase the end user’s skills and boost the organization’s immunity to phishing and social engineering threat factors.

Building Your Own Program

Building a mature and strategic program from the ground up is achievable with executive support and cultural alignment. An awareness program need not equate to thousands of dollars spent on creating flashy presentations and brown bag luncheons to draw crowds. Teaching by example and rewarding good behavior is what will improve upon the user’s awareness:

The point has never been to make everyone experts in security, it has always been to arm the employees with basic knowledge so that in the event something out of the ordinary occurs, it may help notify the security team.²

An important takeaway and key point to remember is that it is not the employees’ responsibility to know the difference between a legitimate phish and spam, or to know that they should be hovering over links in emails before clicking. It is our job to have a program that is open enough and easy enough for them to report abnormalities or when something is not quite right. At the end of this chapter you will find a template slideshow for a security awareness program.

Establish Objectives

The direction of an organization’s security awareness program should be tailor-fit and reassessed periodically. With the constant changing threat landscape, maturity of user understanding, and a progressing industry, the objectives should be thought of as moving targets. An objective one year of decreased malware removals on desktops may mature past that to increased reporting of phishing/vishing attacks. However, establishing an aggressive set of objectives can result in a failed or unrealistic program. Concentrating on one or two achievable objectives at the beginning of a new program will allow you to accomplish a more specific goal. You can then adjust the target periodically to reflect the organization’s and program’s maturity.

Establish Baselines

Many organizations do not have formal security awareness training, so establishing a baseline should begin with a live-fire exercise testing the skills and real-world knowledge of a good subset of your users.  It should be a standard practice to have a realistic outlook on where your security posture stands in relation to not only technical baselines but also cultural norms. It is important to know how the users currently respond to threats and irregularities. Establishing an engagement with a certified and skilled penetration testing company can help you baseline these responses. By having a third party assess the skills of your users with professional phishing campaigns, you will gain valuable insight into data that you may currently not have.

Scope and Create Program Rules and Guidelines

When the user or employee is being treated essentially as a customer, rules and guidelines should be well thought out and strategized. Miscommunications will only impede the learning process, making succeeding with the program more difficult. Align the rules to be consistent with the organization’s culture in order to have a higher adoption rate. Having multiple levels of input will enable you to have clear and concise program instructions and rules, leading to an easier implementation.

Implement and Document Program Infrastructure

You are taught in driver’s education to wear your seat belt, look both ways, and adjust your mirrors. The first time you have a close call or, even worse, a real accident, you now have a real-world experience that your mind falls back on each time you make a decision. It is the same with security awareness. The shock of the accident now gives employees pause when future emails show up that may look a little odd and out of place. Afterward, the training teaches them what could possibly be at risk when they click through the illegitimate link. Setting up the phishing attacks to automatically redirect to a website that aligns with the program theme will create a connection between real-life events and the message being presented for education.

Positive Reinforcement

One of the most important parts of building a program is letting users know that it is OK if they fall victim to the attack. This must be a consistent message throughout the education material. The more comfortable users feel reporting the incident, the more cooperation and adoption you will witness. Assure the user that it will always be better coming from an internal training attempt than a real phishing attack, and practice makes perfect. The training should include what to look for and more importantly, how to report something abnormal. With a great first line of defense and solid Incident Response (IR) procedures, you will be far better off securing the human element, which is sometimes the weakest security link, and a great detection asset. 

Gamification

Gamification is actually a scientific term that roughly means applying game principles to a situation. The simplest definition of those principles is: 1) Goal establishment, 2) Rules, 3) Feedback, and 4) Participation is voluntary.³

Being able to reward good behavior is an essential part of the program as well. Employees should not feel ashamed to come to the right people for help, nor afraid of being reprimanded for making a mistake. Gamification works well in many aspects of life, so why should this be any different? Turn the program into something catchy and a small budget cannot just satisfy your expectations, but exceed them. Making a lottery of gift cards, discounted services, and other items to enforce the brand of the program and put something in the user’s hand will reinforce your message.

Define Incident Response Processes

Incident Response looks different in every organization. If you have a current proven method of IR you are already well on your way to including an awareness program into your current structure. Use the newly created program as a case study for testing procedures and policies. This will allow you to flush out any inconsistencies, inefficiencies, or unplanned situations. Assessing each step of the process will give the necessary information to add or change policies to fit the needs of the organization around certain types of attacks. 

Gaining Meaningful Metrics

Without knowing the baseline for metrics, you won’t know how far your program has improved. Work toward long-term goals regarding the overall security. This will help build metrics into the already forming security program to keep it on track.

Successful metrics programs include well-defined measurements and the necessary steps to obtain them.⁴

Measurements

There are an abundance of measurements to take throughout a security awareness program. Depending on your program and your goals, they may have to be more tailor-fit totals as opposed to the base set listed here.

Here are some common totals to track:

Emails sent

Emails opened

Links clicked

Credentials harvested

Reports of phishing attempts

Emails not reported on

Hits on training sites

Tracking Success Rate and Progress

Keeping track of click percentages, phishes reported, and incidents reported is a good start as well as necessary for any complete metric collection. However, charting your gains and losses with structured data over time will give your organization a deeper understanding of progress. Successful education and retained knowledge will be apparent with the increase and decrease of certain measurements and the success of goals set for metrics. Periodic assessment of shifts in metrics should be performed to assist with guidance of the education program’s goals and other possible implementations or changes in the current environment’s security structure.

Important Metrics

Measures are concrete, usually measure one thing, and are quantitative in nature (e.g., I have five apples). Metrics describe a quality and require a measurement baseline (I have five more apples than I did yesterday).⁵

The metric of how much your security posture has increased in reference to your baseline is the key goal and quality control. Seeing increased reporting changes in suspicious activity on your network should align with a lower amount of malware, DNS queries to blocked sites, or other activity on the network that would lead an analyst to believe the possibility of a targeted attack has been blocked. The ability to link key metrics back to specific departments, buildings, or roles provides the information you need to scope  more directed education.

Conclusion

User education can be a massive undertaking, but it will provide a strong layer of defense and monitoring if done correctly. It should not be the first step toward creating a more secure environment, but can greatly increase protection once a strong baseline has been created. For the best results, remember to always use a carrot and not a stick. In [[Link to Come]], you can find education templates that can be used for user redirection, instruction, and award rules.

¹ David Kennedy, “The Debate on Security Education and Awareness”.

² Ben0xa, “Security Awareness Education”.

³ Ira Winkler and Samantha Manke, “How to create security awareness with incentives”.

⁴ Building an Information Security Awareness Program: Defending Against Social Engineering and Technical Threats - Bill Gardner & Valerie Thomas

⁵ CIO Council, “Performance Metrics and Measures”.

Chapter 6. Incident Response

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 6th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at sevans@oreilly.com.

As the name suggests, incident response is the set of processes and procedures that are initiated once a security incident has been declared. In modern-day computing, incidents range from a single compromised endpoint to complete network compromises resulting in massive data breaches. Data breaches and enterprisewide attacks are becoming more and more common, and thus incident response has grown in meaning beyond merely these processes and procedures to encompass an entire discipline within information security.

In this chapter we will discuss the various processes involved in incident response, tools and technology options, and the most common forms of technical analysis that you are likely to need to perform during an incident.

Processes

Incident response processes are an integral component of being able to react quickly in the event of an incident, determine a nonincident, operate efficiently during an incident, and improve after an incident.  Having processes in place before an incident begins will pay dividends in the long run.

Pre-Incident Processes

The processes associated with incident response are not merely concerned with what happens during an incident. If there are no processes in place to recognize that an incident is taking place, that the incident response process should be initiated, and those responsible for incident response notified, there is little point in having processes to deal with the incident, as they will never be called upon.

The pre-incident processes do not need to be complex; in fact, they should most definitely not be. The point of these processes is merely to determine if there is a potential incident, and to initiate the incident response process—that’s it!

Having been through multiple iterations of internal incident response, we can say that the most effective processes we have worked with include the following:

Leverage existing processes for dealing with events

Most organizations deal with outages, configuration issues, user-reported issues, and other events. Don’t try to set up a parallel set of processes, but leverage what is already there—in all likelihood, the same people who deal with these issues will be the first to hear of an issue anyway. Just modify or supplement existing processes to include calling the incident response contact in the event of an expected incident, much like they already know to call the on-call Unix person when a Linux host fails in the middle of the night.

Define an incident

If you do not define what you class as an incident, you will either get called for every support call or not get called during a breach of four million records. If it is not simple to define what an incident is, you can opt for wording like, “once a manager has determined that an event is a security incident...” This way you have at least defined that any event will have already progressed beyond triage by first-line support and someone experienced enough to make the determination has made a judgment call.

Incidents can also be defined and categorized using certain thresholds. For example with a phishing email that is reported in your organization as making it to email inboxes. If it was one user, it could be a severity 3 and follow one process, but if it ended up in all users inboxes, there will definitely be additional steps (and in multitude) of that same incident. 

The result of a pre-incident process is nearly always to initiate the IR process by declaring an incident and calling the contact for incident response.

Note

An incident turns out to be nothing can always be downgraded again to a standard operations event. It is better to be called for a suspected incident that transpires to be nothing, than to not be called for fear of a false positive.

It is in everyone’s best interest to communicate clearly and early on. It not only saves time and effort fixing miscommunication and hearsay issues, but also allows those individuals fixing the downtime or incident the time to fully concentrate on the issue at hand. No downtime is too small for proper communication!

Incident Processes

The processes that take place during an incident, particularly from a technology perspective, cannot be too prescriptive. Incidents, like many operational problems, are far too varied and numerous to prescribe precise courses of action for all eventualities. However, there are some processes that are worth sticking to:

Define an incident manager

This does not have to be the same person for every incident, but should be someone who is senior enough to make decisions and empower others to complete tasks. The incident manager will run the response effort and make decisions.

Define internal communications

Communication between everyone working on the incident to avoid duplication of work, promote the sharing of information, and ensure that everyone is working toward a common goal is key. We would recommend:

Open a “war room.” That is, use an office or meeting room to perform the role of center of operations for anyone in the same physical location. This is used as the central point for coordination of efforts.

Keep a conference bridge open in the war room. This allows people who are remote to the physical location to check in, update those in the war room, and obtain feedback. If there is no physical war room, this will often serve as a virtual war room.

Hold regular update meetings. Regular updates allow people to move away, work in a more concentrated fashion, and report back regularly rather than feeling as if they are looked over and reporting back haphazardly. Typically, meeting every hour works well until the situation is well understood.

Allocate the task of communicating internally to stakeholders. Management will typically want to be kept abreast of a larger incident. However, sporadic communication from a number of people can send mixed messages and be frustrating for both management and the incident response team. A single point of communication between the two allows stakeholders to receive frequent, measured updates.

Define external communications

In many cases, but not all, some external communication may be required. Typically, this is because customers or other departments will be affected by the incident in some way. This sort of communication should not be taken lightly as it affects the public image of the organization and internal technology department. If you are considering undertaking any external communications yourself, rather than allowing your corporate communication or PR team to do it, we would suggest you read Scott Roberts’ “Crisis Comms for IR” blog post on the topic.

Determine key goals

By determining the goals that you wish to achieve in the event of an incident, you can ensure that all actions are taken with these goals in mind. By goals we do not mean simply “fix it,” but considerations such as “preserve chain of custody for evidence” or “minimize downtime.” This is discussed in more depth in Chapter 7.

High-level technology processes

As mentioned before, it is difficult to account for all eventualities, so being prescriptive with technology-based remedies may be difficult; however, there are some high-level processes that may be in place. For example, there may be policies regarding taking snapshots of affected systems to preserve evidence, ensuring that staff stop logging in to affected systems, or a blackout on discussing incidents via email in case an attacker is reading internal email and will get a tip off.

Plan for the long haul

Many incidents are over in just a few hours, but many last substantially longer, often weeks. It is tempting to pull in all resources to help on an incident in the hopes of a timely conclusion, but if it becomes clear that this is not the case, you should prepare for a longer term course of action. Ensure people are sent away to get rest so that they can come in and cover the next shift, and keep those working fed and watered to prevent fatigue. Try not to burn everyone out, as this can be a game of endurance.

Post-Incident Processes

Once an incident is over, it is very valuable to hold a lessons-learned session. This allows for feedback regarding what worked well and what worked less well. It also allows you the chance to update processes, determine training requirements, change infrastructure, and generally improve based on what you learned from the incident. This information can be captures in an After Action Report (AAR).

It is recommended that this session be held a short time after the incident closes. This offers a few days for people to reflect on what happened, gather some perspective, and recover, without leaving it so long that memories fade or become distorted with time. Using this session to update documentation, policies, procedures, and standards will also allow for updated tabletops and drills.

Outsourcing

Many people do not wish to manage incidents internally, at least not beyond the initial triage point, and would rather bring in external subject matter expertise as required. This is an option that works for many people. However, we would recommend that if this is the route that you decide to take, negotiate contracts, nondisclosure agreements, and service-level agreements before an incident happens. When you are elbow deep in an incident is not the time to be negotiating with a potential supplier about when they can spare someone and what rates you will have to pay.

Tools and Technology

It would be easy to list a large number of technologies that are typically used by incident response professionals, especially in the field of digital forensics. However, a lack of experience in this area can make it easy to misinterpret results, either via a lack of experience with the specific tools or by not understanding the context of what is fully happening.

Fully understanding an environment, knowing what the various logs mean, knowing what should and should not be present, and learning how to use the tools that are already present can vastly increase the chances of managing an in-progress incident. Mid-incident is not the time to learn how to conduct a forensic investigation; it is better left to someone who has some prior experience in this field. That said, a high level appreciation of what can happen during an incident can be achieved by reviewing some high-level topics. We also discuss some example tools that which can be used to assess what is happening in an environment during an incident.

Log Analysis

The first port of call, as with any type of operational issue, is of course the humble logfile.  Application and operating system logfiles can hold a wealth of information and provide valuable pointers to what has happened.

If logs are stored on the host that generated them, you should remain cognizant of the fact that if someone compromises that host, they can easily modify the logs to remove evidence of what is happening. If possible, the logs stored on your Security Information and Event Management (SIEM) platform should be consulted, rather than referring to logs on the target device.  This not only reduces the chances of log tampering but also provides the facility the ability to query logs across the whole estate at once, permitting a more holistic view of the situation. A SIEM also has the ability to show if a gap in logs has occurred.

When reviewing logs on an SIEM, it is likely that the SIEM’s own log query tools and search language will need to be used. It is also possible that the use of commands such as curl or customized scripts will access data via an API.

If the logs are not being accessed on an SIEM, it is recommended to take a copy, if possible, and to analyze them locally with any preferred tools. Personally, we opt for a combination of traditional Unix command-line tools such as grep, awk, sed, and cut, along with scripts written for specific use cases.

Disk and File Analysis

Analysis of artifacts on storage devices can also provide clues as to what has happened during an incident. Typically, a disk image will yield more information than purely examining files, as this contains not only the files stored on the disk that are immediately visible, but also potentially fragments of deleted files that remain on disk, chunks of data left in slack space, and files that have been hidden via root kits. Using a disk image also ensures that you do not accidentally modify the original disk, which ensures the integrity of the original should there be legal proceedings of some kind. To obtain a copy of the disk image traditionally means taking a host down and using a tool such as ddfldd or a commercial equivalent to take an image of the disk, which is saved to another drive and then examined offline. Unfortunately, this causes downtime.

Disk and File Analysis in Virtual Environments

In most virtualized and some cloud computing environments, taking downtime to image a disk is less of a problem because all the major vendors have various snapshot technologies that can be used to take an image of a guest operating system.  However, these technologies will often compress disk images, destroying unused space and losing much of this needed information.

Once a disk image has been obtained, various commercial tools can be used to analyze the filesystem to discover files of interest, construct timelines of events, and other related tasks. In the open source/free space, the old classics The Sleuth Kit and Autopsy remain favorites.

If a simple recovery of files is all that is desired, PhotoRec is a simple-to-use tool that yields surprisingly good results. Despite the name, it is not limited to photos.

Memory Analysis

Code that is executing, including malicious code, is resident in RAM. If you can obtain a memory dump from a compromised host—that is, a file that contains a byte-for-byte copy of the RAM—then analysis can be performed to discover malicious code, memory hooks, and other indicators of what has happened.

The most popular tool to analyze these RAM dumps is the Volatility Framework (see the wiki on GitHub).

Obtaining RAM dumps will vary from OS to OS and it is a constantly changing field, so we would recommend checking the Volatility documentation for the latest preferred method.

For virtualized platforms, however, there is no need to dump RAM using the OS, as the host can take an image of the virtual memory. Following are the three most common examples of how to achieve this:

QEMU

pmemsave 0 0x20000000 /tmp/dumpfile

Xen

sudo xm dump-core -L /tmp/dump-core-6 6

VMWare ESX

vim-cmd vmsvc/getallvms
vim-cmd vmsvc/get.summary vmid
vim-cmd vmsvc/snapshot.create vmid [Name] [Description]
  [includeMemory (1)] [quiesced]

PCAP Analysis

If you have any tools that sniff network traffic inline or via a span port or inline utilities such as an IDS/IPS or network monitoring device, there is every chance that you could have sample packet capture (PCAP) files. PCAP files contain copies of data as it appeared on the network and allow an analyst to attempt to reconstruct what was happening on the network at a particular point in time.

A vast number of tools can be used to perform PCAP analysis; however, for a first pass at understanding what is contained in the traffic we would recommend using IDS-like tools such as Snort or Bro Security Monitor configured to read from a PCAP, as opposed to a live network interface. This will catch obvious traffic that triggers their predefined signatures.

Some staples for conducting PCAP analysis include the following tools:

tcpdump produces header and summary information, hex dumps, and ASCII dumps of packets that are either sniffed from the wire or read from PCAP files. Because tcpdump is command line, it can be used with other tools such as sed and grep to quickly determine frequently occurring IP addresses, ports, and other details that could be used to spot abnormal traffic. tcpdump is also useful because it can apply filters to PCAP files and save the filtered output. These output files are themselves smaller PCAPs that can be fed into other tools that do not handle large PCAPs as gracefully as tcpdump does.

Wireshark is the de facto tool for analysis of PCAP data. It provides a full GUI that allows the user to perform functions such as filtering and tracking a single connection, providing protocol analysis and graphing certain features of the observed network traffic. Wireshark does not, however, handle large files very well, and so prefiltering with tcpdump is recommended.

tshark (bundled with Wireshark) is a command-line version of Wireshark. It is not quite as intuitive or easy to use, but being on the command line allows it to be used in conjunction with other tools such as grep, awk, and sed to perform rapid analysis.

All in One

If you are familiar with LiveCDs such as Kali in the penetration testing world, then an approximate equivalent for Incident Response is CAINE. CAINE is a collection of free/open source tools provided on a single LiveCD or USB thumbdrive. It can be booted without prior installation for quick triage purposes. 

Conclusion

Incident response is not a prescriptive process from beginning to end. However, there are some key areas that can be process driven, such as communication, roles and responsibilities, and high-level incident management. This allows incidents to be effectively controlled and managed without bogging down technical specialists with complex decision tree processes. NIST has a great framework for handling incidents that can be a great asset to be familiar with as well.

Incident response is an area of information security that most hope they will never have to be involved with; however, when the occasion comes you will be glad that you have prepared.

Chapter 8. Industry Compliance Standards and Frameworks

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 8th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at sevans@oreilly.com.

Businesses may be required to conform to one or more regulatory compliance regimes, which are administered by a variety of governing bodies. Failure to comply with these standards can come with heavy fines or in some cases hindering ability to conduct business (such as preventing the capability of processing credit card transactions). Frameworks differ from regulatory compliance standards by the fact that they are not required for a specific industry or type of data; they are more of a guideline.

The requirement to comply with one standard or the next does provide a few benefits to your organization. Certain standards leave significant room for interpretation, giving you the ability to tie security measures that should be implemented to a portion of that same standard. When compliance is involved there are now social, political, and legal components that can be leveraged to implement security controls and process changes that may not have been possible otherwise. It also may present the opportunity to piggyback off another department that has excess budget for a project.

As both standards and frameworks change frequently over time, in this chapter we’ll just focus on explaining what each is used for, and possibly some “gotcha’s” that some may have. 

Industry Compliance Standards

Compliance standards are a minimum guide at best, not a complete security program. It is easy and lazy to be just a “check box checker” when going through implementing controls in a compliance list. It’s possible to technically be compliant with a standard and still not have a secured environment, as well as not be compliant but be completely secure (although less likely). Many standards leave room for the imagination and can be interpreted in different ways. However, following common best practices will usually lead to compliance as a side effect. The majority of standards listed here are from the United States because international organizations have a whole different set of reporting requirements.

Most organizations will have a compliance officer who may not be in the security department because the majority of regulatory standards are not based on the technology behind information security, but as an overall solution to a greater problem. For example, HIPAA is focused on the safety of patients and patient records, no matter if it is on a piece of paper or a piece of technology.

Family Educational Rights and Privacy Act

Family Educational Rights and Privacy Act (FERPA) is a federal law that protects the privacy of student education records in both public and private schools, as well as in higher education. As this was enacted in 1974 it has no specific information related to technology, which leaves a large area open for interpretation in regards to information security practices and protections needed. It contains phrasing that should be interpreted as the prohibition of releasing or disclosing any PII (Personally Identifiable Information), directory information, or educational information of students to a third party.

PII can only be disclosed if the educational institution obtains the signature of the parent or student (if over 18 years of age) on a document specifically identifying the information to be disclosed, the reason for the disclosure, and the parties to whom the disclosure will be made.

Directory information is defined as “information contained in an education record of a student that would not generally be considered harmful or an invasion of privacy if disclosed”—for example, names, addresses, telephone numbers, and student ID numbers.

Educational records are defined as “records, files, documents, and other materials maintained by an educational agency or institution, or by a person acting for such agency or institution.” This includes students transcripts, GPA, grades, Social Security number, and academic and psychological evaluations.

While there have been recorded requests for better cybersecurity recommendations and controls in many of the amendment comments over the years, it remains a very lax standard in regards to any type of real information security requirements.

Gramm-Leach Bliley Act

Gramm-Leach Bliley Act (GLBA) is a law that was passed in 1999 to reform and modernize the regulations affecting financial institutions. It is comprised of seven titles, and title five lists the two paragraphs on information security:

Title V – PRIVACY Subtitle A - Disclosure of Nonpublic Personal Information Section 501 – Protection of Nonpublic Personal Information (a) PRIVACY OBLIGATION POLICY It is the policy of the Congress that each financial institution has an affirmative and continuing obligation to respect the privacy of its customers and to protect the security and confidentiality of those customers’ nonpublic personal information. (b) FINANCIAL INSTITUTIONS SAFEGUARDS In furtherance of the policy in subsection (a), each agency or authority described in section 505(a), shall establish appropriate standards for the financial institutions subject to their jurisdiction relating to administrative, technical, and physical safeguards- (1) to insure the security and confidentiality of customer records and information (2) to protect against any anticipated threats or hazards to the security or integrity of such records; and (3) to protect against unauthorized access to or use of such records or information which could result in substantial harm or inconvenience to any customer.

In 2021 GLBA was amended addressing the increase of widespread data breaches and cyber attacks to include an extremely larger list of information security controls such as:

Access Controls, Audit Trails

Change Management

Cost

Disposal Procedures

Encryption

Multi-Factor Authentication

Secure Development Practices

System Inventory

System Monitoring

Third-Party Standards and Frameworks

Written Risk Assessment

“Sensitive” Customer Information

GLBA compliance is mandatory for financial institutions including banks, mortgage brokers, real estate appraisers, debt collectors, insurance companies, and privacy companies. Initially having such a broad and compact section it left a large amount up to interpretation during implementation. The more current amendment has given greater guidance on best practices. For more information to help with this compliance standard there are two documents; the Interagency Guidelines and the IT Examiners Handbook that were both created to assist with implementing security practices surrounding GLBA compliance.

Health Insurance Portability & Accountability Act

Health Insurance Portability & Accountability Act (HIPAA) was enacted in 1996 as law and establishes national standards for electronic healthcare records. It includes any organization that stores or processes ePHI (Electronic Protected Health Information) healthcare providers, health plans, and clearinghouses. Thankfully, we start to see a little more definition regarding technology in the verbiage of this act compared to others we’ll cover. There are more than fifty “implementation specifications,” divided into  administrative, physical,  technical, and policies, procedures, and documentation requirements. Most of these involve having policies and procedures in place. Addressable specifications involve performing a “risk assessment” and then taking steps to mitigate the risks in a way that’s appropriate for your organization. One of the largest HIPAA penalties against a small organization was levied not because an event occurred, but because the organization failed to address the possibility that it might. Loss of ePHI can cause significant harm to not only the patients whose data has been compromised, but also the provider and individuals at fault as they are required to report violations to the US Department of Health and Human Services (HHS) and the Federal Trade Commission (FTC). They are also the ones who would be on the receiving end of extremely large fines and possibly even jail time. The HHS provides a breakdown of each portion of the security rule portion of HIPAA and assistance with the implementation of the security standards.

Payment Card Industry Data Security Standard (PCI DSS)

Payment Card Industry Data Security Standard (PCI DSS) is a standard for organizations that store, process, or transmit credit cards and credit card data. PCI DSS is required by the card brands (MasterCard, Visa, Discover, American Express, and JCB) and is administered by the Payment Card Industry Security Standards Council. PCI DSS was created to increase security controls around cardholder data in an attempt to reduce credit card fraud. Failure to validate compliance can result in fines or other penalties, even including the removal of credit card processing capabilities.

PCI DSS regulates cardholder data (CHD). Cardholder data is any personally identifiable information (PII) associated with a person who has a credit or debit card. This includes the primary account number (PAN), cardholder name, expiration date, or service code. For current versions of this compliance standard you can visit the PCI SSC website.

Sarbanes-Oxley Act

Sarbanes-Oxley Act (SOX) is a law enacted in 2002 to set forth security requirements for all US public company boards, management, and public accounting firms. Portions also apply to privately held companies in regards to withholding or destroying information to impede any federal investigations. SOX has 11 sections and was created to ensure corporate corruption and scandals such as Enron and Worldcom don’t happen again. Many organizations that have SOX compliance also abide by either the COSO or COBIT frameworks, which we cover later in this chapter.

The two principle sections that relate to security are Section 302 and Section 404:

Section 302 is intended to safeguard against faulty financial reporting. As part of this section, companies must safeguard their data responsibly so as to ensure that financial reports are not based upon faulty data, tampered data, or data that may be highly inaccurate.

Section 404 requires the safeguards stated in Section 302 to be externally verifiable by independent auditors, so that independent auditors may disclose to shareholders and the public possible security breaches that affect company finances. Specifically, this section guarantees that the security of data cannot be hidden from auditors, and security breaches must be reported.

Frameworks

Frameworks are different from compliance standards in that they are not a requirement. They are industry- or technology-specific guidelines created to assist in organizing thoughts, practices, and implementations.

Center for Internet Security

Center for Internet Security (CIS) not only has a framework for assisting with cyber attacks, but also provides benchmarks, workforce development, and other resources such as whitepapers, publications, newsletters, and advisories. It offers in-depth system-hardening guidelines for specific operating systems and applications. CIS also couples with NIST to combine frameworks for the purposes of securing critical infrastructure, and other cross framework and compliance references. 

Cloud Control Matrix

Cloud Control Matrix (CCM) is a framework built specifically with cloud security in mind by the Cloud Security Alliance (CSA). It assists in tying together specific cloud security concerns and practices to all major compliance standards and frameworks. CSA also has some great workgroups for specific sectors using cloud solutions. At the time of this writing, it is composed of 197 control objectives that are structured in 17 domains covering all key aspects of cloud technology and is considered a de-facto standard for cloud security assurance and compliance.

The Committee of Sponsoring Organizations of the Treadway Commission

The Committee of Sponsoring Organizations of the Treadway Commission (COSO) is made up of five organizations: the American Accounting Association, American Institute of CPAs, Financial Executives International, The Association of Accountants and Financial Professionals in Business, and The Institute of Internal Auditors. It aims to provide guidance on enterprise risk management, internal control, and fraud deterrence.

Control Objectives for Information and Related Technologies

Control Objectives for Information and Related Technologies (COBIT) is a high-level framework created by Information Systems Audit and Control Association (ISACA) to assist in creating secure documentation, implementation, and compliance. COBIT is subdivided into four domains—Plan and Organize, Acquire and Implement, Deliver and Support, and Monitor and Evaluate—and aims to align itself with other more detailed standards. While some frameworks are free, COBIT is available for purchase through its website.

ISO-27000 Series

International Organization for Standardization (ISO) is an independent, non-governmental international organization that has created over 20,000 sets of standards across a variety of industries including food services, technology, and agriculture.  Out of these frameworks, the 27000 series has been used for the topic of information security, specifically 27001-27006. ISO standards are also a paid framework.

ISO-27001

Provides requirements for establishing, implementing, maintaining, and continuously improving an Information Security Management System.

ISO-27002

Establishes guidelines and general principles for initiating, implementing, maintaining, and improving information security management within an organization.

ISO-27003

Information Security Management System implementation guidance.

ISO-27004

Provides guidance on the development and use of measures and measurement for the assessment of the effectiveness of an implemented Information Security Management System and controls.

ISO-27005

Provides guidelines for information security risk management (ISRM) in an organization.

ISO-27006

Requirements for bodies providing audit and certification of Information Security Management Systems.

MITRE ATT&CK

The MITRE ATT&CK framework was created in 2013 to document common tactics, techniques, and procedures (TTPs) that advanced persistent threats that are commonly used against Windows environments. Within the MITRE research project FMX there was a need to document adversary tactics and behaviors. It now serves as a common place that both offense and defense can use to improve defense and detection capabilities.

The ATT&CK is updated bi-annually, is broken up into a section for enterprise and cloud environments, as well as a section for mobile TTPs. Additionally MITRE provides a large amount of resources to assist in utilizing the information they provide.

NIST CyberSecurity Framework

The National Institute of Standards and Technology operates as part of the United States Department of Commerce, creating standards for many sections of US infrastructure. This framework was created with both industry and government participation, and it consists of standards, guidelines, and practices surrounding critical infrastructure security. The framework uses common industry business drivers to guide and manage risks, protect information, and safeguard the people using the business’s services. It consists of three parts: the Framework Core, the Framework Profile, and the Framework Implementation Tiers, which all put a majority focus on risk management. The Core provides a set of cybersecurity activities and outcomes in language that is easy to understand.  Profiles are mostly used to identify and prioritize the improvements in the cybersecurity posture at an organization. Lastly, the Implementation Tiers assist organizations by providing context around managing cybersecurity risk.

Regulated Industries

As mentioned previously, some industries are more heavily regulated than others. The basis of these regulations comes from the sensitivity of data and the likelihood of it being stolen and used for malicious purposes. There is a large black market for stolen data that is used for both credit and identity theft. You’ve already read about the different regulation types; however, certain sectors are regulated strictly—or at least strict regulations are attempted—for legitimate reasons.

Financial

The financial industry includes thousands of institutions, such as banks, investment services, insurance companies, other credit and financing organizations, and the service providers that support them. They can vary widely in size and the amount of data processed, ranging from some of the world’s largest global companies with thousands of employees and many billions of dollars in assets, to community banks and credit unions with a small number of employees serving individual communities.

Some of the major risks the financial sector must be concerned with are account takeovers, third-party payment processor breaches, ATM skimming and other Point of Service (POS) vulnerabilities, mobile and internet banking exploitation, and supply chain infiltration.  Studies have shown that this sector is one of the most common to see outdated legacy systems throughout organizations, giving would-be attackers an easy foothold once access has been gained.

Government

The government, and specifically in this case, the United States government, has pretty much every type of data imaginable to protect. From the large three-letter acronym agencies such as NSA, FBI, IRS, and FDA to smaller local government offices that contract with their own IT guy, there is an extremely broad landscape covered in this one sector. With the fast changing security landscape it has been seen to hinder progress due to the fact that government organizations can vary greatly from commercial businesses with the process length of approving changes and upgrades, ability and willingness to adopt new technology, and overall atmosphere of personnel.

The breadth of information contained within government agencies means a constant stream of high-profile attacks by organized crime, hacktivists, and state-sponsored agents.  One of the biggest breaches in 2015 happened to be the government Office of Personnel Management (OPM), which shed some light onto the department’s lack of security. The attack was ongoing for an entire year and an estimated 21.5 million records were stolen. Other data breaches included voting records and information, USPS employee information, IRS tax return data, NSA exploits and attack tools used against foreign adversaries, and other highly sensitive and potentially harmful data.

Healthcare

The healthcare industry at this point in time continues to be one of the least secured industries. Healthcare companies saw a rapid change from a majority of paper records to almost fully electronic patient records, and just like the financial sector, it is riddled with out-of-date legacy devices. For a long time the FDA had strict control over the operating system revision and patch level, restricting the ability for medical devices to be upgraded and still maintain FDA approval. This, coupled with the rapid growth and underestimation of the inherent risk of a large quantity of sensitive medical and patient data stored electronically, creates a largely complicated, insecure environment. While the FDA has relaxed its requirements to allow security patches to be applied, software vendors and organizations have yet to keep up and implement best practice security controls.

Both the Department of Health and Human Services’ (HHS) Office for Civil Rights (OCR) and attorneys general have the power to issue penalties for the failure of following HIPAA guidelines and as the result of PHI breaches. Not only can they enforce financial penalties, but can also include criminal lawsuits. In the past several years (2016-2021) HHS recorded over to $80 million in HIPAA settlement payments. There are several tiers for both financial and criminal penalties:

Financial

Tier 1: Minimum fine of $100 per violation up to $50,000

Tier 2: Minimum fine of $1,000 per violation up to $50,000

Tier 3: Minimum fine of $10,000 per violation up to $50,000

Tier 4: Minimum fine of $50,000 per violation

Criminal

Tier 1: Reasonable cause or no knowledge of violation—Up to 1 year in prison

Tier 2: Obtaining PHI under false pretenses—Up to 5 years in prison

Tier 3: Obtaining PHI for personal gain or with malicious intent—Up to 10 years in prison

Conclusion

While obtaining compliance might be a necessity no matter what industry you may be in, ensure that it is not the end goal. If compliance is set as an end goal, an organization can lose sight of its overall security and the bigger picture. Working with well-defined frameworks and best practices while keeping a compliance standard in mind remains the most effective way to secure an infrastructure. 

Chapter 9. Physical Security

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 9th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at sevans@oreilly.com.

Physical security is often dealt with by the facilities department, especially in larger organizations; thus it is often beyond the remit of the information security team. The security team is responsible for identifying and analyzing possible threats and vulnerabilities and recommending appropriate countermeasures to increase the overall security of a department or the organization as a whole. Physical security is often a feature of regulatory compliance regimes and vendor assessment questionnaires, as well as materially impacting the security of the systems and data that you are tasked with protecting. For this reason, at least a high-level understanding of physical security approaches should be attempted. The physical security aspect should be included in any internal assessments, as well as being in scope for penetration tests.

Social engineering remains to this day a very effective way of accessing the inside of a network. It is within our nature to trust others at their word without verification. The goal of physical security is to prevent an attacker from attempting to mitigate these controls. As is the case with other aspects of information security, physical security should be applied as defense in depth. It is broken into two sections: physical and operational. Physical covers the controls like door locks and cameras, while operational covers employee access, visitor access, and training, just as some examples.

In this chapter you will learn how to manage both the physical and operational aspects of physical security within your environment.

Physical

First and foremost, physical security is composed of the physical properties of your environment.

Restrict Access

The most obvious aspect of physical security is restricting access to the premises or portions of the premises. Physical access controls such as door locks and badge systems prevent unauthorized personnel from gaining access to secure areas where they might be able to steal, interfere with, disable, or otherwise harm systems and data. It is recommended that highly sensitive areas be protected with more than one security control, essentially becoming two-factor authentication for physical assets. Controls that are common to choose from are PIN pads, locks, RFID badge readers, biometrics, and security guards.

In addition to physical controls in the building, some physical security precautions can also be taken at every user’s desk:

Ensure that screens are locked whenever users are not at their desk

Use computer cable locks where appropriate

Enforce a clear desk policy, utilizing locking document storage

Access to network jacks, telephony jacks, and other potentially sensitive connectors should be restricted where possible. In public or semi-public areas such as lobbies, jacks should not exposed so that the general public, or visitors, can easily access them. As mentioned in more depth in Chapter 14, where possible, jacks should not be left enabled unless equipment has been authorized for use via that specific jack. These precautions reduce the chance of a physical intruder being able to find a live jack without unplugging something and risking raising an alarm.

Printers and printer discard boxes and piles can be a treasure trove of interesting information. Sensitive documents should stored in a locked receptacle prior to shredding, or be shredded right away.

Video Surveillance

Video surveillance, or closed circuit television cameras, can be useful not only for physical security teams to notice and record incidents of tampering or theft of equipment, but additionally when correlated with other evidence such as user logons and badge swipes. This video evidence can sometimes be used to confirm attribution. For example, the use of a particular user account does not necessarily incriminate the account owner, as the credentials could have been stolen. Video footage of the owner standing at the console is much harder to dispute.

Cameras would typically be located at major ingress and egress points, such as a lobby area as well as, or particularly sensitive areas such as server rooms. Cameras located in positions such that they are able to capture the faces of people when they swipe a badge allow correlation of logs with badging systems to determine if stolen or borrowed badges are being used to hide the identity of a criminal. In order to ensure that cameras are not tampered with, they should be placed out of easy reach and preferably, within a tamperproof physical enclosure.

Surveillance or CCTV (closed-circuit television) cameras should be placed pointing at entrance doors to the building, areas of high importance or sensitivity (e.g., a prescription medicine cabinet or server room door), and wherever else a high risk has been identified. Figure 9-1 shows a good example of how not to place surveillance cameras.

Figure 9-1. Inefficient surveillance equipment placement

Authentication Maintenance

Can you see what is wrong with Figure 9-2?

Figure 9-2. I’m just going to assume this key code is 3456 and leave it at that (Thanks to @revrance for the image. RIP.)

This reinforces the need to have audits and to not forget that even if something is functional, it may not be secure. Maintenance also includes changes in staff. In the event that a member of staff ceases to be a member of staff, they too should surrender their badge, along with any keys. Any doors or other assets that are fitted with a physical PIN pad should be changed in accordance with that staff member’s previous access.

Secure Media

Controls for physically securing media such as USB flash drives, removable hard-drives, and CDs are intended to prevent unauthorized persons from gaining access to sensitive data on any type of media. Sensitive information is susceptible to unauthorized viewing, copying, or scanning if it is unprotected while it is on removable or portable media, printed out, or left on a desk.

If stored in a nonsecured facility, backups that contain this data may easily be lost, stolen, or copied for malicious intent. Periodically reviewing the storage facility enables the organization to address identified security issues in a timely manner, minimizing the potential risk.

Procedures and processes help protect data on media distributed to internal and/or external users. Without such procedures, data can be lost or stolen, or used for fraudulent purposes.

It is important that media be identified such that its classification status can be easily discernible. Media not identified as confidential may not be adequately protected or may be lost or stolen.

Media may be lost or stolen if sent via a nontrackable method, such as regular mail. Use of secure couriers to deliver any media that contains sensitive data allows organizations to use their tracking systems to maintain inventory and location of shipments. Larger organizations may make use of internal courier services that would need their own security briefing related to their specific role in the company.

Without a firm process for ensuring that all media movements are approved before the media is removed from secure areas, it would not be tracked or appropriately protected, and its location would be unknown, leading to potential loss or theft.

Datacenters

It is important to design physical security into the interior of a datacenter. A variety of situations to plan for range from having co-located equipment to contractors needing physical access to the room, but not the equipment. Use rackable equipment so that locking server racks can be utilized. While they can technically still be moved unless bolted to the floor, the lock on the cabinet itself provides an additional layer of protection. Keys for the racks should remain in a central location to be checked out from and not left in the racks themselves or in the datacenter.

Remote offices are sometimes more difficult to secure as important assets may not have a dedicated datacenter, but instead share space with another department, or be tucked away in a closet somewhere. Normally this equipment is not extremely important, but it is still a potential vector of attack. Equipment like office routers, switches, and maybe a read-only domain controller are all common assets to protect. Many times it is just not feasible to have an entire rack for such a small amount of equipment, but having it in a locked equipment enclosure is a great step to take.

Operational

In addition to the physical properties, there are various operational aspects to physical security.

Identify Visitors and Contractors

Being able to differentiate visitors, staff, and contractors is important so that people can quickly determine an approximate level of trust that they can place on a person with whom they are not already familiar. This ability to quickly differentiate staff from visitors, for example, plays a key role in ensuring that sensitive data is not exposed.

Visitor Actions

All visitors should be signed in and out of the premises, and be escorted to and from the reception area, leaving a permanent record of when they were in the building and who they were visiting, in case this information is required at a later date. Not only should a sign-in/sign-out procedure be required, but any action involving technology, equipment, or potential information gathering should require an employee verification of intent.

Contractor Actions

As contractors by nature will have more access than a normal visitor would, they should be properly identified as well. Proper policy and guidelines should be set on who the contractor works through for identification and access. A proper photo ID should match verification from both the contractor’s department and the contracting company. As with permanent staff, appropriate vetting should take place. In the case of contractors, this typically means their agency attesting to background checks on all contractors on your behalf.

Badges

Visitors should be issued a badge that is easily distinguishable from a staff badge, typically displaying the word “visitor” and being a different color than a staff or contractor badge. Visitor badges should be restricted to only the duration of the visitor’s stay and surrendered when they sign out. There are also badges that will automatically void after a certain time limit, as seen in Figure 9-3.

Note

Badges are fairly simple to spoof with time and effort. Recon can be completed by someone malicious to attempt to re-create a legitimate badge.

Figure 9-3. Over time some badges will void themselves out

Include Physical Security Training

Employees should not only be trained on the digital aspects of social engineering, but on the physical side as well because these methods can be even trickier to detect. Criminals will often pose as internal personnel, contractors, or as security engineers themselves in order to gain access to POS devices, server rooms, or any other endpoint.

Following are some scenarios and types of potential malicious activities to include in training:

Tailgating

Employees should be taught that while holding doors open for others is a normal polite response, they should not do this for badge, key, or other restricted access doors. This is an extremely effective way for an unauthorized person to gain access. Often this is one of the more difficult behaviors to address as many companies have positive and people-friendly cultures. Adding signs to reinforce this idea (as seen in Figure 9-4) can be a helpful reminder.

Figure 9-4. Tailgating reminder sign

Badge cloning

RFID keys can easily be cloned with a tool that costs less than $20. As we’ve recommended elsewhere, highly sensitive areas should be protected with more than one method of authentication. Employees should not allow others to borrow, hold, or “test” their badges at anytime.

Malicious media

While it is recommended that physical communication methods such as USB ports be restricted or disabled and controlled by an endpoint solution, it may not always be possible. In cases where USB ports are enabled and accessible by others, employees should be taught the dangers of this access. Not only can attackers stealthily insert a malicious USB, they may also drop them in public areas with labels such as “Payroll projections 2016” or “Executive Salary Q1,” or ask for a document to be printed out. USB drives can be programmed with software to collect information or create reverse shells back to a waiting malicious device, amongst other attacks.

Restricted access

The time and effort to clone a badge isn’t needed. Someone can just ask an employee using a predetermined persona and dialogue called a pretext to gain access to restricted areas.

Pretexts

Many criminals will try to fool personnel by dressing for the part (for example, carrying tool boxes and dressed in workwear), and could also be knowledgeable about locations of devices. It is important that personnel are trained to follow procedures at all times. Another trick criminals like to use is to send a “new” system with instructions for swapping it with a legitimate system and “returning” the legitimate system to a specified address. The criminals may even provide return postage as they are very keen to get their hands on these devices. Personnel should always verify with a manager or supplier that the device is legitimate, expected, and came from a trusted source before installing it or using it for business.

Conclusion

With the abundance of digital threats that we face day to day, sometimes the old-school physical methods of protection are pushed by the wayside. It is important to keep in mind that the information security team should be actively working with whichever department is in control of physical security to provide feedback on current threats and gaps.

About the Authors

Lee Brotherston is a senior security advisor, providing information security consulting services to a range of clients. Having spent nearly two decades in information security, Lee has worked as an internal security resource across many verticals, including finance, telecommunications, hospitality, entertainment, and government, in roles ranging from engineer to IT security manager.

Amanda Berlin is an Information Security Architect, co-hosts the “Brakeing Down Security” podcast, and writes for several blogs.