Architecture Case Study – Part 1

Architecture Case Study – Part 1

Architecture Case Study – Part 2

In the Security Architecture Series of blogs we have shared all of the steps involved in requirements gathering, baseline, product and solution selection and, through to realizing the architecture. This blog presents an Architecture Case Study that uses those principles and recommendations as a practical example. The illustration provides a conceptual simplified view of the program use case.

Defense in Depth Part 1 (this blog) takes the reader from Architecture development through to the Technical Recommendation then Part 2 takes the reader from Design to Deployment strategy with Implementation and Migration.

Program Overview

The overall goal of the project was to seek to standardize across the organization and all of the 20+ business units. The business units are primarily autonomous with different types of technology and infrastructure and at varying degrees of maturity and security. The status quo presented a series of risks to both the organization as well as each of the business units.

The cost/benefits were multiple not just in terms of standardization but also the total cost of ownership (TCO) and Return on Security Investment (ROSI) to purchase technology at volume discount while gaining increased visibility and support from the vendor(s). However, the primary goal and benefit was to establish and foster a spirit of collaboration, sharing and, cross-pollination to work together towards a common vision.

Architecture Case Study

The overall high-level approach is defined, as follows:

  • Develop Architecture
  • Requirements
    • Specify Functional Requirements (RFI)
    • Request Information from the vendor community (Distribute RFI)
    • Review RFI responses
    • Select vendors for product/solution evaluation
  • Vendor/Product Selection
    • Conduct bake-off testing with business unit participation
    • Review evaluation scorecard results
    • Conduct pilot of highest ranked solution
    • Review pilot results
    • Technical Recommendation
  • Develop Design
  • Develop Implementation Program
    • 4 Phases
    • Alpha Test
    • Beta Test
  • Deploy/Execute
    • 9 Step Program

Baseline Network Standard Architecture

I worked on the project as a consultant in the role of Program Technical Architect as part of the overall Architecture Governance and Steering Committee. My role was both to guide the direction and act as a technical lead as well as perform a lot of the detailed work to develop the actual deliverables based on collaboration and interaction.

Network Security Working Group (WG)

The first step was to develop an Network Security Working Group (WG) that included stakeholders from the various business units to contribute at two levels, as follows:

  • Level 1 – A small representative sample of core members who were involved in the brainstorming sessions to represent their business unit and contribute input on both the architecture as well as unique requirements
  • Level 2 – A stakeholder from every business unit who was involved in monthly or quarterly (as appropriate) review and approval of the emerging work product and progress to enable consensus and buy-in.


Collaboration was key to the success of the project. We wanted to involve stakeholders in every stage of the process and to ensure that their contribution was captured and recorded. Brainstorming sessions were used extensively at various locations with preparation to seed and stimulate the discussion with a facilitator as well as scribes to record and document.

Architecture Draft Review

A series of review cycles were used with a broader and broader audience to ensure that the architecture aligned with both current and future strategy and needs for the business units. The architecture document contains the following sections:

  • Architectural Principles
  • Network Models
  • Physical Layer Design
  • Supported Protocols
  • Network Performance Architecture
  • Network Security Architecture
    • Areas
    • Perimeters
    • Zones
    • Controls
    • Management
  • Network Management Architecture
  • Enabling Services
  • Appendix
    • Profiling BU Network Traffic
    • Modeling Steps
    • Example of Modeling a BU Network

The finalization and ratification of the Network Baseline Standard Architecture was a major accomplishment for the organization because not only did it lay the groundwork for the success of this specific program it also laid the framework for future projects across initiatives such as Wireless and Evolving Security.

Requirements Specification (RFI)

The RFI – Network Security Functional Requirements document was developed next by the Network Security Working Group. The team worked closely together to identify the functional requirements and assign a relative priority of High, Medium, or Low.

RFI Evaluation Criteria

Once RFI was completed and reviewed the Network Security Working Group convened a meeting to establish the RFI response evaluation criteria and scorecard to be used for the analysis of response from bidding vendors. The functional requirements that were originally identified as High were further examined and 19 requirements were selected and rated as MUST by the group. A functional requirement with a MUST designation implied that the associated security device would be eliminated from further consideration if it did not comply.

RFI Requirements Sample

All functional requirements were then scored with either a maximum possible score of 10, a maximum possible score of 5 or, a maximum possible score of 3 respectively.

Evaluators and Decision-makers

To ensure that the RFI responses were analyzed in an independent and objective manner the Network Security Working Group assigned an Evaluation Team, which comprised primarily of consultants. The Evaluation Team was solely responsible for conducting the RFI response analysis to select vendors and solutions and also performed the network security equipment testing. However the team did not participate in any decision-making and only acted as advisors. The Decision-maker Team is comprised of members of the extended Network Security Working Group (Level 2).

RFI Evaluation Scorecard

The Evaluation Team developed an Evaluation Scorecard that took all response format files from bidding vendors and consolidated them into the Consolidated Vendor Response Form file. This consolidated file contained macros to process the entries from all of the bidding vendors and to create two worksheets, as follows:

  • Product Stack Ranking – summary of scores based upon device category
  • Vendor Stack Ranking – summary of scores by vendor

Vendor Evaluation

RFI Evaluation Methodology

The Evaluation Team adopted an objective method of evaluation focused on the functional requirements as defined by the Network Security Working Group development team, and communicated to the vendors in the Security RFI. It led to the following step procedure.

  1. Evaluate ‘Best of Breed’ responses to derive the top 3 vendors in each of the following four categories – Firewall, VPN, IPS, and Management.
  2. If possible, select the 4 most populous vendors from these rankings for inclusion in the network security equipment testing.
  3. Evaluate ‘Integrated Portfolio’ responses, if any, from all remaining vendors to derive the top 3 portfolio vendors.
  4. Select the best vendor from this ranking for inclusion in the network security equipment testing.

For quality control a Conformance Check was also conducted to ensure that all ‘Yes’ or ‘Partial’ responses had an associated supporting Response Reference Section and/or comment to backup the statement by the vendor.

Vendor / Product Solution Selection

Of the fifteen network security equipment manufacturers that responded to the RFI, five vendors who best met the functional and operational requirements were invited to participate in the bake-off.  Each vendor brought and installed equipment in the lab to allow members of the working group to conduct technical evaluations.

Bake-Off Testing Methodology

The primary goal of the bake-off testing was to further measure the fit of the proposed solutions, with a focus on holistic integration against the functional requirements that were documented in the Network Security RFI. It is interesting to note that most vendors acknowledged that this was the first time ever they had deployed and integrated their solutions holistically and operated them in a real-world scenario. Most vendors had only participated in one-off point solution evaluations.

The bake-off testing objectives sought to identify, test and select one or more manufacturers of network security solutions (Firewalls, IDS/IPS, VPN and, Management/Monitoring) to proactively meet the following goals:

  • Secure core business unit network infrastructure devices
  • Add network security components to protect and segregate critical assets
  • Integrate security components to current network management systems (NMS)
  • Develop a Network Security Management System (NSMS).

The evaluation team designed and deployed an inherently insecure Network Security Evaluation Lab that simulates a typical business unit network and provides both the network areas and security zones that need to be protected with sample assets. The testing viewed potential threats and attacks from External (outside the border perimeter) as well as from Internal/Insider (business unit networks) towards Data Center Zone and Management Zone(s) targets.

Test group scenarios were developed that made use of various typical threat and attack categories (e.g., signature based, anomaly based, DoS). The controlled attacks were initiated by a penetration tester from both external and internal sources. In addition, a traffic generation load/stress testing tool was utilized to exercise functionality and simulate normal traffic (client connections and sessions).

Network Security Evaluation

The above diagram provides a simplified illustration of the test groups and targets across the Functional Requirement categories, as follows:

  • Detection
  • Response
  • Alert / Logging
  • Correlation
  • Reporting
  • Management

These are the functional requirement categories that are documented in the Network Security RFI and the reference codes refer to the specific line item requirements.

The goal of the bake-off testing was to initiate multiple discrete sets of tests as ‘triggers’ that initiate a sequence of events that then flow through the functional requirements categories and elements in the diagram. The Evaluation team asked the manufacturer to demonstrate if/how the proposed security solution detected, responded, alerted, logged, correlated (where appropriate), and reported as a consequence of these sequences, and how any generated events are managed. The testing also evaluated the utility of the solution as well as factors, such as integration, management and, monitoring. Decoys and scans were used to generate noise while stealthy attacks were employed.

The Evaluation Team was restricted to performing the testing and to providing an objective report to the Network Security WG attendees, in addition to an independent and objective report from the penetration tester and traffic generation tester. The attendees made use of an Evaluation Scorecard and each stakeholder contributed a score.

Evaluation Scorecards

The Evaluation Team developed a set of scorecards that would be used by Network Security WG and business unit stakeholder attendees to the bake-off sessions. There were two scorecards used across two days, as follows:

  • Objective Scorecard – Validate Compliance to Requirements
  • Business Unit Scorecard – Validate Fit to Business Unit

For the Objective Scorecard the Evaluation team reviewed both tests and vendor demo together with Q&A to determine if the functional requirements were met by the implemented solution as cited by the Vendor in their RFI response. The team referenced the Vendor Summary and RFI Response sheet.

For the Business Unit Scorecard the attendees individually assessed how well the solution satisfies the requirements and fits the needs of their business unit and determined the total category score, per section.

RFI Scorecard

Technical Recommendation

The technical evaluation had consisted of more than 100 evaluation criteria and over 50 repeatable tests conducted on four network security components: Firewall, VPN, IDS/IPS and Management/Monitoring.  The RFI was designed to allow the option to select either the best suite of tools from a single manufacturer or to select the ‘best of breed’ (the best components from multiple vendors).

The Evaluation Team’s technical scores were tallied and submitted as a Technical Recommendation to complement the financial total cost of ownership analysis.  The Technical Recommendation and Financial Analysis were provided as a summary of the findings and recommendations as an outcome of this evaluation process.

Technical Recommendation

Implementing network security equipment on a business unit network is a large and challenging proposition.  The group recommended that there be Alpha and Beta deployments to validate technical elements and further understand the integration complexity with the proposed solution.  In addition, the Alpha and beta process helped to develop a deployment methodology which will allow for a deliberate approach for addressing important business unit and organization-wide security concerns.

Next Steps

Architecture Case Study – Part 2 will continue this series to take the reader from the Technical Recommendation on into Baseline Network Standard Design as well as the Deployment strategy with Implementation and Migration process.

Think You’re Secure? Think Again.
Security Architecture Primer
Security Architecture Baseline
Risk-Aware Security Architecture
Develop Security Architecture
Product and Solution Selection
Security Architecture Implementation
Adaptive Security Lifecycle

Thanks for your interest!

Nige the Security Guy.


Security Series Master Index

Security Series Master Index

This blog will be maintained to provide a Security Series Master Index summary of all of the blogs to assist in site exploration and navigation – as well as preview upcoming blogs.

The currently planned series are, as follows:

  • Security Architecture Series
  • APT Strategy Series
  • Security Governance Series
  • Risk Management Series
  • Security Assessment Series
  • Miscellaneous Topics

Security Architecture Series

Security Architecture Series

The Security Architecture Series build upon each other and contains:

APT Defense Strategy Series

APT Strategy Series

The APT Strategy Series contains, as follows:

  • APT Strategy Series
    • An introduction to the APT Strategy Series together with the rationale behind the need for a well-designed Secure Architecture and Design foundation
  • Defensible Security Posture
    • The basic idea of a Defensible Security Posture is that you aren’t striving for an absolute, but rather for a position (or posture) that is able to be defended even when it’s infiltrated
  • Defensible Security Posture – Part 2
    • How can you leverage the Defensible Actions Matrix? A defensible actions matrix defines processes and procedures that can impact an attacker’s capability at various stages of the cyber kill chain.
  • Advanced Threat Defense – Part 1
    • Focuses on top-down Architecture & Strategy to address Advanced Threats with ability and visibility to Detect and Contain
  • APT Defense Puzzle
    • Focuses on bottom-up Actionable Security Posture Improvements: Checks, Practices, Controls, Indicators
  • APT Detection Framework
    • In order to begin to understand and to be able to defend against targeted attacks a detection matrix is needed for visibility, analysis and, to ensure that all threat scenarios are considered with no gaps in defense
  • APT Detection Framework – Part 2
    • There is a trend underway in the information security field to shift from a prevention mentality — in which organizations try to make the perimeter impenetrable and avoid breaches — to a focus on rapid detection, where they can quickly identify, contain and mitigate threats.
  • APT Detection Indicators – Part 1
    • In a world where organizations need to be watching or monitoring their networks continuously knowing what to look out for is critical
  • APT Detection Indicators – Part 2
    • Advanced Persistent Threats (APT) typically exhibit recognizable attributes and patterns that can be monitored by readily available, open source tools
  • Adaptive Zone Defense – Part 1
    • Limiting and intelligently managing communications between services and systems on an organizations network helps contain an infection or compromise to keep malware or a persistent threat from running rampant
  • Adaptive Zone Defense – Part 2
    • In Adaptive Zone Defense – Part 2 we develop another key foundation, known as Application Architecture Taxonomy that talks to application and system placement, organization and, management within the proposed zones.
  • APT Red Teams – Part 1
    • How do you prevent an APT? Red Teams enable continuous improvement and optimization from counter-intuitive sources to help mitigate advanced threats
  • APT Red Teams – Part 2
    • Addressing security more aggressively and working to identify areas of weakness is a more sensible, and ultimately, more effective approach than working to build a “bigger wall” that you hope attackers can’t get through
  • APT Response Strategy – Part 1
    • How do you implement a Plan C? Organizations are starting to recognize the ever increasing importance of rapid and well orchestrated incident response capabilities as a key component in their defense-in-depth strategy.
  • APT Response Strategy – Part 2
    • [Coming Soon]
  • APT Threat Analytics – Part 1
    • How can you predict emerging threats? Threat intelligence and analytics continues to dominate the headlines and attention of organizations seeking viable options in their escalating battle against advanced threat actors.
  • APT Threat Analytics – Part 2
    • With the increase in advanced, multidimensional threats, more and more organizations are considering development of an in-house threat intelligence program
  • APT Operational Maturity – Part 1 [Coming Soon]
    • How can you evolve towards effective Visible Security Operations? An update to my 90’s Security Maturity Model based on recent sophisticated threats and determined attackers.
  • APT Intelligent Operations – Part 1 [Coming Soon]
    • How can you build an Intelligence-Driven Security Operations platform using commercial and/or open-source tools.

Security Governance Series

Security Governance Series

The Security Governance Series contains, as follows:

Security Assessment Series

Security Assessment Series

The Security Assessment Series contains:

Thanks for your interest!

Nige the Security Guy.

Security Architecture Implementation

Security Architecture Implementation

This blog continues the Security Architecture Series by building a Back to Basics foundation before developing upon that foundation with adaptive security in hyper-extended enterprises for a defensible security posture.


The security architecture defines and justifies a number of solution implementation, integration and/or improvement projects each year, based on budget, resources and, priority. As such, a master project plan should be created that takes into account identified dependencies, integration points and any parallel tasks.

To plan implementation of a security solution, you must identify where project execution resources will come from, develop an implementation plan, obtain buy-in for the implementation plan, and create a detailed design for the configuration and deployment of the security infrastructure.

Implementation planning must be viewed as a process first and documents second. In addition, you should keep the technical staff or consultants used during the architectural review on tap to provide implementation planning process oversight. In many projects we have used a Program Technical Architect as the keeper of the vision, direction and, strategy.

Identify Project Resources

First, you must understand and define the scope and resources required for the  deployment. These resources include the following:

  • Security project manager
  • Support functions (administration, contracts, procurement, and others)
  • Security engineering development staff (own and vendor’s)
  • Security integration team (including part-time contributors and other support organizations within your environment)
  • Network operations staff
  • Ongoing security operations and support staff

Unless you have a particularly strong and deep security development team on staff, many organizations don’t you will want to make use of vendors’ professional services. They can, or should, be able to install, configure, and do development work for your product better than anyone else.

Security Drivers

However, you’ll also need to have a strong complement of dedicated staff involved with managing and developing the security infrastructure during its early deployment. You also need to have good relationships and well-spelled-out agreements with staff borrowed from other internal functions.

Ultimately, day-to-day operation and support of the security infrastructure will be turned over to an operations organization. However, a security project manager and a few technical support and development resources should remain in place for ongoing integration. You’ll also need staff for pre-production validation, security audit, monitoring, and incident response.

Develop Implementation Plan

The implementation plan should:

  • Restate the architecture as a high-level design, plugging in the selected solution components
  • Propose a security implementation team (which may be similar or overlapping with the original security architecture team)
  • Define an overall project plan and detailed project schedule
  • Identify required resources and other critical success factors, such as a test lab, pilot participants, firewall, or other application configuration changes that must be agreed upon far in advance
  • Define project completion and acceptance test plan
  • Write an execution-focused risk analysis and mitigation plan
  • Develop a communications and training plan
  • Compile cost estimates

The architecture, the vendor’s proposal, the high-level design, early project plans, and other working documents prepared during the project must all be consumed, synthesized, and polished to create a realistic implementation plan for going forward (see Security Plan Template). The devil is in the details and time invested upfront in planning and preparation is proven to save lots of time and enable success in implementation and cutover.

However, the high-level design and implementation plan needs to be more than just a piece of paper. Selling the security project is an on-going process.

Security Program

Implementation planning may well be combined with obtaining the funding and approval to purchase off-the-shelf solution components. It should also be at this point that explicit commitments are obtained to rely on part-time, cross-functional security integration team members. Once obtained, these commitments must be continually reconfirmed; projects fail for lack of stakeholder commitments to do their part for interconnection or configuration.

Obtaining Buy-In and Support

An implementation plans success also depends on the vendor(s). Vendors will provide resources for some stages of the plan, and vendors will be dependent on some of the plans customer-provided deliverables, such as a test lab. The full plan requires a successful triumvirate of dedicated security staff, vendor staff, and the staff of the cross-functional resources that will be integrated or consolidated in this initial implementation.

Security Plan

Think of an implementation plan as a process as much as a document. Your security implementation and deployment will depend on the efforts of many people, and they must be engaged in and buy into both the planning process and the activities it entails. Anticipate that the vendor(s) will have to make changes to accommodate your needs. Expect some push back from vendors and cross-functional team members.

Develop Detailed Design and Test Plans

At this stage, a security project moves from architecture and high-level design to detailed design. This process must be conducted by the vendor or at least in close collaboration with the vendor. However, you will want to keep your technical staff or consultants on hand to ensure that the design stays true to the architecture.

Implementation Documents

Detailed design refines each of the security projects physical, logical, integration, access, management, and administration fine points. It identifies deployment details down to the configuration of roles, entitlements, or attribute syntax.

In parallel with developing the detailed design, an enterprise needs to create detailed test plans. Testing should not be done as an afterthought. It should be baked into the design, roll out, and acceptance process.

Operations Cutover

When a key part of the security architecture has been realized, the solutions selected, and the infrastructure implemented and tested, it is time to wrap process around that architecture and transition to the team that will operate and support it. If not, then the architecture and engineering teams will be burdened by the new infrastructure, limiting their ability to move forward strategically. The architecture and security engineering teams will shift their focus to further communicating, developing, and validating the architecture and realizing the next project phase, while at the same time providing tiered support to the operations personnel.

Security Process

This section enables you to document the all-important processes and procedures for cutover, identify operations roles and responsibilities and don’t forget to —conduct user awareness and training to fully complete the deployment.

Document, Document, Document

Security support functions fill a vital role within the security of any system. Security standards, procedures, and checklists are recommended as they guide the day-to-day security activities. These processes and procedures should be developed by a knowledgeable person, and be reviewed, documented, formally published, and given to management and staff personnel. The development of procedures helps support a consistent application of the security policy.

Identify Responsibilities and Train Operations

The continuity of service and the high availability of IT services are imperative to the success of operations. Those imperatives will play an increasingly important role in the future. Once a security solution is implemented, there must be someone who is ultimately in charge and who has the responsibility to ensure implementation consistency. The most effective way of ensuring this is through management visibility and support. As such, an effective security management team should be put into place with assurance that they are properly trained.

It may be possible to assign a single individual to manage the overall security of the IT environment and then give personnel with existing responsibilities, additional responsibilities to support security. IT administrators are often very good choices to support, configure, and administer the security of a system/network, owing to their intimate technical knowledge of the operations and services provided. There are also procedures that ensure that the personnel managing the security of the system/network do not overstep their boundaries (i.e., auditing of events and actions taken).

The following high-level items should be considered during this phase:

  • Security Operations and Administration. Operation of a system involves many security activities discussed in this blog. Performing backups, holding training classes, managing cryptographic keys, keeping up with user administration and access privileges, and updating security software are some examples.
  • Operational Assurance. Operational assurance examines whether a system is operated according to its current security requirements. This includes both the actions of people who operate or use the system and the functioning of technical controls.
  • Audits and Monitoring. To maintain operational assurance, organizations use two basic methods: system audits and monitoring. These terms are used loosely within the computer security community and often overlap. A system audit is a one-time or periodic event to evaluate security. Monitoring refers to an ongoing activity that examines either the system or the users. In general, the more “real-time” an activity is, the more it falls into the category of monitoring.

User Awareness and Training

User training and awareness is critically important to the continued success of any security program. Company insiders either accidentally or intentionally cause approximately 85% of security breaches. Security training instills awareness and enables users to understand the importance of security. In most organizations, security is regarded as required, but not viewed as vital. Most security training material distributed to users is either stuck in a drawer or immediately given away to someone else.


The training program should cover the following “general” types of information:

  • Explanation of the security program, its importance, and value
  • Who is in charge of the program and specific lines of authority
  • Security services, functions, and mechanisms
  • Responsibility of the user and management community
  • Specific information and information services defined to be sensitive and critical
  • The “insider” threat and how to report security incidents
  • User and operator errors and how to prevent against these common security problems
  • Penalties for security breaches

The most effective way for security training to take root within an organization is for upper-level management to make security an important issue. Once this commitment is made at the higher levels, it will be passed on through the management chain. Migrate from reactive policy, controls and, rules to engage “people” with the skills, knowledge and positive attitude. Empower resources as part of an extended security team to recognize and report potential breaches, issues and/or incidents – provide them the responsibility and accountability to play a role.

Coming Soon!

In future blogs we will cover the Adaptive Security Lifecycle, Identity and Role-Based Access, Application Architecture Taxonomy together with Data Center Trust Zones as key components for the hyper-extended IT enterprise (insourced, co-sourced, managed).

Thanks for your interest!

Nige the Security Guy.