When it comes to DevOps, Continuous Integration, Continuous Testing, ARA and Continuous Delivery, people who work with IT and software are usually focused on the ability to get things done.
We are focused on crossing tasks off our list: get it out the door as quickly as possible. For that, we usually look to have reliable automation to be able to give developers the kind of feedback that enables powerful, robust and as-bug-free-as-possible code and services. We look for speed, controls, graphical workflows, scalability, API's, REST, JSON, speed (again), agent-based architectures, and so forth. (Did I say speed? I like speed.)
We focus on the technology, and how to execute on it.
The reality, though, is that IT technology and procedures are, and must be, closely tied to the business goals and business operations. IT initiatives are always the result of a corporate initiative that is tied to a strategic goal of the organization. In any enterprise, the Business is IT's primary customer, and any service provider must provide value to its customer by aligning to their needs.
In my opinion, the best description of this concept lies in the definition of the "goals cascade" that is the core of the COBIT® 5 framework (Controlled OBJectives for Information Technology).
" Every enterprise operates in a different context; this context is determined by external factors (the market, the industry, geopolitics, etc.) and internal factors (the culture, organization, risk appetite, etc.), and requires a customized governance and management system. Stakeholder needs have to be transformed into an enterprise’s actionable strategy."
"The COBIT® 5 goals cascade is the mechanism to translate stakeholder needs into specific, actionable and customized enterprise goals, IT-related goals and enabler goals. This translation allows setting specific goals at every level and in every area of the enterprise in support of the overall goals and stakeholder requirements, and thus effectively supports alignment between enterprise needs and IT solutions and services."
The goals cascade states that external and internal factors drive stakeholders in the enterprise to seek change. The first step is to translate those drivers into needs , which can be generally organized into the realization of benefits, optimization of risk and/or optimization of resources.
In the second step, through enterprise governance, those needs are then translated into enterprise goals. The goals are organized around the four major quadrants of a classic Balanced Scorecard :
Financial
Customer
Internal
Learning and growth
When those goals require actions on the part of IT, they are then further refined into IT goals, aligned to the same four balance scorecard quadrants, that are required in order to achieve the enterprise goals.
IT management must then translate those IT goals into actionable steps that use enablers for successful completion. Those enablers include:
Processes
Organizational structures
Culture, Ethics and Behavior
Information
Services, Infrastructure and Behavior
People, Skills and Competencies
I could go on for hours just on this subject, but let me step back to the larger point to be made -- nothing should happen in IT unless IT is responding to a need that stretches all the way back to the enterprise as a whole .
To emphasize this point, I'd like to talk about a stakeholder that is driven by external factors, which these days has been knocking heads right and left all over the planet: audits . (Cue the screams, hair on fire, thunder and lightning.)
The dreaded audit compliance
There isn't a soul in any enterprise who doesn't live in fear of failing an audit for non-compliance. As a former auditor, I can safely assert that every time I walked in the door of a company to begin an audit, I knew the Orc-In-Chief in "Lord of the Rings" was spot-on when he said that “on a good day, when the wind is right, you can smell fear.”
It is also common knowledge that when it comes to audit compliance, IT is not exempt from its share of requirements (and pain). SOX (USA), J-SOX (Japan), CLERP 9 (Australia), C-SOX (Canada), Clause 49 (India), Euro-SOX (EU), the Combined Code of Corporate Governance (UK), Law 262/2005 (Italy) and Loi sur la Sécurité Financiére (France)… these are all legislative frameworks that mandate compliance requirements. All of them, in one way or another, mandate compliance from an IT standpoint as well, because of a very simple reality -- no modern enterprise can operate without IT, and IT has to be governed and managed in order for corporate governance to succeed.
What do you need to demonstrate in order to pass an audit?
In order to pass an audit and achieve full compliance you need to exhibit two things:
Evidence of intent - can be something as simple as a process diagram that details actions to be taken when, for example, a customer creates a service ticket to report an outage.
Records of action - the audit trail of records detailing what actions were taken in a specific case (in the example above, in response to the creation of the ticket- from initial handling to resolution, closure and review as part of a continuous improvement culture.)
In all but the smallest organization, it is literally impossible to examine every process and every audit trail for activities undertaken since the last audit. This is why the auditor usually requests a representative sampling of evidence for both the above categories. An experienced auditor will also know what kinds of samples to ask for to ensure the enterprise doesn't provide something that is not representative of reality.
Your DevOps and IT processes, and the inevitable Audit:
What does this all mean in the context of DevOps and the tools an IT organization picks to implement automation and other components of Continuous Integration, Continuous Testing, Deployments or Continuous Delivery?
It means that IT should not even begin an RFP process for such tools without creating, among many other things, a list of detailed requirements to support the need to demonstrate evidence of intent and records of action to an auditor who can - and will - request anything, at random, or on purpose.
Here at CloudBees we are constantly telling our customers that our tools are enterprise-grade. And sure, that includes massive scalability, high availability, robust features and high performance. But – very importantly – it also includes auditability. Painstaking auditability. For the needs of today’s enterprises – no matter how regulated or complex the industry (thing finance, automotive, healthcare..).
We constantly ask ourselves whether or not our claims for would stand up to scrutiny, because if we didn't ask, our customers would. And they do. So, where do we stand? Let’s take a look.
What does enabling end-to-end auditability look like in CloudBees’s solutions?
Let’s review how CloudBees Flow Deploy, Electric Flow Build/Test and our CloudBees Flow automation platform enable auditability. To do that, let’s examine the products’ capabilities against a real-world scenario that’s only too common in today’s enterprises.
Company X is gearing up to deploy app Y – and explaining the hell out of it:
We will address a very real example of a software release planned by a company which, due to a number of issues that have occurred within the last year, has become the subject of additional scrutiny by the IT auditors.
In this example, the reason behind the audit was that a number of non-conformances were found last year associated with problems deploying web applications to both the company web site as well as the company's intranet. These situations led to fines and public relations issues that could have been avoided had IT instituted better controls and automation to reduce manual, error-prone processes and homegrown scripts that were no longer maintainable due to normal staff turnover. When audits were conducted to get to the bottom of the problem and find ways to avoid any further instances of these failed deployments, IT was unable to provide clear process documentation or audit trails that could enable effective root-cause analysis and process improvement. Senior management has determined that this situation does not meet the enterprises' high standards and has directed IT management to take corrective action and report back.
Part of the root cause of the non-conformances was determined to be the lack of an enterprise tool that could be used to codify and automate IT's processes and support their growing DevOps initiative. There were pockets of improvement in some groups using open source tools such as Jenkins, but there was no one "single pane" view in IT for the entire end-to-end delivery process, and therefore no way to centralize governance and control. In addition, while easy to implement and low-cost at first, the tools chosen did not support enterprise scalability. In the end, IT was only digging itself into a deeper hole. The CIO came to the conclusion that this was too important an issue to allow individual teams to take disconnected actions, and a strategy was crafted that would require centralized enterprise-grade tools that support the needs of the larger corporate initiative.
As part of the selection process, the CIO has chosen to evaluate CloudBees Flow, and IT has gone through the process of implementing the tool in a test environment with the help of one of our Solution Engineers. The goal of the initial project was to have Flow handle software updates to one application: the company's web storefront. A POC (Proof of Concept) has been performed to setup the tool and associated processes to handle deployments to the storefront, and an internal audit has been scheduled to get an opinion as to whether or not CloudBees Flow has the functionality required to provide the information needed to pass an audit. The internal auditor will be looking at both evidence of intent and records of action, with particular emphasis on security controls and the ability to trace an incident back to its root cause using the tool's audit trail capabilities.
In order to produce the information, we have to provide documentation of the processes and controls that are in place for the deployments (because some of the issues identified were illegal changes to infrastructure that were not subject to change controls) and the audit trails associated with deployments (because when the auditors tried to understand what happened, the company was previously unable to produce the required records). Let the games begin..
What does Compliance look like?
1Rounding up your Evidence of Intent:
Your deployment plan:
From an "evidence of intent" point of view, CloudBees Flow can produce clear and unequivocal information. The structure of the storefront application in its full multi-tier glory is documented in a graphical manner than anyone can understand.
CloudBees Flow Application Modeling
The processes associated with each component of the application in each tier are also clearly documented…
CloudBees Flow Process Modeling
…as well as the overall process to deploy the entire application to the selected environment.
CloudBees Flow Deployment Process Modeling
Speaking of environments, this IT group has three of them, one for development, one for QA testing, and one for Production.
CloudBees Flow Environments
The auditor will want to see how the company has defined them and if the configuration matches the set up of the environments in the real world.
CloudBees Flow Environments Modeling and Configuration
The auditor may also want to verify that the environment definitions point to the right resources, either physical or virtual. We have that covered as well.
CloudBees Flow Environment Configuration
Security, permissions and access controls:
When it comes to evidence of control, security is a common source of non-conformances, and CloudBees Flow can easily demonstrate controls. Interfaces to ActiveDirectory and LDAP can provide authentication, or you can use the built-in authentication mechanisms. Users and roles are represented in the underlying automation platform interface, and can be grouped according to function, permissions, etc.
CloudBees Flow Automation Platform - Users and Roles
CloudBees Flow Automation Platform - Users and Roles
When it comes to permissions, CloudBees Flow's ACL (Access Control List) mechanism is designed to reward trust, but can also allow permissions to be set all the way down to individual job steps. This screen shows security control definitions for the procedure that creates IT tickets in the ServiceNow integration that was implemented as part of the POC.
CloudBees Flow Automation Platform - Access Controls
2 Producing Records of Action
This is all well and good, but the one area where most auditors find non-conformances is not intent, but in the records of action -- the audit trails . It's one thing to say you're doing something, it's quite another to be able to prove it, and CloudBees Flow is up to the job here as well.
For example, here we see the initiation of the deployment of the latest version of the storefront application to the production environment.
CloudBees Flow - Initiate Deployment
The aftermath of the operation shows that a number of tasks were performed, but this view does not answer the auditor's questions as to the details of what happened.
CloudBees Flow - Deployment Steps
By clicking on the job ID of this deployment, we can drill-down into the automation platform interface and view the complete details of the log of the deployment, including who initiated the deployment, when it was initiated, any diagnostics , retrieved artifacts , etc. You can also add links to this page to custom reports as necessary to fulfill any additional, specialized audit trail requirements.
Drill down to deployment log details
Need to show the auditor details of specific steps? Click on the icon in the Log column for even more detail to the audit trail.
Details of deployment step
In addition, external integration can add additional information to the audit trail. In this case, the deployment was initiated through an approved change request that was stored in the enterprise's ServiceNow SMS (Service Management System), a key component of IT's ongoing ISO 20000 certification initiative. Email notifications are generated to notify that the request needs to be fulfilled…
Initiate deployment from external tool/integration
…with a link to the physical ticket, which becomes part of the audit trail, and is updated when the deploy is completed.
Ticket details in ServiceNow
It’s important to note that CloudBees Flow is tools-agnostic. We know your process is comprised of many point-tools, and we provide you with central management and auditing for all of them. Flow integrates with any external tool and orchestrates your entire end-to-end toolchain - whatever that may be.
Shared visibility:
Because CloudBees Flow's interface is hosted on an Apache web server, anyone in the enterprise with the proper credentials can access and operate it, as well as request and view reports, logs and any other required information, from desktop or mobile devices.
Enterprise-grade, Enterprise-support:****
Rather than being a disparate collection of tools that some vendor has collected (likely through acquisitions) and hastily stitched together like the good Dr. Frankenstein's creation, with the inevitable integration nightmares and finger-pointing that follows, CloudBees Flow was designed and created, and is maintained and supported by one company and one engineering team -- ours.
This, folks, is what we mean when we say that CloudBees's tools are designed from the ground up to be enterprise-grade. The end result is not just scalability, flexibility and speed, but also the ability to fulfill a key requirement of any serious enterprise: demonstrate that there are intentions to meet the requirements of a DevOps/CI/CT/CD strategy, and prove that the enterprise can talk the talk and walk the walk when it comes to aligning IT with the enterprise and consistently delivering value.
Kaplan, Robert S.; David P. Norton; "The Balanced Scorecard: Translating Strategy Into Action"; Harvard University Press, USA, 1996
"COBIT®5: A Business Framework for the Governance and Management of IT"; pp. 17, © 2012 ISACA.
ISO/IEC JTC 1/SC 40 - IT Service Management and IT Governance, http://www.iso.org/iso/home/store/catalogue_tc/catalogue_tc_browse.htm?commid=5013818