A COBIT 5 PAM Update Compliant With ISO/IEC 330xx Family

Determining the level of process maturity for a given set of IT-related processes allows organizations to determine which processes are essentially under control and which represent potential pain points.¹ Process maturity has been a core component of COBIT for more than a decade,² however, in the latest version of COBIT³ there was a change from the maturity model used in COBIT 4.1 to a process capability model.⁴

Currently, the COBIT 5 Process Assessment Model (PAM),⁵ which presents six process capability levels defined in an ordinal scale from Incomplete to Optimizing, is based on the International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) standard ISO/IEC 15504,^{6, 7} which is a global reference for conducting process capability assessments. Process assessments can have various purposes (e.g., capability determination, process improvement, benchmarking, supplier selection), but all process assessment projects share a common feature: being founded on process models.

Meanwhile, a new standard, the ISO/IEC 330xx family,⁸ has replaced and extended the ISO/IEC 15504 family. The ISO/IEC 330xx family enlarges the ISO/IEC 15504 application field and scope and focuses on the key concepts of process reference model, process assessment model and process measurement framework.

Because of this replacement, this article proposes an update of ISACA’s COBIT Process Assessment Model (PAM): Using COBIT 5. This article presents the main changes that should be applied to ISACA’s current publication to make it compliant with the ISO/IEC 330xx family.

The Updated COBIT 5 PAM

The following sections outlines the proposed changes to the COBIT 5 PAM.

Introduction
To make the current version of COBIT 5 PAM compliant with the ISO/IEC 330xx family, the normative references should be updated to:

• ISO/IEC 33001:2015, Information technology—Process assessment—Concepts and terminology
• ISO/IEC 33003:2015, Information technology—Process assessment—Requirements for process measurement frameworks
• ISO/IEC 33004:2015, Information technology—Process assessment—Requirements for process reference, process assessment and maturity models
• ISO/IEC 33020:2015, Information technology—Process measurement framework for assessment of process capability

For the purposes of the updated COBIT 5 PAM, the terms and definitions given in ISO/IEC 33001:2015 apply. There are two key definitions⁹ that are different from the original COBIT 5 PAM publication:

• Process quality dimension (replaces the capability dimension concept)—“(S)et of elements in a process assessment model explicitly related to the process measurement framework for the specified process quality characteristic”
• Process quality (replaces the process capability concept)—“(A)bility of a process to satisfy stated and implied stakeholder needs when used in a specified context”

Overview of the COBIT 5 Process Assessment Model
This section of the current COBIT 5 PAM publication will need the capability dimension content revised to make it conform to the ISO/IEC 33020 requirements for a process assessment model. Thus, it can be used as the basis for conducting an assessment of the capability of each COBIT 5 process. The capability levels and process attributes should be updated as follows:¹⁰

• PA 4.1 Process measurement should be replaced by PA 4.1 Quantitative analysis.
• PA 4.2 Process control should be replaced by PA 4.2 Quantitative control.
• Level 5 Optimizing process should be replaced by Level 5 Innovating process.
• PA 5.2 Process optimization should be replaced by PA 5.2 Process innovation implementation.

Among the main novelties in this updated COBIT 5 PAM are the new rating levels. In the new COBIT 5 PAM, the ordinal scale may be further refined for the measures P and L as defined in ISO/IEC 33020:¹¹

• P+ Partially achieved—“There is some evidence of an approach to, and some achievement of, the defined process attribute in the assessed process. Some aspects of achievement of the process attribute may be unpredictable.” This is defined as greater than 32.5 percent to less than or equal to 50 percent achievement.
• P- Partially achieved—“There is some evidence of an approach to, and some achievement of, the defined process attribute in the assessed process. Many aspects of achievement of the process attribute may be unpredictable.” This is defined as greater than 15 percent to less than or equal to 32.5 percent achievement.
• L+ Largely achieved—“There is evidence of a systematic approach to, and significant achievement of, the defined process attribute in the assessed process. Some weaknesses related to this process attribute may exist in the assessed process.” This is defined as greater than 67.5 percent to less than or equal to 85 percent achievement.
• L- Largely achieved—“There is evidence of a systematic approach to, and significant achievement of, the defined process attribute in the assessed process. Many weaknesses related to this process attribute may exist in the assessed process.” This is defined as greater than 50 percent to less than or equal to 67.5 percent achievement.

Furthermore, a new section on process attribute rating methods should be created in the updated COBIT 5 PAM. This section should highlight that process outcomes and process attribute outcomes may be characterized as an intermediate step to providing a process attribute rating:

When performing rating, the rating method employed shall be specified relevant to the class of assessment. The use of rating method may vary according to the class, scope and context of an assessment. The lead assessor shall decide which (if any) rating method to use. The selected rating method(s) shall be specified in the assessment input and referenced in the assessment report. Three rating methods are proposed in ISO/IEC: R1, R2 and R3.¹²

The explanation of these methods follows:¹³

• Rating method R1—The approach to process attribute rating shall satisfy the following conditions:
  • “Each process outcome of each process within the scope of the assessment shall be characterized for each process instance, based on validated data.”
  • “Each process attribute outcome of each process attribute for each process within the scope of the assessment shall be characterised for each process instance, based on validated data.”
  • “Process outcome characterisations for all assessed process instances shall be aggregated to provide a process performance attribute achievement rating.”
  • “Process attribute outcome characterisations for all assessed process instances shall be aggregated to provide a process attribute achievement rating.”
• Rating method R2—The approach to process attribute rating shall satisfy the following conditions:
  • “Each process attribute for each process within the scope of the assessment shall be characterised for each process instance, based on validated data.”
  • “Process attribute characterisations for all assessed process instances shall be aggregated to provide a process attribute achievement rating.”
  • Rating method R3—Process attribute rating across assessed process instances shall be made without aggregation.

Furthermore, a section regarding aggregation methods should also be included in the updated COBIT 5 PAM. When performing an assessment, ratings may be summarized across one or two dimensions. For example, when rating a process attribute for a given process, “one may aggregate ratings of the associated process (attribute) outcomes—such an aggregation will be performed as a vertical aggregation (one dimension).”¹⁴ When rating a process (attribute) outcome for a given process attribute across multiple process instances, “one may aggregate the ratings of the associated process instances for the given process (attribute) outcome—such an aggregation will be performed as a horizontal aggregation (one dimension).”¹⁵ When rating a process attribute for a given process, “one may aggregate the ratings of all the process (attribute) outcomes for all the processes instances–such an aggregation will be performed as a matrix aggregation across the full scope of ratings (two dimensions).”¹⁶

The use of aggregation of ratings may vary according to the class, scope and context of an assessment. Process attributes are rated using an ordinal scale. An aggregation approach requires that the ordinal ratings be converted to interval values to perform aggregation. The validity of this conversion from ordinal ratings to interval values is dependent on two conditions:¹⁷

1. “The ordinal scale must be sufficiently constrained that the ordinal values are reasonably evenly spread. The rating scale defined in this international standard meets the requirement of being evenly spread.”
2. “There must be evidence of adequate sample size to assure adequate accuracy of the ordinal values. This condition is met for class 1 and class 2 assessments, both of which are sufficiently rigorous to require an adequate sample size.”

Since these conditions are met, then the ordinal ratings can be converted to interval values.

Process Dimension and Process Performance Indicators
This section defines the processes and the process performance indicators, also known as the process dimension, of the process assessment model. Since the proposed updated PAM is still based on COBIT, this section remains the same as in the original publication.

Process Capability Indicators
This section presents the process capability indicators related to the process attributes (PAs) associated with capability levels 1 to 5 defined in the capability dimension of the process assessment model.

Generally speaking, regarding the outcomes and generic practices (GPs), one can argue that only the following ISO/IEC 33020 attributes have changed:

• PA.2.1 Performance management process attribute
• PA.4.1 Quantitative analysis process attribute
• PA.4.2 Quantitative control process attribute
• PA.5.1 Process innovation process attribute

Due to space limitations, only a subset of the changes that need to be updated in this section will be detailed.

Regarding the PA.4.1 Quantitative analysis process attribute, two new outcomes and GPs should be included.¹⁸

Outcomes:

1. “The process is aligned with quantitative business goals.”
2. “Measurable relationships between process elements that contribute to the process performance are identified.”

GPs:

• GP1—“Align the process with quantitative business goals.”
• GP4—“Identify measurable relationships between process elements that contribute to the process performance.”

In addition, one outcome and related GP will disappear:

3. “Measurement results are used to characterise process performance” and its related GP 4.1.6.¹⁹

Regarding the generic work products, the manner in which they are addressed in the new ISO/IEC standard has completely changed.

Conclusion

The new ISO/IEC 330xx family of standards presents a more detailed and well-defined process assessment model than the older ISO/IEC 15504 family. The gaps regarding rating methods and aggregation methods perceived in the older standard have now been resolved with clear and standardized guidance on how to perform the appropriate actions. In addition, the definitions of some process attributes, outcomes and base practices are now more consistent. Therefore, for all these reasons, updating ISACA’s COBIT Process Assessment Model (PAM): Using COBIT 5 to this new standard is not only a necessity, but also an opportunity to improve the assessment of COBIT 5 processes.

Endnotes

¹ De Haes, S.; W. Van Grembergen; R. S. Debreceny; “COBIT 5 and Enterprise Governance of Information Technology: Building Blocks and Research Opportunities,” Journal of Information Systems, 2013, vol. 27, iss. 1, p. 307–324
² Ibid.
³ ISACA, COBIT 5, USA, 2012
⁴ Pasquini, A.; E. Galie; “COBIT 5 and the Process Capability Model: Improvements Provided for IT Governance Process,” Proceedings of FIKUSZ ’13 Symposium for Young Researchers, Obuda University Keleti Faculty of Business and Management, 2013, p. 67–76
⁵ ISACA, COBIT Process Assessment Model (PAM): Using COBIT 5, USA, 2012
⁶ International Organization for Standardization, ISO/IEC 15504-1, Information technology—Process assessment—Part 1: Concepts and vocabulary, 2004, http://www.iso.org/standard/38932.html
⁷ International Organization for Standardization, ISO/IEC 15504-2, Software engineering—Processassessment—Part 2: Performing an assessment, 2003, http://www.iso.org/standard/37458.html
⁸ International Organization for Standardization, ISO/IEC 33000, Series on Process Assessment, 2015, http://www.iso.org/home.html
⁹ Ibid.
¹⁰ Ibid.
¹¹ Ibid.
¹² Ibid.
¹³ Ibid.
¹⁴ Ibid.
¹⁵ Ibid.
¹⁶ Ibid.
¹⁷ Ibid.
¹⁸ Ibid.
¹⁹ Ibid.