Service Management Metrics

From YaSM Wiki
Jump to: navigation, search

share this page on LinkedInshare this page on Twittershare this page
auf Deutsch


 

Process metrics (at times also referred to as 'key performance indicators' or 'KPIs') are used to assess if the service management processes are running according to expectations.

Proceed to: Process metrics for the service lifecycle processes | Process metrics for the supporting service management processes

 

Defining process metrics

Service management metrics (Key Performance Indicators - KPIs) are used to measure the performance of the YaSM service management processes.

Defining KPIs is above all about deciding what exactly is considered "successful" process execution. Once this is established, suitable metrics can be defined and subsequently measured. The process owners are then in a position to evaluate the quality of their processes, which in turn is the basis for process improvement.

The exact definitions of the metrics will vary depending on the nature of the service provider organization, so the key performance indicators contained in this wiki can only be suggestions.

Generally speaking, there is no shortage of recommendations for service management metrics. If further ideas are needed, the various service management frameworks like ITIL® [1], COBIT® [2] and USMBOK™ [3] may be consulted - but we advise using a focused set of metrics or KPIs that can be managed and acted upon in the long term.

A widely accepted principle holds that process metrics should be SMART:

  • Specific
  • Measurable
  • Achievable
  • Result-oriented
  • Timely.

 

Quantitative and qualitative metrics

When developing sets of process KPIs, a distinction is often made between quantitative metrics and qualitative metrics (although the line between the two may at times be blurred):

  • As the name suggests, quantitative metrics are typically straightforward measurements of volumes and frequencies (e.g. the number of service requests reported to 1st level support). Such metrics may not say much about the quality of the service request fulfillment process, but they can be useful for managing the resources required to fulfill service requests.
  • Qualitative metrics have a more direct relationship to successful process execution. For example, if a service provider aims to resolve a high percentage of incidents directly in 1st level support, the 1st-line resolution rate would be a suitable qualitative metric.

 

KPIS for lifecycle processes

How to measure the performance of the YaSM service lifecycle processes? The following tables provide you with suggestions for quantitative and qualitative process metrics.

 

Metrics for the service strategy process

Process metrics: "Set the strategic direction"
Quantitative process metrics Definition
Number of strategic reviews
  • Number of strategic reviews performed during the reporting period.
Number of strategic objectives
  • Number of strategic objectives defined following strategic reviews.
Number of strategic initiatives
  • Number of strategic initiatives launched during the reporting period.
Qualitative process metrics Definition
Share of strategic objectives pursued through initiatives
  • Percentage of strategic objectives which are being actively pursued with defined strategic initiatives.
Number of successful strategic initiatives
  • Number of strategic initiatives which were successfully completed during the reporting period.
Number of new services introduced
  • Number of new or significantly changed services which were introduced following a strategic review.
Number of service improvements implemented
  • Number of service improvements implemented due to insights gained during strategic reviews.
Number of services retired due to strategic reviews
  • Number of services which were retired due to insights gained during a strategic review.
Number of process improvements implemented
  • Number of process improvements implemented due to insights gained during strategic reviews.

Process description | → top

 

 

Metrics for the service design process

Process metrics: "Design new or changed services"
Quantitative process metrics Definition
Number of service designs
  • Number of new or significantly changed services designed during the reporting period.
Number of service implementation blueprints
  • Number of implementation blueprints produced for new or significantly changed services.
Number of requirements specifications
  • Number of requirements specifications produced for infrastructure items or other service components which were needed to provide new or significantly changed services.
Qualitative process metrics Definition
Percentage of new services introduced using the service design process
  • Percentage of new or significantly changed services which were introduced using the documented service design process vs. new services introduced without using that process.
Number of renegotiated service requirements
  • Number of new or significantly changed services where the original requirements had to be renegotiated during the design phase, possibly grouped by reasons like "Economically unfeasible", "Technically unfeasible", ...
Number of services unexpectedly impacted
  • Number of other services which were unexpectedly found to be impacted by the introduction of new or significantly changed services, due to errors made during the service design phase.
Completeness of service implementation blueprints
  • Percentage of service implementation blueprints which covered all required aspects. There may be one metric for every aspect covered, for example "Percentage of blueprints covering capacity and performance aspects", "Percentage of blueprints covering availability aspects", ...
Percentage of designed services not delivering the expected outcomes
  • Percentage of new or significantly changed services that failed to deliver the expected outcomes due to errors made during the service design phase.

Process description | → top

 

 

Metrics for the service build process

Process metrics: "Build new or changed services"
Quantitative process metrics Definition
Number of new services implemented
  • Number of new or significantly changed services implemented during the reporting period.
Number of tests and test cases
  • Number of tests and test cases carried out, possibly grouped by services to be implemented, test types, ...
Qualitative process metrics Definition
Percentage of new services introduced using the service build process
  • Percentage of new or significantly changed services which were introduced using the documented service build process vs. new services introduced without using that process.
Duration of service implementation
  • Duration of service implementation projects from autho-rization of the service development project to confirmation of service readiness.
Percentage of automatic software distribution
  • Percentage of software components deployed by means of automated distribution.
Percentage of failed component acceptance tests
  • Percentage of service components which failed to pass acceptance tests.
Number of identified errors
  • Number of identified errors during testing, possibly grouped by services to be implemented, test types, ...
Time for error fixing
  • Average times until re-submission of fixed components after errors were found, possibly grouped by services to be imple-mented, component types, ...
Incidents caused by new service components
  • Number of incidents attributable to newly deployed service components, grouped by services and service components.
Percentage of implemented services not delivering the expected outcomes
  • Percentage of new or significantly changed services which failed to deliver the expected outcomes due to errors made during the service build phase.

Process description | → top

 

 

Metrics for the service operation process

Process metrics: "Operate the services"
Quantitative process metrics Definition
Number of services monitored
  • Number of services which are constantly monitored in order to pro-actively avoid service interruptions or deficiencies in service quality.
Number of services where quality is measured and reported
  • Number of services whose quality levels are regularly measured and reported, as a basis for improving the services.
Number of generated significant events
  • Number of events generated by event monitoring systems, grouped by categories (e.g. events related to particular systems, events requiring human interaction, events triggering an automated response, ...).
Qualitative process metrics Definition
Share of monitored services
  • Percentage of services which are constantly monitored.
Share of automated responses to events
  • Number of events where an automated response is triggered vs. the total number of events requiring a response.
Share of services with quality reporting
  • Percentage of services whose quality levels are regularly measured and reported.
Number of service incidents due to inadequate maintenance
  • Number of service incidents which could have been avoided by more adequate preventive maintenance.
Number of service level breaches
  • Number of service levels breached, possibly grouped by types of service level, services, clients, ...
Unused capacity
  • Percentage of capacity reserves at times of normal and maximum demand, possibly grouped by services, systems, ...
Number of improvement initiatives
  • Number of issues detected during service quality reporting which result in a service improvement initiative being started.

Process description | → top

 

 

Metrics for the incident resolution process

Process metrics: "Resolve incidents and service requests"
Quantitative process metrics Definition
Number of incidents and service requests
  • Number of incidents and service requests logged by 1st level support, possibly grouped by priorities, categories, clients, ...
Number of major incidents
  • Number of incidents resolved by invoking the special procedure for major incidents.
Qualitative process metrics Definition
Average initial response time
  • Average delay between the time a user reported an incident or service request and the time that 1st level support responded to that incident or request, possibly grouped by priorities, categories, clients, ...
Average resolution time
  • Average time for resolving incidents or service requests, possibly grouped by priorities, categories, clients, ...
Resolution within agreed time
  • Percentage of incidents and service requests resolved within the target resolution times specified in the service agreements, possibly grouped by priorities, categories, clients, ...
First time resolution rate
  • Percentage of incidents and service requests resolved by 1st level support during the first call, possibly grouped by priorities, categories, clients, ...
Number of standard incidents and service requests
  • Number of incidents and service requests which were resolved by applying known resolution methods (typically defined in incident models) , possibly grouped by priorities, categories, clients, ...
Incidents resolved remotely
  • Number of incidents and service requests resolved remotely (i.e. without carrying out work at the user's location), possibly grouped by priorities, categories, clients, ...
Incidents resolved pro-actively
  • Number of incidents reported and resolved pro-actively (i.e. incidents resolved before impacting business processes on the client side), possibly grouped by priorities, categories, clients, ...
Share of escalated incidents
  • Percentage of incidents where a hierarchic escalation occurred.
Average resolution effort
  • Average work effort for resolving incidents and service requests, possibly grouped by priorities, categories, clients, ...

Process description | → top

 

 

Metrics for the problem resolution process

Process metrics: "Resolve problems"
Quantitative process metrics Definition
Number of resolved problems
  • Number of resolved problems, possibly grouped by priorities, categories, ...
Number of workarounds
  • Number of workarounds provided to incident management.
Qualitative process metrics Definition
Average resolution time
  • Average time for resolving problems, possibly grouped by priorities, categories, ...
Problems identified pro-actively
  • Percentage of problems raised as part of the pro-active problem identification process.
Number of incidents per known problem
  • Number of reported incidents linked to the same problem after a problem has been identified.
Average time until identification of the cause
  • Average time between problem registration and identification of the underlying cause.
Average resolution effort
  • Average work effort for resolving problems, possibly grouped by priorities, categories, ...

Process description | → top

 

 

Metrics for the service improvement process

Process metrics: "Improve the services"
Quantitative process metrics Definition
Number of service reviews
  • Number of formal service reviews carried out during the reporting period.
Qualitative process metrics Definition
Number of identified weaknesses
  • Number of weaknesses to be addressed by improvement initiatives which were identified during service reviews, possibly grouped by services.
Number of defined service improvement initiatives
  • Number of defined improvement initiatives, resulting from identified weaknesses during service reviews, possibly grouped by services.
Number of successful service improvement Initiatives
  • Number of service improvement initiatives which were completed successfully during the reporting period, possibly grouped by services.

Process description | → top

 

 

KPIs for supporting processes

How to measure the performance of the YaSM supporting service management processes? The following tables provide you with suggestions for quantitative and qualitative process metrics.

 

Metrics for the SMS process

Process metrics: "Set up and maintain the service management system"
Quantitative process metrics Definition
Number of documented processes
  • Number or service management processes which are documented in the process model.
Number of process reviews and audits
  • Number of formal process reviews and audits carried out during the reporting period.
Qualitative process metrics Definition
Share of processes with defined service owners
  • Percentage of service management processes which are assigned to a process owner.
Share of processes with defined process metrics
  • Percentage of service management processes where process metrics are defined, measured and acted upon.
Share of processes with regular reviews
  • Percentage of service management processes which are subjected to regular process reviews or audits according to the process review plan.
Number of identified weaknesses
  • Number of weaknesses to be addressed by improvement initiatives which were identified during process reviews or audits, possibly grouped by processes.
Number of defined process improvement initiatives
  • Number of defined improvement initiatives, resulting from identified weaknesses during process reviews or audits, possibly grouped by processes.
Number of successful process improvement Initiatives
  • Number of process improvement initiatives which were completed successfully during the reporting period, possibly grouped by processes.
Number of changes to the service management policies
  • Number of changes to the service management policies resulting from process reviews or audits, possibly grouped by policies.

Process description | → top

 

 

Metrics for the portfolio maintenance process

Process metrics: "Maintain the service portfolio"
Quantitative process metrics Definition
Number of services
  • Number of services managed through the service portfolio, grouped by customer and (internally or externally supplied) supporting services.
Number of new services
  • Number of services added to the service portfolio during the reporting period.
Number of changed services
  • Number of services in the service portfolio where new versions have been introduced.
Number of retired services
  • Number of services that were retired.
Qualitative process metrics Definition
Services documented in the services portfolio
  • Percentage of services offered by the service provider which are documented in the service portfolio.
Services defined by service definitions
  • Percentage of services in the service portfolio which are defined by authorized service definitions.
Services covered by agreements
  • Percentage of services in the service portfolio which are covered by agreements (i.e. customer services covered by customer service agreements and supporting services covered by operational or external service agreements).
Number of detected service inconsistencies
  • Number of inconsistencies between service definitions detected during service portfolio reviews (especially inconsistencies between customer services and supporting services).

Process description | → top

 

 

Metrics for the CRM process

Process metrics: "Manage customer relationships"
Quantitative process metrics Definition
Number of customer meetings
  • Number of formal customer meetings held during the reporting period.
Number of customer satisfaction surveys
  • Number of formal customer satisfaction surveys carried out during the reporting period.
Number of customer complaints
  • Number of received customer complaints during the reporting period.
Qualitative process metrics Definition
Customer satisfaction
  • Average measured customer satisfaction for each service, determined by means of customer satisfaction surveys.
Number of accepted customer complaints
  • Number of received customer complaints which were accepted (deemed justified and acted upon).
Number of new services introduced
  • Number of new or significantly changed services introduced due to insights gained during customer meetings or satisfaction surveys.
Number of service improvements implemented
  • Number of service improvements implemented due to insights gained during customer meetings or satisfaction surveys.
Number of new customers
  • Number of newly won customers during the reporting period.
Number of lost customers
  • Number of customers which were lost to competing service providers.

Process description | → top

 

 

Metrics for the configuration management process

Process metrics: "Manage configuration information"
Quantitative process metrics Definition
Number of CI types
  • Number of configuration item types managed through the configuration management system (CMS).
Number of CIs
  • Number of configuration items managed through the configuration management system (CMS), possibly grouped by CI types.
Number of changes to CIs
  • Number of recorded changes to configuration items, possibly grouped by CI types.
Qualitative process metrics Definition
Number of incidents owing to inaccurate CMS Information
  • Number of service incidents reported where the underlying cause of the incident was the result of inaccurate configuration information.
Verification frequency
  • Frequency of physical verifications of configuration information in the CMS.
Share of automatic verification
  • Percentage of configuration items in the CMS for which detection and verification can be done in an automated way.
Effort for CMS verifications
  • Average work effort for physical verifications of the CMS contents.
CMS coverage
  • Percentage of infrastructure components for which data is kept in the CMS.
Number of unauthorized changes detected
  • Number of unauthorized changes identified as a result of an audit.
Number of CMS errors
  • Number of errors found in the CMS as a result of an audit.

Process description | → top

 

 

Metrics for the change assessment process

Process metrics: "Assess and coordinate changes"
Quantitative process metrics Definition
Number of change assessments
  • Number of proposed changes (RFCs) assessed by the change assessment process, possibly grouped by change types, related configuration items, ...
Number of changes
  • Number of changes implemented, possibly grouped by change types, related configuration items, ...
Number of major changes
  • Number of major changes assessed by the change assessment process.
Number of emergency changes
  • Number of emergency RFCs assessed by the change assessment process.
Number of change models
  • Number of defined pre-authorized standard changes, documented in change models.
Number of post-implementation reviews
  • Number of changes that were subjected to a review after implementation, possibly grouped by change types, related configuration items, ...
Qualitative process metrics Definition
Time for change authorization/ rejection
  • Average time from registering an RFC with the change manager until a decision on the RFC is reached (i.e. until it is either approved or rejected).
Change authorization rate
  • Number of authorized vs. rejected RFCs.
Change success rate
  • Percentage of changes implemented successfully.
Number of unauthorized changes
  • Number of changes that were implemented without authorization (identified, for example, during configuration audits).
Number of change-related incidents
  • Number of unexpected service incidents caused by the implementation of authorized changes.

Process description | → top

 

 

Metrics for the project management process

Process metrics: "Manage projects"
Quantitative process metrics Definition
Number of projects
  • Number of projects started, in progress and completed during the reporting period.
Qualitative process metrics Definition
Percentage of projects with project charters
  • Percentage of projects which were started with a signed project charter in place.
Number of changes to project scope
  • Number of changes to the project scope after project start, per project.
Adherence to planned resources
  • Actual vs. planned consumption of financial and human resources, per project-
Adherence to project schedule
  • Actual vs. planned milestone and project completion dates, per project.
Number of cancelled projects
  • Number of projects that have ended without achieving their stated goals.

Process description | → top

 

 

Metrics for the security process

Process metrics: "Ensure security"
Quantitative process metrics Definition
Number of security risks
  • Number of risks managed through the register of security risk.
Number of security policies
  • Number of underpinning security policies maintained and published by the security manager.
Number of security alerts
  • Number of security alerts issued by the security manager, possibly grouped by type of security threat, services affected, infrastructure components affected, ...
Qualitative process metrics Definition
Percentage of security risks with risk responses
  • Percentage of identified security risks which are covered by suitable risk responses.
Number of implemented security improvements
  • Number of security improvements which were implemented in response to newly identified security threats.
Time for implementation of risk responses
  • Average duration from the identification of a new security risk to the implementation of a suitable response.
Number of security breaches
  • Number of detected security breaches, possibly grouped by type of security breach, services affected, infrastructure components affected, ...
Number of security incidents
  • Number of reported security incidents, possibly grouped by incident priority, services affected, infrastructure components affected, ...
Number of security-related service downtimes
  • Number of security incidents causing service s to be unavailable.
Number of security training sessions
  • Number of security training sessions carried out, possibly grouped by type of training, target group, ...
Number of security tests
  • Number of security tests carried out, possibly grouped by type of test, security systems or mechanisms tested, ...
Number of identified issues during security tests
  • Number of identified issues during security tests to be addressed by security improvement initiatives.

Process description | → top

 

 

Metrics for the disaster preparation process

Process metrics: "Prepare for disaster events"
Quantitative process metrics Definition
Number of managed disaster events
  • Number of (types of) disaster events for which prevention measures are deemed necessary and which are managed through the register of managed disaster events.
Number of continuity plans
  • Number of continuity plans maintained by the continuity manager.
Qualitative process metrics Definition
Percentage of managed disaster events covered by continuity arrangements
  • Percentage of identified and managed disaster events which are covered by suitable continuity plans.
Number of implemented continuity arrangements
  • Number of continuity arrangements which were implemented in response to newly identified threats from disaster events.
Time for implementation of continuity arrangements
  • Average duration from the identification of a new threat from a disaster event to the implementation of a suitable response.
Number of disaster events
  • Number of disaster events occurred during the reporting period.
Number of disaster events with continuity arrangements in place
  • Number of disaster events occurred during the reporting period for which a suitable continuity plan was in place.
Number of continuity training sessions
  • Number of continuity training sessions carried out, possibly grouped by types of training, target groups, ...
Number of continuity tests
  • Number of continuity tests carried out, possibly grouped by types of continuity arrangements, ...
Number of issues identified during continuity tests
  • Number of issues identified during continuity tests to be addressed by continuity improvement initiatives.

Process description | → top

 

 

Metrics for the compliance process

Process metrics: "Ensure compliance"
Quantitative process metrics Definition
Total number of compliance requirements
  • Number of compliance requirements managed through the compliance register, possibly grouped by types of requirements (e.g. legal, ISO standard, internal policy, ...).
Number of added compliance requirements
  • Number of compliance requirements added to the compliance register during the reporting period due to new laws or regulations, possibly grouped by types of requirements.
Number of removed compliance requirements
  • Number of compliance requirements removed from the compliance register during the reporting period, possibly grouped by types of requirements.
Qualitative process metrics Definition
Percentage of compliance requirements with controls in place
  • Percentage of compliance requirements where conformity is ensured through suitable controls or mechanisms, including defined responsibilities for their implementation and operation.
Percentage of compliance requirements covered by reviews
  • Percentage of compliance requirements which are covered by regular compliance reviews and audits.
Number of compliance reviews
  • Number of compliance reviews and audits carried out.
Number of issues identified during compliance reviews
  • Number of non-compliance issues identified during compliance reviews and audits, which are to be addressed for example by modifications to processes or services.
Number of issues reported outside compliance reviews
  • Number of non-compliance issues reported outside of formal compliance reviews and audits.
Time for fixing compliance issues
  • Average time lag between identification and resolution of non-compliance issues.

Process description | → top

 

 

Metrics for the HR management process

Process metrics: "Manage human resources"
Quantitative process metrics Definition
Number of staff
  • Total number of staff employed by the service provider organization.
Number of skills
  • Number of skills managed through the skills inventory.
Number of staff entering the organization
  • Number of employees who joined the service provider organization during the reporting period.
Number of staff leaving the organization
  • Number of employees who left the service provider organization during the reporting period.
Qualitative process metrics Definition
Number of skills added
  • Number of skills added to the skills inventory during the reporting period.
Number of skills removed
  • Number of skills removed from the skills inventory during the reporting period.
Number of skill development measures
  • Number of measures carried out through a skills development plan to improve the skills of individual employees, possibly grouped by type of skill, type of training, ...
Share of skills covered by development plans
  • Percentage of skills in the skills register for which training and education is managed through a skills development plan.

Process description | → top

 

 

Metrics for the supplier management process

Process metrics: "Manage suppliers"
Quantitative process metrics Definition
Number of suppliers
  • Number of external suppliers, possibly grouped by types, status, ...
Number of preferred suppliers
  • Number of external suppliers with "preferred supplier" status.
Number of new suppliers
  • Number of suppliers added to the supplier portfolio during the reporting period.
Number of terminated supplier agreements
  • Number of suppliers whose contracts were terminated during the reporting period.
Number of purchase orders
  • Number of purchase orders processed, possibly grouped by product categories, purchase order types (e.g. orders for non-standard products and services, orders for standard products), ...
Qualitative process metrics Definition
Share of external services covered by agreements
  • Percentage of externally supplied supporting services which are covered by external service agreements and corresponding service definitions.
Share of preferred suppliers
  • Share of goods and services procured from external suppliers with "preferred supplier" status, as a percentage of the value of all purchase orders.
Share of reviewed agreements
  • Percentage of supplier agreements which were reviewed for continued relevance during the reporting period.
Number of supplier meetings
  • Number of formal supplier meetings held during the reporting period.
Number of detected contract breaches
  • Number of contract breaches, for example detected during supplier review meetings or service reviews.
Time for issuing purchase orders
  • Average time between receiving internal purchasing requests and sending out purchase orders to suppliers, possibly grouped by product categories, purchase order types, ...

Process description | → top

 

 

Metrics for the financial management process

Process metrics: "Manage service financials"
Quantitative process metrics Definition
Total budget
  • Total budget of the service provider organization, possibly broken down into cost and revenue categories.
Budget change on previous period
  • Percentage of budget increase or decrease compared to the previous budgeting period, possibly broken down into cost and revenue categories.
Qualitative process metrics Definition
Adherence to financial forecasts
  • Deviation between predicted and actual spending and revenues, possibly grouped by cost and revenue categories.
Share of costs allocated to services
  • Percentage of overall costs which are allocated to the provision of particular services, possibly grouped by cost categories.
Share of charged services
  • Percentage of services where the customers are charged for service provision.
Share of profitable services
  • Percentage of customer services where revenues exceed the cost for service provisioning.
Share of investments with business cases
  • Percentage of investments approved during the reporting period where the budget request included an assessment of costs vs. benefits.
Share of investments with realized benefits
  • Percentage of investments made during the reporting period where the predicted benefits were realized.

Process description | → top

 

 

Notes

[1] ITIL® is a registered trade mark of AXELOS Limited.
[2] COBIT® is a registered trademark of ISACA (Information Systems Audit and Control Association).
[3] USMBOK™ is a registered trade mark of Virtual Knowledge Solutions International Incorporated (VKSII).

By:  Andrea Kempter   and  Stefan Kempter Contributor: Stefan Kempter, IT Process Maps GbR, IT Process Maps.

 

Defining process metrics Quantitative and qualitative metrics KPIs for lifecycle processes KPIs for supporting processes