Six Building Blocks for Effective Benchmarking | ISG

The practice of benchmarking has evolved over time, but the fundamentals still remain. In its simplest form, a benchmark is a comparison of one metric against another. It sounds simple, but to elicit real value, a benchmark needs to be built on top of a number of building blocks. Only then will it to uncover meaningful insights that lead to performance improvement opportunities.

There are six essential building blocks that need to be applied to elicit meaningful and insightful recommendations. The building blocks are applicable to all benchmarks but the insights below are garnered specifically from IT price and cost benchmarks. In this context, cost refers to internal performance benchmarks and price refers to external contract benchmarks, in which a provider is driven by commercial terms and returns.

Benchmark Fundamentals for Meaningful Insights

1. Consistency – Only a common data model will enable a direct comparison. This model needs to include easy-to-understand and defined terms, a common measurement time period and transparency on accounting practices like depreciation and tax. These steps are critical for attributing the outcome from the comparison to actual performance differences and not to a misalignment of data.
2. Validation – Review and validate the collected data to ensure it aligns to expectations and is consistent with the principles you have established. Perform a sense check to identify any potential aberrant data. A common option is to aggregate the information to a summary level so you can validate your data against market standard measures. This validation is often compared against a cost or price per workload unit. In the case of IT cost benchmarks, a broad range of measures like revenue numbers and productivity can also be used. This acts as a feedback loop to mitigate data integrity issues. It is also an opportunity to discuss, debate and refine a common understanding to ensure all participants buy into the benchmark process.
3. Comparison – The comparison data set needs to be based on the same data principles and model to deliver appropriate insights on areas of potential improvements. This is where the level of granularity and type of data is important. The more questions you ask, the more metrics you will have for analysis and this, in turn, will broaden the number and range of future improvements. However, you will need to consider the potential value of the information against the amount of effort expended to collect the data. This is where the target depth of insight will dictate the number and granularity of questions. As a rule of thumb, a strategic benchmark usually covers five to ten high-level measures; a detailed benchmark can run into hundreds.
4. Peers - At a strategic level, an industry peer group is used to combine business and IT metrics for a view on the value and cost of IT against business measures. Most tactical cost benchmarks use generic measures, so performance can be assessed to harness best practices across industry sectors. If the comparison is against external peers, then it is common to work with a benchmark specialist to ensure the peer database can accommodate your requirements and the data is current.
5. Targets – Deciding on the level of performance you want to achieve will have a direct bearing on your investment decisions, so value has to be assessed in this context. A common measure often used in benchmarking is the top quartile or 25^th percentile. This is when 25% of the data points are below the reference number and 75% are above. This measure is often used in conjunction with the peer average cost/price to assess where your numbers are in relation to your peers. Best practice would be to aim for a cost/price that is within this range. The logic is that, if your costs/prices are below the top quartile, then you may be underinvesting in the service. If the costs/prices are above the peer average, then the service is uncompetitive. Most companies aim for the “Goldilocks Zone” between the peer top quartile and the average.
6. Action plans – What’s next after generating the output? Analysis, analysis and more analysis! The real value of any benchmark, after gaining an understanding of how you compare to peers, is learning how to improve suboptimal performance and what actions to take to be more cost competitive. The easiest way to be cost competitive is to hold off on investing in the environment. This works well in terms of optics but not so well when the users and stakeholders receive a substandard service. The key, then, is to provide value where the cost of the service is balanced against the gain the business receives through the investment. Remember benchmarking is a journey, not a destination. This exercise should be the basis of a continuous improvement plan, where improvements can be accurately quantified and mapped against milestones.

Benefits of IT Cost and Price Benchmarks

The benefits of benchmarking are many and varied and are markedly different depending on whether the IT service is delivered internally or by a third party.

Cost (Internal) – Organizations don’t know what they don’t have sight of, so the insights a benchmark is able to provide can lead to marked improvements in performance. Benchmarking is the map that allows an organization to drive from A to B with consistent terms and frames of reference. Organizations need to choose the vehicle and route that is right for them. Most organizations think of benchmarking in terms of a financial outcome, but the real value in benchmarking is how it drives efficiency and effectiveness initiatives that bring about positive change. These initiatives can be wide ranging from improvements in automation, process maturity, procurement practices, customer satisfaction, sourcing, delivery locations, technology deployments and service levels to name a few. 

Often a refrain from senior IT leaders is that the business sees IT purely as a cost and, therefore, is often forced into a cycle of cost-cutting exercises to improve margins. The ability to provide objective evidence as to the cost that equivalent peer organizations spend on their IT can often change the complexion of this dialogue. Higher costs can sometimes be justified when there is direct evidence that it brings about measurable business benefits or mitigates service risk. This is especially true for security, where a lack of investment can have a significant and direct impact on the business.  

Communication is a key benefit of cost benchmarking, as it promotes informed interactions and engenders a more cooperative attitude, especially when it comes to prioritizing investments. This is often the conduit to achieving a partnership approach between the business and IT, which can accelerate business outcomes.

Some organizations have actively used benchmarking in their annual budgeting cycle to assess future investment levels. This forms one of the pillars of their continuous improvement program with costs that are fully transparent and open to external market testing.

Price (Sourcing) – When an organization decides to have a service delivered by a third party for a fixed price, it is important to build in pricing safeguards. Most IT managed service contract durations have now settled on three-to-five-year periods, which is an advance on the early days of outsourcing when 10-year contracts were not uncommon. However, even with these shortened time frames, prices can change dramatically. In ISG’s recent ITO Pricing Trends report, most asset-heavy services saw an annual price drop of between -4% and -9%. When assessed across the full range of IT services, annual price changes varied from +6% to -9%. It is therefore extremely beneficial to include a benchmark clause in your contract, especially if there is no automatic year-on-year price reduction to ensure your service remains price competitive. This clause will also provide a defined mechanism to establish the right price without the need to actively negotiate with providers.

Organizations have also used benchmarking when a contract is due to end with a favored provider, to ensure the new contract price will be market competitive without the need to work through a laborious Request for Proposal (RfP) process. 

Service providers can also benefit from benchmarking their services, as this gives them insights as to whether the price for a service is market competitive. This helps the service provider sell or negotiate at the right price. Where prices have not been competitive, internal cost benchmarks have helped identify where greater efficiencies can be garnered in the way services are delivered.

Benchmarking gives organizations a framework to understand where they are today and identifies potential optimization opportunities to guide them to where they should be in the future. If you invest the right amount of effort, performance rewards are only a benchmark away!

ISG benchmarking services help companies track and meet their cost, performance and quality objectives. Our data is unmatched – it comes from detailed engagements with client organizations and providers rather than just from market research. Contact us to find out how to get started.