Call Center Benchmarking Peer Group Selection

Download This Article Forward To A Friend These inconsistencies become even more pronounced when looking at costs, and specifically cost per call, one of the foundation metrics that every call center should be tracking (see MetricNet’s article, The Essential KPI’s, in the April, 2007 edition of Call Center Magazine for a more detailed explanation of foundation metrics).  Because of differences in the way call centers account for their costs – depreciating vs. expensing an asset, for example – cost metrics can vary dramatically between call centers even when their spending levels are exactly the same. My point is that any data used for benchmarking is imprecise.  That’s right…it lacks precision!  No one else in the industry will admit to this fundamental flaw in benchmarking, but it is critical to understand this limitation.  Too often, those who engage in benchmarking draw conclusions based upon small performance gaps.  This can lead to serious problems because a call center may take action based on a perceived performance gap that does not really exist.  Despite this, benchmarking is still an extremely valuable tool.  However, it is a blunt instrument, and the lack of precision has profound implications for how the results or your benchmark are interpreted. Benchmarking is good for identifying large performance gaps, but is simply ineffective at identifying small differences in performance between your call center and a peer group.  The implication is that small performance gaps are usually meaningless.  These small performance gaps are “down in the noise”, as they say.  As a rule of thumb, whenever I come across a benchmarking performance gap in the 1% - 5% range, I ignore it.  Benchmarking data is simply not precise enough to guarantee accuracy to within plus or minus 5%.  Performance gaps in the 5% - 10% range, however, may be meaningful.  But at this level or I look for corroborating evidence from other metrics before I assume that a real performance gap has been uncovered.  It is only when the performance gap is 10% or greater that I can conclude with confidence that a real performance gap exists.  In the jargon of the industry, this is “directional accuracy”.  The results of any benchmark are never precise, but they are directionally accurate, meaning that you can act on them with confidence when the performance gap is large enough to outweigh any inconsistencies in the benchmarking data that is reported. The second myth I would like to address is the notion that you can only benchmark against other call centers that look just like yours.  This is pure fallacy.  First off, if a call center looked just like yours, there would be no point benchmarking it because it would perform just like yours!  Secondly, there is no such thing as a call center that looks just like yours”…it doesn’t exist!  There are simply too many differences between call centers to even hope that you will find a peer group of call centers that look just like yours.  These differences include the types of transactions handled, the volume of transactions, geographic location, and a host of other factors. Both of these points are designed to make a larger point, which is that you have a lot of latitude and flexibility when it comes to peer group selection.  In fact, this is one of the most creative parts of the benchmarking process.  Please rid yourself of the notion that you can only benchmark against other call centers from your industry.  As you will see below, some of the best benchmarking candidates are likely to come from outside of your industry.

Dynamic Peer Group Selection

Dynamic Peer Group Selection (DPGS) ^TM is a clearly defined process of selecting companies for benchmarking that ensures a fair and valid comparison of data from one call center to the next. The DPGS^TM process assumes that the benchmarking data you are compared to is timely and accurate, and that there is a common methodology to collect the data.    There should also be a set of measurements, or KPI’s, agreed upon for the benchmarking comparison. DPGS^TM is built upon a number of criteria that should be considered when selecting a peer group for any benchmark.  These criteria include:

• Willingness to benchmark
• Transaction types
• Scale (i.e. volume of transactions handled)
• Industry
• Geography

These are among the most important criteria to consider in peer group selection, but this is by no means an exhaustive list.  Additionally, even taking these factors into consideration, the benchmarking performance comparisons by themselves may be invalid unless other adjustments are made to the data.  As I discuss each of the major criteria that make up the DPGS^TM methodology, I will also explain how performance differences due to these factors can be normalized out of any benchmarking comparison. Let’s take a closer look at each component of  DPGS^TM.

Willingness to Benchmark

Contrary to what some may believe, benchmarking is not a cloak-and-dagger exercise that is performed surreptitiously, without the knowledge of the call centers being benchmarked.  Although it may be possible using competitive analysis techniques to learn some things about a call center without their participation, true quantitative benchmarking requires the active participation of every call center in the peer group. Some companies fear that such active participation in benchmarking may publicly reveal information about their performance that they would rather keep private.  In this case, it is important to implement measures to ensure the privacy of their data.  When participating in any benchmark, you should make sure that your benchmarking consultant or facilitator takes precautions to protect the identity and security of your confidential data, just as MetricNet does in its syndicated benchmarks. At MetricNet, we believe that willingness to benchmark is the single most important factor in selecting call centers for your benchmarking peer group.  Call centers that are actively and enthusiastically engaged in the benchmarking process are far better candidates for benchmarking than those who only grudgingly share their data, or who otherwise don’t invest the time necessary to provide valid data for their benchmark. Although such a willingness to share data is no panacea, it is a prerequisite to successful benchmarking.  That, in addition to committing to metrics; putting the benchmarking infrastructure in place; consistently collecting accurate and complete data; and rigorously analyzing the performance gaps. This first component of  DPGS^TM may seem obvious, but it is surprising how often this bit if wisdom is ignored.  So above all, you should seek out call centers for your peer group that are willing to share their data on an open and candid basis.

Transaction Types

It should be immediately obvious that you would never benchmark a collections call center against a sales call center, or a customer service call center against a retail sales call center.  The transaction types are simply too different to obtain a valid benchmarking comparison.  The handle times will be different, the nature and complexity of the calls will be different, the agent skill sets will be different, and the performance metrics will be different. So the second criteria in  DPGS^TM is to ensure that the call centers you benchmark against are handling similar transaction types.  The peer group doesn’t necessarily have to have the same transaction volumes, but the types of transactions handled should be very similar.  If your call center handles credit approval and inbound sales, then the call centers you benchmark against should do the same thing. But keep in mind that handling the same types of transactions does not necessarily ensure a fair benchmarking comparison.  You must also take into account the relative volumes of each transaction.  As Figure 1 below illustrates, different call centers can handle the same types of transactions, but if the percentage of each transaction type is different, the aggregate handle times will also be different even if the handle time for each transaction type is the same.  Since call handle time is the single biggest driver of labor, and hence cost, these call centers may appear to have differences in their cost per call, when in fact it is the percentage of each transaction type that is driving the cost differences shown.  Fortunately these differences are easily normalized by making adjustments for the unique mix of calls in your call profile.

Scale

Virtually everything in the call center is subject to scale economies.  This is particularly true when it comes to the volume of contacts handled.  The approximate scale effect for volume is 7%.  What this means is that every time the number of transactions doubles, you should expect to see the cost per contact decline by 7%.  So, for a call center that handles 50,000 transactions per month at a cost of $5 per contact, you could expect to see the cost per contact drop by 7%, to $4.65 per transaction, if the volume doubled to 100,000 transactions per month.  Likewise, if the volume doubled yet again, to 200,000 transactions per month, the cost per contact would decline by another 7%, from $4.65 to $4.32 per transaction.  This is one of the major drivers of the trend towards consolidation that we see in the industry.  Larger call centers are simply more efficient than smaller call centers due to this scale effect.  This trend is illustrated in Figure 2 below, which shows the effect of scale for more than 100 different call centers.

When selecting a peer group for benchmarking comparison, you should strive to identify peers that are similar to yours in the number of transactions handled.  Additionally, you should make adjustments for any differences in scale between your call center and the peers by adjusting your cost per contact using the 7% rule mentioned above.

Industry

Benchmarking solely within your own industry and against direct competitors may be appropriate during the early stages of benchmarking, when the competitive “gap” between your organization and the best in your industry is the widest.  But as your organization’s performance improves, the gap will narrow and it will become necessary to reach for loftier goals.  To achieve world-class performance, superior practices from non-competitors must be adopted.  This requires that you benchmark against out-of-industry companies. Having call centers from your particular industry is the least important factor in  DPGS^TM.  It is not uncommon to see multiple industries represented in a call center benchmark.  In fact, it is encouraged!  As you will see below, you often gain more insight by benchmarking against call centers from outside of your industry.  Nevertheless, call centers from a particular industry, whether from utilities, financial services, retail, or any other industry, do share certain common characteristics.  So it is worth considering these in-industry call centers for your peer group.  But once again, industry considerations should take a back seat to other factors such as scale and transaction type when selecting your peer comparison group. The more important point when it comes to selecting benchmarking peers from a particular industry is the potential to gain “breakthrough insights” by benchmarking against call centers from outside of your industry.  Federal Express beat the competition in the early 1990’s by developing the most advanced package tracking system in the express delivery business.  They were the first in their industry to use bar coding and computerized package tracking, and other competitors soon followed suit.  But they did not adopt this technology from another express company.  Rather, the bar code technology was originally “borrowed” from the grocery store industry, and computerized package tracking was “borrowed” from a government logistics operation.  Benchmarking companies outside of the express package industry gave Federal Express a decided lead in their industry at the time.  It helped them achieve the highest profitability of any player in the industry, and forced the competition to play catch up. The same is true of call centers.  MetricNet has seen literally hundreds of examples of call centers that have gained a competitive advantage by adopting ideas, processes, and technologies from outside the industry.  As you can see in Figure 3 below, the collections call centers from the credit card industry are much more effective at collections than their counterparts from the energy, insurance, or telecommunications industries.  As such, it would be a good idea for any company that is benchmarking a collections call center to include some top performers from the credit card industry in the benchmarking peer group.  The same point is valid for any other type of call center.  When selecting your benchmarking peer group you should you should always consider candidates from outside your industry because they often provide the greatest insights for call center improvement. The cardinal rule when selecting benchmarking candidates is to maximize the amount of learning you take away from the process.  Selecting companies in the same industry that tell the same story will often yield fewer insights than benchmarking against diverse organizations, each of whom takes an innovative and unique approach to the functions being benchmarked.

Geography

The main factor that is affected by geography is cost; specifically labor cost.  North American call centers, for example, never benchmark against call centers from India or other low-cost regions of the world because the labor cost differential is simply too great.  Even within the United States and Canada, starting salaries can vary by as much as 50% depending upon where the call center is located. Cost differences due to geographic disparities can be normalized or “factored out” when doing your benchmark.  The basic approach is to look at the cost of living index for a particular region, and adjust the labor costs accordingly.  These cost of living indexes are published by a variety of services, but the one used most frequently by MetricNet is produced by the American Chamber of Commerce Association.  Figure 4 below shows the cost per contact for various bank call centers in different regions of North America.  You can see that the cost per contact when unadjusted for cost of living differences is quite substantial.  However, once the cost of living differences are factored into each call center’s costs, the range of values for cost per contact is not nearly so great.  In fact, one call center in New York that initially appeared to be high cost, was actually lower cost than the average of the peer group after adjusting for cost of living differences.

These types of normalizations or “adjustments” for things like scale and geography are critically important when doing benchmarking.  Without them, the reported performance gaps are simply not valid.  Worse, call centers that do not make these adjustments to their benchmarking data run the risk of taking action based on “false” performance gaps.

The Law of Large Numbers

One of the perennial problems in benchmarking is finding a peer group large enough to produce a meaningful benchmark.  As a veteran of several consultancies, I am convinced that the vast majority of call center benchmarks do not have nearly enough data points to draw any valid conclusions. Obviously the more companies you have in your benchmarking peer group, the better.  The validity of benchmarking data increases geometrically as the peer group grows in size.  With fewer than five companies in a peer group, a benchmark is simply invalid; there is not enough data in such a small peer group to draw any meaningful conclusions.  With a peer group of five to ten, the data becomes more meaningful, but is reliable only for diagnosing large performance gaps.  With a peer group of 10 or more, the benchmarking data begins to gain some statistical validity.  Among other things, with a peer group this size you have an error canceling effect, whereby data errors on the plus side tend to be cancelled out by data errors on the negative side. Although benchmarking syndicates – large groups of call centers that join together for the purpose of benchmarking – are relatively new, MetricNet favors this type of benchmarking consortia because they guarantee a larger peer group than the “one off” benchmarks that are so common in the industry.  Participating in a benchmarking consortium, in turn, greatly improves the statistical validity of your benchmarking results.  Whenever possible, your call center benchmark should be done as part of a benchmarking syndicate or consortium.  This will ensure that your benchmarking peer group is large enough to produce meaningful results.

Conclusion

When it comes to benchmarking peer group selection, there are no hard and fast rules.  Nevertheless, there are some guidelines that will dramatically improve the value of your benchmark, and the validity of your benchmarking results.  Specifically, you should look first for a willingness to benchmark on the part of other call centers.  This is the single most important factor in selecting peers for your benchmark.  Secondly, make sure that your benchmarking peer group handles similar transaction types as your call center.  If the types of transactions handled by the peer group is different than those handled by your call center, the benchmarking comparison will be invalid.  Thirdly, recognize that scale has a significant impact on your costs.  When benchmarking against call centers with different transactions volumes, you can make adjustments for scale based on the 7% rule explained in this article.  Fourth, don’t get hung up on the idea of benchmarking only against other call centers from your industry.  Some of the most valuable benchmarking insights are gained by looking at call centers from outside of your industry.  And finally, keep in mind that the geographic location of a call center will have an impact on labor costs, and hence the cost per call.  These labor cost differences can be normalized as explained in this article. Finally, you should include as many call centers in your benchmarking peer group as possible.  The larger the peer group of valid benchmarking candidates, the more statistical validity you will have in your benchmark.  Participation in large scale benchmarking syndicates of the sort sponsored by MetricNet is the best way to ensure a robust peer group for your benchmark. Due to space limitations, this article barely begins to scratch the surface on the topic of benchmarking peer group selection.  In subsequent articles, MetricNet will continue its series on Successful Benchmarking for the Call Center, with articles on:

• The Benchmarking Performance Gap:  Diagnosing the Causal Factors Behind Your Call Center’s Performance Gaps
• The Cost vs. Quality Tradeoff: How Benchmarking Can Help You Achieve the Right Balance Between Cost and Quality in Your Call Center
• The Benchmarking Payoff: How to Build a Hard-Hitting Action Plan From Your Call Center Benchmark

Stay tuned for next month’s article! Download This Article Forward To A Friend