11 Reasons Outsourcing Relationships Fail - Client Does Not Understand Metrics in Vendor Reports

In the first article in this series, I outlined the 11 most common failure modes for IT outsourcing relationships.  These are summarized below for your reference:

• The vendor over-promises, and fails to deliver on their commitments
• The client fails to exercise proper governance over the vendor contract
• The vendor underprices the contract and fails to earn a profit
• The contract fails to align vendor with client goals and objectives
• Vendor reports contain raw data, but rarely include proper diagnosis
• The client does not understand the metrics included in vendor reports
• Both client and vendor view the contract as a zero-sum game
• Vendors spin data and reports to cast themselves in the most favorable light
• Continuous improvement is ill defined or not included in the contract
• Vendors experience extremely high turnover on a client project
• Vendors and/or the client do not adequately train personnel

In this seventh installment of the series, I will address the problem of clients not understanding the metrics in their vendor reports. Understanding the Metrics in Vendor Reports Vendor reports are at the core of the vendor/client relationship. When they are clear, concise and well understood, the vendor/client relationship benefits.  Yet the majority of managed service clients complain that vendor reports are unclear at best, and misleading at worst.  Why is this, and what can be done about it? The starting point for interpreting any vendor report is to understand the metrics in the report.  At the very least, your service provider should report the following metrics on a monthly basis:

• Price per Ticket
• Customer Satisfaction (or net promoter score)
• First Contact Resolution Rate
• Agent Job Satisfaction
• Monthly Agent Turnover
• Average Speed of Answer (voice channel), or Mean Time to Resolve (other channels)

Definitions for each of these metrics can be found here, and a full discussion of these metrics can be found here. Additionally, your service provider should include the performance target and a trend for each of these metrics.  I recommend a 12-month trend, but if your service provider has been with you for less than 12 months the trend will, of course, not include a full 12 months.  A very simple monthly report with a nine-month performance trend might look something like this: Beyond understanding the definitions of each metric contained in your vendor reports, it is crucial to understand the cause-and-effect relationships of these metrics.  These cause-and-effect relationships are displayed in the diagram below.  These relationships are the key to diagnosing your performance.  For the sample reports above, we can reasonably deduce the following:

• The trend in Cost per Ticket is mostly flat. The monthly fluctuations in cost are probably the result of seasonal fluctuations in ticket volume.
• The upward (positive) trend in Customer Satisfaction is most likely the result of the upward trend in First Contact Resolution Rate.
• Analyst Job Satisfaction is increasing, which is always good since Job Satisfaction impacts every other metric in service and support.
• The Average Speed of Answer has decreased dramatically, which could be the result of increasing agent headcount and/or increasing agent occupancy and schedule adherence.

A more comprehensive treatment of the cause-and-effect relationships of these metrics can be found here. As mentioned in the last article in this series, your vendor should be providing a diagnosis along with the monthly reports.  But the best way to ensure that your vendor is performing in a way that meets both their contractual requirements and your customers’ needs is to do your own diagnosis and ask lots of questions. A competent service provider should have no trouble explaining their performance – good and bad – and being held accountable for achieving the performance targets specified in their contract. Some Final Thoughts on Vendor Reporting Vendor reports are designed to inform, educate, and prompt actions that lead to continuous improvement.  This, in turn, requires an understanding of the metrics contained in vendor reports. Fortunately, it does not take a huge amount of effort to master these metrics.  Moreover, once you become proficient with the basic metrics outlined above, you will be in a position to manage your vendor more effectively, and deliver the quality of service that your customers expect and deserve.