Types Of Metric

This page is a stub which is currently being written.

Major Volumes

Most ways of measuring I.T. begin by counting the work that I.T. does. For example:

  • how many calls does the Help Desk receive each month?
  • how much disk storage is provided for this database?
  • how many person-hours will a project take to deliver?

In METRIPEDIA, Major Volumes are those metrics which are used as denominators to calculate other measures, particularly cost and productivity metrics. For example, Help Desk calls per month is used in metrics such as:

  • Help Desk cost per call
  • Help Desk price per call
  • Help Desk calls resolved per FTE per month

as well as in ratios such as Help Desk calls per user per month.

Because Major Volumes are used as denominators, it is very important that they can be defined and counted in an accurate way. A 10% variation in interpreting a Major Volume will mean that all metrics based on it will very by 10%.

Most of the services defined in the Benchmark Map can be sized using one Major Volume. Some cannot - an alternative approach is needed for those: see Process Maturity below.

Minor Volumes

Minor Volumes are metrics which count some aspect of an I.T. service but which are not likely to be used as a denominator.

An example might be Number of languages spoken by the Help Desk. This is a useful metric which provides important context when understanding a Help Desk's services and staffing. However, it is not widely used in other metrics. For example, a metric such as Help Desk calls per language per month would not tell managers anything very useful about the Help Desk.1 At most, we might find Minor Volumes used in some simple ratios such as Languages spoken per Help Desk agent, on average.

Costs

Financial measures are, by far, the easiest type of metric to work with, simply because money is much easier to quantify than services, processes, or satisfaction.

For each of the services in the Benchmarking Map, a unit cost is defined and explained: for example Cost per Help Desk call. Following ITIL, Metripedia breaks down these unit costs into five different types.

Hardware Costs

Hardware costs are cost of computer hardware such as personal computers, servers, disk storage, tape libraries, network switches and routers, and so on.

Hardware costs will be found in two quite different areas of an I.T. organisation's financial reporting: depreciated assets, and operating leases. This can make them quite hard to compare. For example: Alpha Corporation buys a Model 301 server for $36,000, and the finance department capitalises the purchase and depreciates it on a straight-line basis over three years. This creates a depreciation charge of $1,000 per month, but is not considered part of the operating expenses which the I.T. department are expected to control. Beta Corporation leases the same Model 301 server from a finance company, or perhaps from the financial services arm of the server vendor. Each month they pay $1,050 in lease charges, and at the end of three years the server is returned to the leasing company.2

As a result, comparing an organisation's average hardware cost (eg hardware cost per personal computer, storage hardware costs per Gbyte) does not provide the metric for buying efficiency that it might appear to. It does not take into account the specification of the hardware, the length of time that the computer hardware is operated for, the volume discounts that may have been achieved when buying the hardware, or the point in the technology cycle when the hardware was bought.

An alternative approach is to benchmark the price at which a given item of computer hardware was bought: hardware price per personal computer. Some organisations do this in order to measure the efficiency of their purchasing organisation, or the competitiveness of a hardware supplier to whom they are bound by a long term contract. Commercial benchmarks of this kind are available from IDC, Ideas International, and Context.

Software Costs

Software costs are the cost of buying, renting, or maintaining licences to use software products written and sold by a software supplier. Benchmarking software products costs is fairly easy for smaller, cheaper products, and extremely difficult for larger more expensive products. This is because it is common for software suppliers to bundle several software products together under a single enterprise or site licence in such a way that the price of individual products is not broken out. Comparing your enterprise software licence costs with those of another therefore relies on finding others who bought exactly the same bundle in exactly the same volume - a difficult task. There are commercial consultancies who make a living by advising on this kind of comparison. However, they keep a fairly low profile because they are extremely unpopular with the software suppliers, who feel that they break contractual obligations of confidentiality about the price charged to one customer, in order to help other customers negotiate lower prices.

There are also variations in the way that software licences are charged for which must be taken into account when benchmarking. For example, some software vendors charge a large one-off fee and a smaller regular software maintenance fee which pays for support and software updates. Other vendors charge a smaller one-off fee and a larger software maintenance fee.

Not all software is purchased, off course. Some of the software used by an organisation will have been written by the I.T. personnel at the organisation, or perhaps by contractors or consultants working for them. Generally, the cost of developing software is treated as an operating expense in the year that it is expended, and therefore does not generate further charges to I.T. finances in the years that the software is used. Some organisations seek to level these expenses: see capitalising software costs.

Personnel Costs

Personnel costs cover the cost of employing or hiring the people who work in the I.T. organisation. These are typically found as a recognisable line item in the chart of accounts operated by the finance department for payroll, salaries, or benefits. The cost of personnel may include:

  • the base salary paid to an employee
  • the cost of benefits paid to an employee, such as
    • pension or 401K contributions
    • healthcare
    • bonus payments
    • company car and/or fuel
  • the indirect cost of an employee, such as
    • the cost of office space for the employee to work in
    • the cost of I.T. facilities for the employee
    • travel and subsistence costs for employees who are mobile
    • the cost of training and/or certifying employees.

Additionally, comparing personnel costs should take into account any consultants, contractors, temporary workers, or free-lancers who are paid on a time & materials basis (that is, per day or per hour). For example, imagine that Company A has 10 salaried employees working on the Help Desk, whereas Company B has 4 salaried employees and 8 contractors. They both handle the same number of calls. If you benchmarked only employees, it would appear Company B was more efficient, whereas in fact Company A is.

Personnel costs are driven by two factors: the number of people working, and the cost of those people. This second factor, labour rates, is itself dependent on a number of factors: the cost of living in the city in which the labour is based, the cost of benefits and employer taxation in the country where they are based, the level of competition for labour in that city, the skills of the personnel, the experience of the personnel, the industry sector, and the employer's use of salary and bonus levels to motivate and retain the personnel. Most of these factors are external, and can only be changed by the I.T. organisation by moving the I.T. teams to a cheaper labour location, such as off-shoring.

The number of people working, however, is much more easily controlled by I.T. managers. Improvements in processes and tools should directly lead to productivity improvements (more work done per person), meaning that either fewer people are needed to do a given amount of work, or that more work can be done by the same people. Either way leads directly to improvements in unit costs.

Staffing Levels And Productivity

Simply counting the number of people working (often referred to as the number of heads) is not an accurate approach, however, since some people work part time, people will leave part way through a year, and some new people will join mid-way through a year. In I.T. benchmarking, as in many other business areas, the concept of a Full Time Equivalent (or FTE) is used to count personnel. In summary, 1 person working full time is 1 FTE. Two people job-sharing, working half the week each, is also 1 FTE.

Even counting FTEs does not lead to an entirely accurate count of labour, however. Consider factors such as holidays or vacations (in Germany, 30 days is normal; in the US, only 10), public/state/bank holidays, overtime. Working hours vary (some organisations work 40 hours a week, some work 35, some expect 50 or 60 hours from their workers), as well as internal overheads which consume time (internal meetings, travel between company locations. As a result, 1 FTE at company A can deliver only 70% of the working hours that 1 FTE at company B. Comparing productivity ratios based on FTEs are therefore potentially subject to at least a plus/minus 15% margin of error.

Despite this, FTE counting is the most common technique used in benchmarking I.T. personnel levels. Productivity metrics such as Help Desk calls per FTE per day or Servers per FTE are common. These metrics are frequently defined and calculated the other way round from most other cost metrics, where the Major Volume is the denominator. For example, Servers per FTE measures the same thing as FTEs per server (or FTEs per hundred servers, as it is normally scaled). The difference is that if you increase servers per FTE, you are improving productivity, decreasing the number of FTEs you need, and decreasing FTEs per hundred servers and cost per server per month overall.

FTE counting works well for operational steady-state activities where the work done by a person does not change greatly during the year. It is however, weak at measuring the productivity of people delivering one-off projects. The kind of work done during the project will vary depending on the length and size of the project. For example FTEs will shift from specification work, to design work, to coding, to testing, and then to roll-out and implementation during the course of a project. Counting FTEs would tell you something about your current project portfolio, but little about the productivity of the people. For this reason per hour productivity metrics are usually used instead of FTEs for measuring project
work. (Per day metrics suffer from inaccuracy and variance based on the different number of working hours in the day in different organisations.)

Outsourced Costs

All I.T. organisations buy some services from external service providers. For example, the network circuits which link a Wide Area Network are nearly always provided by a telecommunications carrier company - it is simple not cost-effective for an I.T. organisation to lay it's own cables. Likewise, hardware maintenance for computers is nearly always purchased as a service, either from the hardware supplier (sometimes as warranty) or from a third party maintenance company.

These services are not normally thought of as "outsourcing", but the measurement techniques needed to manage them are exactly the same as for more conventional outsourcing contracts.

When I.T. work is outsourced, then hardware, software, or personnel costs are normally reduced as a result. By outsourcing work, an I.T. organisation no longer needs to hire employees to do this work, and outsourcers often provide their own hardware and software.

It is not possible to 100% outsource the cost of an I.T. organisation, because, even when all work is transferred to an outsourcer then a small retained function will be needed to manage, monitor, and direct the outsourcer. However, outsourced costs can be as high as 95% of total I.T. costs - and therefore the costs of hardware, software, and personnel as a percentage can in some cases be extremely small.

One man's cost is another man's price. The costs of using an I.T. outsourcer is made up of the prices which the outsourcer charges for their services.

Outsourced Prices

Outsourced prices come in hundreds of permutations. There are at least seven basic forms:

  • flat (the outsourcer charges the same "flat" amount each month for the service, no matter how much of the service the customer needs)
  • flat-band (the outsourcer charges the same "flat" amount each month for the service, provided that the customer consumes an amount of the service within a pre-agreed range)
  • base + unit (the outsourcer charges a flat fee which is the same each month regardless of volume, plus an additional amount for each quantity of the service consumed)
  • unit priced (no fixed or flat fees, the outsourcer charges a price for each quantity of the service which is consumed. This is often known as P*Q or Price*Quantity pricing)
    • capacity-based (the unit price is based on the capacity which the customer wishes to have available for use, regardless whether they use it or not)
    • utilisation-based (the unit price is based on the capacity which the customer actually uses)
  • volume breaks (different unit prices are set, depending on the volume or quantity the customer consumes)
    • base (the unit price which the customer will pay if the quantity they consume is within a pre-agreed range. This range is sometimes known as a dead-band or dead-zone.)
    • ARCs (Additional Resource Charges - in other words, a unit price which the customer pays if the price is above the dead-band)
    • RRCs (Reduced Resource Charges - a unit price which the customer pays if the price is below the dead-band range)
  • blended or bundled prices (services which could be counted seperately are bundled together and charged under the same unit price. For example, it is possible to charge seperately for Desktop Moves and for each mailbox, but it is common to include both as part of an overall "per desktop" price.

Some outsourced contracts use three or four pricing mechanisms only. Others use a combination of the methods listed above.

There are at least thirty different pricing units which are commonly used in I.T. organisations. Once different options for hardware configuration, service levels, scheduled annual price changes, multi-currency pricing, and volume breaks are taken into consideration, the number of individual price points in a contract can rapidly multiply. It is not unusual to find outsourcing contracts where the charges are calculated from a complex matrix containing over 2000 different unit prices.

Comparing outsourced prices is easy to do quickly but hard to do well. While there is some industry concensus about common pricing units, there is very little standardisation about how they are used. For example: "per desktop" pricing is sometimes used to price a basic hardware maintenance service which might only cost $5 per desktop per month. On another contract, the price might be as high as $200 per desktop per month, but includes not only hardware maintenance but also an embedded desktop hardware lease, engineering work to design standard software configurations, installs, moves/adds/changes, first level Service Desk, second level desktop support, on-site desktop activities, file servers, printers and print servers, email, backup of servers, software distribution, local area networking, wide area networking, and a VIP service providing instant support to top executives.

Other Costs

In most I.T. organisations, 90% of costs are consumed by some combination of hardware, software, personnel, and outsourcing. There will remain, however, some additional costs.

One of the most significant costs can be the cost of office and building facilities for the I.T. department. The cost of office space for I.T. workers is not significantly different from that for other employees, but the cost of Data Centre space for centralised computers is much greater. Data Centres require

  • high voltage electricity from secure and reliable sources
  • air conditioning to cool the heat produced by the servers and storage
  • raised floors to allow cabling and cooling to reach the servers safely
  • high security levels, so that the servers cannot be interrupted by intruders or attackers
  • high speed network connections

Additional costs are found in consumables, in particular in printing (paper, toner, etc) and in archive storage (tapes and optical disks).

Beyond this most remaining costs in an I.T. organisation are from allocations made to the I.T. cost centre from other departments who support them. ITIL refers to these as "transfer" costs. Examples are:

  • Corporate overheads (for executive management and legal, for example)
  • Finance department cross-charges
  • Personnel or Human Resources department cross-charges
  • Buildings, facilities, or real estate charges

Care must be taken when benchmarking these to avoid recursion. For example, the I.T. department might cross-charge the H.R. department for I.T. support, while the H.R. department cross-charges I.T. for personnel management.

Service Levels

Availability

Responsiveness

Other Service Levels

Service Quality

Process Quality, Process Maturity

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License