Sources Of IT Metric Data

Where can I.T. managers get I.T. metrics data from today?
There are over 40 possible sources, which can be roughly grouped into four categories:

  • Consulting Firms
  • Industry Groups
  • Publications
  • Software Firms

Some sources are expensive, some are cheap. Some are authoritative, some can be in accurate. There are no sources which are Authoritative, Affordable, Accurate, and Comprehensive.

Consulting Firms

Tier 1 Benchmarkers

These are individually commissioned and priced benchmarking projects in which the benchmarker compares the customer's metrics with data from a database built from previous benchmarks. Such projects typically take 8 to 12 weeks. Forrester Research say they can cost up to $200k. There are four main players:

  • Gartner Market leader, 20 year track record. Global reach, though recently closed their Australian benchmarking practice.
  • Compass 20 year track record. Global reach.
  • Maturity. Newer player, founded in 2002 in Germany.
  • Nautilus Another newer player, founded in the US in 2005. Offer innovative Pro-Benchmark tool.

Other Benchmarkers & Consultancies

There are a number of smaller players who carry out benchmarking work within specific IT departments, or in some countries only.

  • Hackett Group Atlanta GA based Hackett have offered some form of benchmarking for many years, though they have not competed vigorously with the larger commercial benchmarkers. Used to offer a "Book Of Numbers", but now seem to offer a no-fee 45 to 60 minute benchmark, resulting in a report comparing your performance against others in their database.
  • Global Information Partners. Small US-based benchmarker associated with IMF (see below)
  • Metri are a European benchmarking company started in 2005
  • Hedra Consulting provide a benchmarking service largely aimed at UK government organisations through IT Perceptions
  • Lexta Consulting in Germany. This page is in German but can be translated via Google
  • IDC Australia have announced a benchmarking service in Australia in 2008
  • Metric's Based Assessments Mainframe and server cost benchmarks, mainly US NE seaboard, experienced ex META benchmarker.
  • David Consulting Application Development/Function Points only
  • QPMG Application Development/Function Points only
  • SPR Application Development/Function Points only, associated with Capers Jones, a leading thinker on software engineering productivity.

Industry Groups

Industry Process Models

Other Industry Bodies

  • AQPC. US based not-for-profit organisation offering high to medium level list of recommended IT metrics.
  • IMF US based CIO trade federation (of which HP is a member!) which includes a benchmarking service for members. Experienced ex-Compass benchmarking experts.
  • SOCITM Society of UK local government IT managers who regularly benchmark on 10-15 metrics
  • SPEC. Industry body providing standard performance ratings of CPU/motherboard configurations
  • TPC. Industry body providing standard performance ratings of servers running (mostly) database workloads.
  • Uptime Institute

Publications

Publications are books or libraries which can be bought for a one-off or a monthly subscription fee.

Industry Analyst Firms

Other Publications

  • Open Source TCO Model. Little used Swiss-run open source TCO software.
  • Computer Economics. Web site selling publications.
  • Jochen Mikels runs a German language web site selling market prices
  • Peter Brooks's book on Metrics For Service Management. Book based on ITIL v2 processes, defining 100+ metrics for ITIL process management, many impractical. No target data published, but see also ITSMF Benchmarking.
  • Implementing Metrics For Service Management by David Smith
  • ISBGS/UKSMA CD-based report on software development project productivity using Function Point counts.
  • ITMPI Sponsored by Application Development outsourcer to document best practices.
  • ITMWeb. IT Managers web resource, not updated since 2003.
  • Janco's IT Productivity Center offer standard job descriptions and US based salary data for various IT roles. One of their products is a Metric's Handiguide (304 pages, last updated August 2007) containing lists of 504 metrics, including graphs, for US$599. It's not clear from the sample how much detail is provided on the definition of the metrics, or any indication of good/bad performance. However an "update" service for $379 offers "12 months of updates".
  • See also David Parmenter's web site and his resources at Better Management
  • Computer Economics book

Software Vendors

Software vendors tend to talk about the metrics their product can deliver, but don't go so far as to provide either definitions or data for these.

The most interesting effort is from Mirror 42 who have also started a free library of 367 IT metrics. They give brief definitions only, little detail. Little or no data. Strong orientation on process metrics.

Other references to IT metrics from software vendors can be found at:

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License