How Indexes Can Be Deceiving

One of the most common methods for making comparisons in business is to use division to create ratios. However, these ratio metrics can be implicitly confusing, for division often involves different denominators that make cross-comparisons extremely misleading.

One form of these misleading metrics is the index. Often, advertising agencies will say “this segment over-indexes by 1.6 on X”, of which “X” could be valuing environmental product, reading golf magazines, or having fish as pets. What is not shown, however, is the size of the overall population who actually cares about “X”. For example, if only 5% of the overall population cares about “X”, then an over-index of 1.6 means that 8% of this segment’s population cares about “X”, and 92% of the segment does not care. The use of an index here does not add any particular insight, and in fact hides the real takeaway.

To further complicate the issue, ratios like indexes are difficult to compare for they involve changing denominators. If the ad agency also says “this segment over-indexes by 1.3 on Y”, most people’s intuitive response is to compare 1.6 to 1.3 and conclude that of this particular segment, more people care about “X” than “Y”. However, if 50% of the overall population cares about “Y”, then an over-index of 1.3 means that 65% of this segment also cares for “Y”, which is significantly higher than the 8% who cares for “X”. The use of indexes here is not only misleading but is likely to lead the ad agency’s client to wrong business implications.

An easy solution to create a better index is to switch from doing comparisons in multiples (division) to comparisons in absolute numbers (subtraction). Instead of showing the size of “X” and “Y” as multiples of 1.6 and 1.3, show the preference above overall population as absolute differences of 3% (8% – 5%) and 15% (65% – 50%). The difference between “X” and “Y” now becomes clear and the right conclusions can then be drawn.