How can I tell whether an online comparison uses the same measure on both sides?

Start with the definition behind each label, not just the label itself. Ask: what exactly is being counted, and what counts as one instance? A label such as “users,” “successes,” or “cost” can cover different things in different reports.

Then note the unit and the group included. One figure might be a count of people, while another is a count of visits. One might cover all respondents, while the other includes only people who completed a survey. Similar-sounding names do not establish that the measures match.

A useful first-pass note is: “Side A counts ___; Side B counts ___.” If you cannot fill in both blanks from the post or its linked explanation, mark the comparison as unclear rather than assuming the definitions are identical.

Check the time period, denominator, and method. A monthly total is not the same measure as an annual total. A percentage also depends on its denominator: “20%” of 50 people and “20%” of 5,000 people are the same proportion, but represent different counts. Confirm what group each percentage is a share of.

Look for differences in collection or calculation, too. For example, one result might come from a survey and another from recorded activity. Those may answer different questions even if both are presented as a rate. Also check whether one figure is an average and the other a median, or whether categories were combined differently.

You do not need to decide which method is better to spot a mismatch. The immediate question is whether both sides use the same definition, population, period, unit, denominator, and calculation. If not, say which parts differ.

Suppose a post says, “Service A has a 90% completion rate; Service B has a 75% completion rate.” Treat the numbers as hypothetical until you check the underlying definitions. The percentages alone do not tell you what “completion” means, who was counted, or how the rate was calculated.

Make a comparison note:

“Service A: completion means ___; group counted: ___; period: ___; denominator: ___; method: ___.” “Service B: completion means ___; group counted: ___; period: ___; denominator: ___; method: ___.”

Imagine you find that A counts people who finish an initial step, while B counts people who finish the entire process. That is a definition mismatch. You cannot fairly conclude from these figures alone that A performs better. You could say: “The post compares two completion rates, but the definitions appear different: A counts an initial step and B counts the full process. The figures are not directly comparable on that basis.”

If the post provides no definitions, use a more limited statement: “The post reports 90% and 75%, but does not explain whether the rates use the same definition or denominator.” That accurately describes the gap without guessing what the figures mean.

Separate what you can verify from what remains unknown. Check whether the post identifies where each figure came from and whether that source explains its terms. If it links to separate reports, compare the definitions in those reports rather than relying on the post’s summary.

When key details are absent, do not fill them in from context or treat matching labels as proof of matching methods. You can report the figures with a clear caveat, or leave the comparison out if the missing information would change its meaning. A careful conclusion might be: “I could not confirm that these figures cover the same group and period, so the comparison may not be like for like.”

That does not prove the figures are wrong; it means the information available does not establish that they measure the same thing. Keep your wording proportional to what you checked.