Recent chats
ASKGEEZ Q&A GUIDETechnology & devices

Why can a character look correct but still fail an exact text match?

Quick answer

A character’s appearance does not prove the underlying text is identical. Two strings can use different code-point sequences, and some combinations that are canonically equivalent can be made consistent through Unicode normalization. But normalization does not make every lookalike character equivalent. Compare the code points, check for extra or invisible characters, and confirm that both sides use the normalization expected by the application. If the text is HTML, also check for a leading U+FEFF BOM; handling can vary by format and application. [S1] [S2]

LET’S WORK THROUGH IT

The question, the details, the next step.

Question

Two strings look identical on screen, but an exact comparison says they are different. What should I check first?

Geez

Treat the display as a clue, not proof. The same-looking text can be made from different code-point sequences, and visually similar characters are not necessarily the same character. [S1]

Start by preserving both original strings, then inspect the underlying sequence for each one. Compare them character by character, including the beginning and end; do not rely on a screenshot or a copied visual rendering. If the sequences differ, note the first point where they diverge and whether either has an extra character. The useful question is not only “Do these glyphs look alike?” but “What exact sequence is each side comparing?”

For example, suppose both entries display as “key,” but inspection shows one contains an additional character after the visible text. That extra character may explain why the exact comparison fails, even though the displayed words look the same. Record the difference before changing either string.

Follow-up question

What if I find different sequences that seem to represent the same character?

Geez

Check whether the difference is a canonical representation difference, then find out whether the application expects a particular Unicode normalization form. Unicode normalization can make canonically equivalent sequences consistent for matching, but it does not turn every lookalike into an equivalent character. [S1]

For diagnosis, compare the original sequences first, then compare normalized versions using the same form on both sides. Record which form you tested and where normalization occurs in the workflow. Do not normalize one side arbitrarily and assume that solves the underlying issue: the application’s expected behavior matters. If the sequences remain different after the relevant normalization, investigate other differences rather than treating appearance as evidence that they should match.

Follow-up question

How do I look for invisible characters or a byte-order mark without guessing?

Geez

Inspect the complete sequence, not just the visible middle. Look for characters that render with little or no visible mark, and check whether one string has an extra character at the start or end. A leading U+FEFF is associated with the byte-order mark and can cause unexpected display or software problems if it is not handled correctly. [S2]

If the text is HTML, checking for a leading BOM is a reasonable part of the investigation. The cited guidance is specifically about BOM use in HTML; it does not establish how every other format or application handles U+FEFF. [S2] For another format, record what you observe and consult that format’s or application’s documentation rather than assuming the same behavior.

Follow-up question

Can you give me a practical sequence to follow and a note I can share with a teammate?

Geez

Use this order so you do not alter the evidence before you understand it:

  1. Save the two original strings exactly as received.
  2. Inspect and compare their code-point sequences; note the first difference and any extra characters. [S1]
  3. Check whether the difference is a canonical-equivalence issue, and test the normalization form the application expects on both strings. [S1]
  4. If the text is HTML, check for a leading U+FEFF BOM; do not generalize HTML handling to other formats. [S2]
  5. Retest using the application’s actual comparison path and record the result. The cited guidance does not establish that a particular application normalizes automatically or how it handles every character. [S1] [S2]

Copyable investigation note:

“Expected text: [placeholder]. Compared text: [placeholder]. First differing code point or position: [placeholder]. Extra or invisible character found: [yes/no; details]. Normalization form checked on both sides: [placeholder]. Text format and application: [placeholder]. Leading U+FEFF checked: [yes/no/not applicable]. Result after controlled retest: [placeholder].”

Keep the original strings alongside the note. That makes it easier for someone else to verify what was compared without relying on how the text appeared on screen.

A practical checklist

  • Keep unchanged copies of both strings before testing.

  • Compare the underlying code-point sequences and record the first difference. [S1]

  • Check for extra or invisible characters, including at the string boundaries.

  • Test the normalization form expected by the application on both strings. [S1]

  • For HTML, check for a leading U+FEFF BOM; do not assume other formats handle it the same way. [S2]

A few more useful answers

Does normalization make every visually similar character match?

No. It can make canonically equivalent sequences consistent, but visually similar or confusable characters may remain distinct. [S1]

Can I assume the application normalizes text automatically?

No. The cited guidance does not establish that every application normalizes input automatically. Check the application’s expected behavior and compare the actual sequences. [S1]

Should I remove U+FEFF whenever a match fails?

Not automatically. Check whether it is present and consider the text format; the cited BOM guidance concerns HTML and does not establish handling for every format or application. [S2]

Sources

References used for this guide.

  1. [S1] W3C Character Model: String matching and Unicode normalization
    w3.org · Checked
  2. [S2] W3C: The byte-order mark (BOM) in HTML
    w3.org · Checked

YOUR QUESTION. YOUR DETAILS.

Let’s work through your situation.

Get a tailored answer or help with your next task. Starts at $1, with a quote before you pay.

Ask your own question
← Back to Geez at Work