Why can a character look correct but still fail an exact text match?
Quick answer
A character’s appearance does not prove the underlying text is identical. Two strings can use different code-point sequences, and some combinations that are canonically equivalent can be made consistent through Unicode normalization. But normalization does not make every lookalike character equivalent. Compare the code points, check for extra or invisible characters, and confirm that both sides use the normalization expected by the application. If the text is HTML, also check for a leading U+FEFF BOM; handling can vary by format and application. [S1] [S2]
The question, the details, the next step.
A practical checklist
Keep unchanged copies of both strings before testing.
Compare the underlying code-point sequences and record the first difference. [S1]
Check for extra or invisible characters, including at the string boundaries.
Test the normalization form expected by the application on both strings. [S1]
For HTML, check for a leading U+FEFF BOM; do not assume other formats handle it the same way. [S2]
A few more useful answers
Does normalization make every visually similar character match?
No. It can make canonically equivalent sequences consistent, but visually similar or confusable characters may remain distinct. [S1]
Can I assume the application normalizes text automatically?
No. The cited guidance does not establish that every application normalizes input automatically. Check the application’s expected behavior and compare the actual sequences. [S1]
Should I remove U+FEFF whenever a match fails?
Not automatically. Check whether it is present and consider the text format; the cited BOM guidance concerns HTML and does not establish handling for every format or application. [S2]
Sources
References used for this guide.
- [S1] W3C Character Model: String matching and Unicode normalization w3.org · Checked
- [S2] W3C: The byte-order mark (BOM) in HTML w3.org · Checked
