How can I read a UTF-8 text file that may start with a byte-order mark?

A UTF-8 byte-order mark (BOM) can appear at the start of a file. Choose between two opening options based on what you want your text to contain:

1. Use encoding="utf-8" to decode normally. If a BOM is at the start, it remains in the decoded text as U+FEFF.
2. Use encoding="utf-8-sig" to skip an optional UTF-8 BOM at the start.

For a file you want to read without that leading marker, utf-8-sig is often the convenient choice. The two options are shown here with a placeholder path:

with open("[sample-file.txt]", "r", encoding="utf-8-sig") as file:
    text = file.read()

Python’s utf-8-sig handling applies to the signature at the beginning of the data; it is not a general-purpose way to remove U+FEFF everywhere in a document. [S1]

Read the file with ordinary UTF-8, then check whether the resulting string starts with U+FEFF. In Python, write that character as \ufeff:

with open("[sample-file.txt]", "r", encoding="utf-8") as file:
    text = file.read()

has_starting_bom = text.startswith("\ufeff")
print(has_starting_bom)

If the check prints True, the first character in text is the marker. If it prints False, that particular check did not find U+FEFF as the first character. It does not tell you that the file has no marker elsewhere or establish the file’s encoding. [S1]

Yes. Read the file once with each codec and inspect the start of each resulting string. The following is a small comparison; [sample-file.txt] is a placeholder that you should replace with your local file’s path.

path = "[sample-file.txt]"

with open(path, "r", encoding="utf-8") as file:
    ordinary_text = file.read()

with open(path, "r", encoding="utf-8-sig") as file:
    signature_text = file.read()

print(ordinary_text.startswith("\ufeff"))
print(signature_text.startswith("\ufeff"))

For example, if the file has a BOM at its start, the expected comparison is True for the ordinary UTF-8 read and False for the utf-8-sig read. Treat those as expected results for that stated assumption, not as a result measured from your file. The codec skips the optional signature at the start; this comparison does not check for markers at other positions. [S1]

If the ordinary UTF-8 text does not start with U+FEFF, you have not found a BOM at the beginning using this check. That alone does not establish the file’s encoding or confirm that every byte is valid for the encoding you intended. [S1]

Keep encoding="utf-8" if you want ordinary decoding and will handle a possible leading U+FEFF yourself. Keep encoding="utf-8-sig" if an optional starting UTF-8 BOM should be skipped when present. Either way, make the choice explicit in the open() call so the behavior is clear to someone reading your code.