Home / Guides

Byte Order Mark (BOM): What It Is and How to Remove It

Published 2026-10-22

A byte order mark, usually shortened to BOM, is an invisible character that some programs write at the very start of a text file to say which encoding the file uses. It is harmless in many places and a nuisance in others: it turns the first column of a CSV into an unfindable name, makes some JSON parsers refuse a file, and can appear on screen as the strange string . This guide explains where it comes from and how to remove it.

The BOM in one table

EncodingBOM bytesHow it looks if misread as Windows-1252
UTF-8EF BB BF
UTF-16, little-endianFF FEÿþ
UTF-16, big-endianFE FFþÿ

All three are the same character, U+FEFF, written in different encodings. In UTF-16 the order of the two bytes tells a program whether the file is little-endian or big-endian, which is where the name comes from. UTF-8 has no byte order to signal, so there the mark only says "this file is UTF-8".

Where BOMs come from

Problems a BOM causes

WhereSymptom
CSV filesThe first column header is not found. A script looking for id gets \ufeffid
JSONSome parsers reject the file. The JSON standard says a BOM must not be added, though parsers may choose to ignore one
PHP"Headers already sent" errors, because the BOM is output before the page starts
Shell scriptsThe #! line is not recognised, so the script does not run
Web pages appears at the top of the page when the encoding is declared wrongly
ComparisonsTwo values look the same but do not match, because one starts with an invisible character

When you should keep it

The BOM is not always a mistake. Excel relies on it to open UTF-8 CSV files correctly; without it, accented letters and other non-English characters can appear garbled. If people will open your CSV in Excel by double-clicking it, saving it with a BOM is often the right choice. For files read by programs, websites and APIs, UTF-8 without a BOM is the safer default.

How to check whether a file has one

On the command line

head -c 3 file.csv | xxd

If the output starts with efbb bf, the file begins with a UTF-8 BOM.

In a code editor

Visual Studio Code shows the encoding in the status bar, for example "UTF-8 with BOM". Many other editors show it in the same place or in the save dialog.

In Python

with open("file.csv", "rb") as f:
    print(f.read(3) == b"\xef\xbb\xbf")

How to remove it

Visual Studio Code

Click the encoding in the status bar, choose "Save with Encoding", then "UTF-8".

Notepad on Windows

Choose Save As and pick "UTF-8" rather than "UTF-8 with BOM" in the encoding list.

Command line (GNU sed)

sed -i '1s/^\xEF\xBB\xBF//' file.csv

Python, when reading

Open the file with the encoding utf-8-sig. It reads files with or without a BOM and drops the mark if it is there:

with open("file.csv", encoding="utf-8-sig") as f:
    text = f.read()

Pasted text

The option Remove invisible characters in the AI Text Cleaner removes U+FEFF wherever it appears in the text you paste, and lists it in the summary as a byte order mark. That fixes values copied from the top of a file; to fix the file itself, save it again without a BOM as described above.

A BOM in the middle of text

The same character, U+FEFF, was once also used inside text as a "zero-width no-break space" to stop a line from breaking. That use is deprecated; the word joiner, U+2060, does the job now. When U+FEFF turns up in the middle of a sentence today, it is almost always an accident left over from joining files together. It behaves like the other invisible characters described in our invisible character list.

Quick answers

What does  mean at the start of a file? It is a UTF-8 byte order mark being read with the wrong encoding. Remove the BOM or open the file as UTF-8.

Does UTF-8 need a BOM? No. UTF-8 works without one, and most programs expect it to be absent. Excel is the main exception.

Is the BOM the same as a zero-width space? No. The zero-width space is U+200B; the BOM is U+FEFF. Both are invisible, and both are covered in our guide to the zero-width space.

Pasted text starts with a hidden character? The cleaner removes stray byte order marks along with other invisible characters. Open the AI Text Cleaner

Related guides