UTF-8 vs UTF-16: Differences, Sizes and Which to Use
UTF-8 and UTF-16 are two encodings of the same Unicode characters. They can store exactly the same text, but they turn it into bytes differently, which affects file size, compatibility and the kinds of bugs you run into. UTF-8 dominates files and the web; UTF-16 lives inside Windows, Java and JavaScript. This guide compares them and explains how to tell which one a file uses.
UTF-8 vs UTF-16 at a glance
| UTF-8 | UTF-16 | |
|---|---|---|
| Unit size | 8-bit bytes | 16-bit units |
| Bytes per character | 1 to 4 | 2 or 4 |
| ASCII compatible | Yes, ASCII text is valid UTF-8 | No |
| Byte order | Not an issue | Little-endian or big-endian |
| BOM | Optional, usually left out | Common, to show byte order |
| Typical use | Web pages, files, APIs, Linux and macOS | Windows APIs, Java, JavaScript and .NET strings |
How UTF-8 stores characters
UTF-8 uses a variable number of bytes. Characters in the ASCII range take one byte, exactly as in ASCII. Most other alphabets, such as accented Latin letters, Greek, Cyrillic, Hebrew and Arabic, take two bytes. Chinese, Japanese and Korean characters and most other symbols take three. Emoji and other characters outside the Basic Multilingual Plane take four.
The design has two big advantages: English and code stay compact, and any program that handles ASCII can pass UTF-8 text through without damaging it. That is a large part of why the great majority of web pages use UTF-8.
How UTF-16 stores characters
UTF-16 stores most common characters, including Chinese, Japanese and Korean, in a single 16-bit unit, which is two bytes. Characters beyond the first 65,536 code points, including emoji, need two units, called a surrogate pair, for a total of four bytes.
Because each unit is two bytes, the order of those bytes matters. UTF-16LE puts the low byte first and UTF-16BE puts the high byte first. Files often begin with a byte order mark so readers can tell which order was used.
Size comparison
| Text | UTF-8 | UTF-16 |
|---|---|---|
Hello | 5 bytes | 10 bytes |
café | 5 bytes | 8 bytes |
你好 | 6 bytes | 4 bytes |
😀 | 4 bytes | 4 bytes |
For English, code, HTML and most European languages, UTF-8 is smaller. For text that is mostly Chinese, Japanese or Korean, UTF-16 can be smaller, though compression usually narrows the gap.
Where you meet UTF-16 without noticing
- JavaScript string lengths. JavaScript counts UTF-16 units, so
"😀".lengthis 2, and a family emoji built with joiners can report a length of 8 or more. - Windows. Many system functions work in UTF-16 internally.
- Excel's "Unicode Text" export saves UTF-16 little-endian, which surprises programs expecting UTF-8.
- Java and .NET strings are sequences of UTF-16 units.
Signs a file is in the other encoding
- Spaces or strange gaps between every letter in an editor: a UTF-16 file being read as single bytes, with the zero bytes showing up.
ÿþorþÿat the very start: a UTF-16 byte order mark read with the wrong encoding.- Garbled accented letters such as
éforé: UTF-8 read as a Western code page. That pattern is described in our guide to mojibake.
To check from the command line, file name.txt reports the encoding it detects, and head -c 2 name.txt | xxd shows fffe or feff for UTF-16 with a byte order mark.
How to convert between them
iconv -f UTF-16 -t UTF-8 input.txt > output.txt
In Python, read with one encoding and write with the other:
text = open("input.txt", encoding="utf-16").read()
open("output.txt", "w", encoding="utf-8").write(text)
In a code editor such as Visual Studio Code, use "Reopen with Encoding" to read the file correctly, then "Save with Encoding" to save it as UTF-8.
Which one should you use?
For files, web pages, APIs, databases and anything you exchange with other people or programs, use UTF-8. It is the default almost everywhere, it is compact for most text and it avoids byte-order problems. Use UTF-16 when a platform or file format requires it. For the background on how both relate to older encodings, see ASCII vs Unicode.
Quick answers
Can UTF-8 and UTF-16 store the same characters? Yes. Both can represent every Unicode character.
Is UTF-16 faster? Rarely in a way that matters. Choose based on compatibility, not speed.
What is UTF-32? A third encoding that uses four bytes for every character. It is simple but large, so it is mainly used inside programs.