What Is Unicode? A Plain-English Explanation with Examples
Unicode is a universal list that gives every character in every writing system its own number, called a code point. The letter A is U+0041, the euro sign is U+20AC, and the grinning face emoji is U+1F600. Unicode is why one document can hold English, Arabic and Japanese side by side, and also why a pasted paragraph can hide characters you cannot see. This guide explains what Unicode is, how code points differ from encodings such as UTF-8, and where the invisible characters come from.
What is Unicode?
Computers store numbers, not letters. To store text, everyone has to agree which number stands for which character. Unicode is that agreement. It is maintained by the Unicode Consortium, started in the early 1990s, and has been extended in regular releases since. It now covers well over 150,000 characters across modern and historic scripts, symbols, punctuation and emoji, and it can address 1,114,112 code points in total, from U+0000 to U+10FFFF.
Characters, code points and the U+ notation
A code point is the number Unicode assigns to a character. It is written as U+ followed by at least four hexadecimal digits.
| Character | Name | Code point | Bytes in UTF-8 |
|---|---|---|---|
| A | Latin capital letter A | U+0041 | 1 (41) |
| Γ© | Latin small letter e with acute | U+00E9 | 2 (C3 A9) |
| β¬ | Euro sign | U+20AC | 3 (E2 82 AC) |
| δΈ | CJK ideograph for "world" | U+4E16 | 3 (E4 B8 96) |
| π | Grinning face | U+1F600 | 4 (F0 9F 98 80) |
Every code point also has a name and a set of properties, such as whether it is a letter, a digit, a space or an invisible format character. Software uses those properties to decide how to sort, wrap, search and display text.
Unicode is a list, not a file format
Unicode says that Γ© is U+00E9. It does not say how to store that number in a file. That job belongs to an encoding.
- UTF-8 uses one to four bytes per character, keeps ASCII text identical to ASCII, and is the default for web pages and most files.
- UTF-16 uses two or four bytes and is used inside Windows, Java and JavaScript.
- UTF-32 uses four bytes for every character.
All three carry the same characters. Our guide to UTF-8 vs UTF-16 compares them, and ASCII vs Unicode covers how Unicode grew out of the older 128-character standard. The short version is this: Unicode decides what the characters are, and the encoding decides how they become bytes.
How the code space is organized
The 1,114,112 code points are divided into 17 planes of 65,536 each.
| Plane | Range | What lives there |
|---|---|---|
| 0, Basic Multilingual Plane | U+0000 to U+FFFF | Most scripts in daily use, common punctuation and symbols |
| 1, Supplementary Multilingual Plane | U+10000 to U+1FFFF | Emoji, historic scripts, musical and mathematical symbols |
| 2, Supplementary Ideographic Plane | U+20000 to U+2FFFF | Rare Chinese characters |
| 14, Supplementary Special-purpose Plane | U+E0000 to U+EFFFF | Tag characters and variation selectors |
Most of the plane space is unassigned and reserved for the future. Characters in planes above 0 are why a JavaScript string can report a length of 2 for a single emoji.
What makes a "character" tricky
People think of a character as the thing they see. Unicode has several levels, and they do not always match.
- One visible symbol, several code points. The letter Γ© can be one code point (
U+00E9) or two:eplus a combining accent (U+0301). They look the same and compare as different unless the text is normalized. The Normalize Unicode (NFC) option in the cleaner combines them into the single form. - One emoji, many code points. A family or flag emoji is built from several code points joined by invisible glue, the zero-width joiner.
- Different symbols that look alike. The Cyrillic letter
U+0430looks exactly like the Latin lowercasea(U+0061) and is a different character. Such lookalikes are called homoglyphs.
Why copied text hides invisible characters
Unicode includes characters that are meant to be seen and characters that are meant to steer the software. The second group is the reason pasted text can contain things you cannot find.
| Group | Examples | Purpose |
|---|---|---|
| Zero-width characters | U+200B zero-width space, U+200C non-joiner, U+200D joiner, U+2060 word joiner | Allow or prevent line breaks, control how letters join |
| Direction marks | U+200E, U+200F, U+202A to U+202E | Mix left-to-right and right-to-left text |
| Byte order mark | U+FEFF | Marks the start of a file and its encoding |
| Soft hyphen | U+00AD | Marks where a word may break |
| Special spaces | U+00A0 no-break space, U+2009 thin space, U+3000 ideographic space | Fixed or unbreakable gaps of different widths |
| Tag characters | U+E0000 to U+E007F | Language tags and the details of some flag emoji |
None of these is a mistake. Each has a legitimate job in a typeset document. But web pages, word processors and chat tools write them into text, and copy and paste carries them along. To a human the text looks normal. To a program it is a different string. For the full picture, see our guides to invisible characters and how to show hidden characters.
Where Unicode shows up as a problem
- Curly quotes, long dashes and ellipsis characters replace their ASCII cousins in word processors and chat tools, and break code and data that expect plain characters.
- Search and comparison fail when two strings that look the same use different code points.
- Garbled text appears when a file saved in one encoding is opened in another. The classic sign is
ΓΒ©whereΓ©should be. See the mojibake guide. - Length limits count code points, bytes or UTF-16 units, depending on the system, so one emoji may count as 1, 2 or 4.
How to look at the Unicode of your own text
In the AI Text Cleaner
Paste text into the AI Text Cleaner and open Characters found in your text. It lists invisible characters, special spaces and control codes with their U+ code, name and count. It runs in your browser, and the text is not uploaded. It does not list ordinary visible characters such as curly quotes, though it can straighten them.
In Python
import unicodedata
for ch in "Γ©β¬π":
print(hex(ord(ch)), unicodedata.name(ch))
# 0xe9 LATIN SMALL LETTER E WITH ACUTE
# 0x20ac EURO SIGN
# 0x1f600 GRINNING FACE
In JavaScript
"β¬".codePointAt(0).toString(16) // "20ac"
"π".length // 2, UTF-16 units
[..."π"].length // 1, code points
In an operating system
The Character Map app on Windows and the Character Viewer on a Mac (Ctrl + Cmd + Space) show code points and let you search by name.
Common misunderstandings
- "Unicode is UTF-8." UTF-8 is one way to store Unicode, not Unicode itself.
- "Unicode is 16 bits." That was an early plan. Unicode now needs more than 16 bits for its full range.
- "One character is one byte." That is true only for ASCII in UTF-8.
- "Invisible means empty." An invisible character is still a character. It counts, it is stored, and it can be searched.
Quick answers
What is Unicode in simple terms? A single worldwide list that gives every character a number, so text works the same on every device and in every language.
What does U+0041 mean? It is the code point of the capital letter A. The digits after U+ are hexadecimal.
Is Unicode the same as ASCII? No. ASCII has 128 characters. Unicode includes them as its first 128 code points and adds the rest of the world's writing.
Why does copied text contain characters I cannot see? Unicode includes formatting characters with no visible shape, and apps often write them into text.