How to Clean Text: A Step-by-Step Checklist
To clean text, work in a fixed order: look at what is actually in it, remove invisible characters, replace odd spaces, fix quotes and dashes, tidy spaces and blank lines, strip leftover formatting, then check the result. This guide is a routine you can follow every time, a way of thinking about how to clean text rather than a pile of one-off tricks. It links to a detailed guide for each step, and shows how a text cleaner handles the steps in one go.
How to clean text: the short version
- Find out what the problems are.
- Remove invisible characters.
- Replace unusual spaces with normal ones.
- Fix quotes, dashes and the ellipsis.
- Tidy repeated spaces and blank lines.
- Remove formatting and leftover Markdown.
- Check the result.
The order matters. Invisible characters and odd spaces first, because they make later steps miss things. Spacing last, because the earlier steps can leave extra spaces behind.
Step 1: see what is in the text
Text that looks fine can contain things you cannot see. Before changing anything, check for the usual causes.
| Symptom | Likely cause | Guide |
|---|---|---|
| Search or equality check fails on text that looks identical | Invisible characters or non-breaking spaces | Invisible characters |
| Words will not wrap, or two words stay glued together | Non-breaking spaces or hyphens | Non-breaking space |
Strange symbols such as ’ or é | Wrong encoding | Mojibake |
| Quotes look curved and break code or CSV | Smart quotes | Smart quotes |
| Long dashes everywhere | Pasted AI or word-processor text | Em dash |
| Lines break in the middle of sentences | Hard line breaks from a PDF or email | Remove line breaks |
| Asterisks and hashes around words | Markdown | Remove Markdown |
| Fonts, colours and sizes carried over | Formatting | Remove formatting |
Practical ways to look:
- In Word, press
Ctrl+Shift+8to show spaces, tabs and paragraph marks. In Google Docs, choose View > Show non-printing characters. - In Visual Studio Code, the default settings highlight many invisible Unicode characters in a box.
- Compare the character count before and after pasting into a plain-text box. A count that is higher than the visible text suggests hidden characters. The input and output boxes of the AI Text Cleaner both show a count.
Our guide to showing hidden characters has the full list of apps.
Step 2: remove invisible characters
These are the characters with no width: the zero-width space, zero-width joiner and non-joiner, the word joiner, the byte order mark, the soft hyphen and the left-to-right and right-to-left marks. They arrive with text copied from websites, chat tools, PDFs and some editors.
Delete them before anything else, because they sit inside words and can stop later find-and-replace steps from matching. See zero-width space and byte order mark for the details of the two most common ones.
One exception is worth knowing. The zero-width joiner is part of how some emoji and some scripts are written. Removing it from a family emoji splits the emoji into separate people, and removing it from certain scripts changes how letters join. Clean those texts with care, and see zero-width joiner. The cleaner keeps the joiner and non-joiner where they hold an emoji sequence together.
Step 3: replace unusual spaces
The non-breaking space, thin space, em space and ideographic space all look like gaps but are different characters from the ordinary space. Replace them with a normal space so that searches, comparisons and line wrapping behave. The list is in the guide to Unicode spaces.
Step 4: fix quotes, dashes and the ellipsis
These are typographic characters that look correct to a reader and cause trouble for software.
- Curly quotes break code, JSON and CSV files. Straighten them when the text is going into any of those.
- Long dashes may need to become a hyphen, a comma or a space, depending on where the text goes. Keep them when you are editing prose for readers.
- The ellipsis character counts as one character and does not match three typed periods. See the ellipsis character.
- En dashes in number ranges can be turned into hyphens when a form or spreadsheet cannot accept them. Check em dash vs en dash before you swap them all.
Decide by destination. A blog post can keep its typography. A database field usually should not.
Step 5: tidy spaces and blank lines
With the odd spaces gone, collapse any run of repeated spaces into one, and remove spaces at the ends of lines. Then reduce long gaps of empty lines. The detailed methods are in remove extra spaces and remove empty lines.
If the text is hard-wrapped, with a line break at the end of every visual line, joining the lines is a separate decision. See how to remove line breaks.
Step 6: remove formatting and leftover Markdown
Formatting means fonts, colours, sizes and links. The quickest fix is to paste as plain text, as described in paste without formatting, or to convert the document with the methods in convert to plain text. For symbols such as ** and # that came from a chat tool, see remove Markdown from text.
Step 7: check the result
- Read it once, as a person, to confirm nothing important changed.
- Paste it into the place where it caused trouble, and run the search or comparison again.
- Look at the character count. After cleaning, it should drop slightly if you removed hidden characters.
- Keep the original. Cleaning cannot be reversed from the cleaned copy.
Which options to use for which job
| Destination | Suggested settings in the cleaner |
|---|---|
| Web form, spreadsheet or CSV | Defaults on, including quote straightening and the long-dash replacement |
| Blog post or document that readers will see | Keep invisible characters and odd spaces on. Turn off quote straightening and set long dashes to keep, if you want the typography |
| Text from an AI chat | Defaults, plus Strip Markdown formatting if the answer has asterisks or hashes |
| Code, JSON or configuration | Invisible characters, spaces and quotes on. Leave Markdown off, because it removes backticks and asterisks |
| Text mixing Latin letters with other alphabets | Add Fix lookalike letters if you see words that search cannot find |
What the cleaner does for you
The AI Text Cleaner runs steps two to five in the correct order whenever you paste. By default it removes invisible characters and control codes, replaces unusual spaces, straightens quotes, swaps long dashes and the ellipsis character, collapses repeated spaces, trims line ends, reduces blank lines and trims the text. Strip Markdown formatting, en dash replacement, lookalike letters and Unicode normalisation are optional and off by default. A summary shows how many changes it made. All of it runs in your browser, and the text is not uploaded.
It does not change the wording, fix spelling, convert case, remove HTML tags or delete punctuation. Those jobs need a different tool, and some of them are covered in the guides linked above.
Mistakes to avoid
- Cleaning in the wrong order. Collapsing spaces before replacing non-breaking ones leaves some behind.
- Cleaning everything with the same settings. Code, prose and data want different options.
- Trusting what you see. Clean text and dirty text can look the same on screen.
- Overwriting the source. Work on a copy.
Quick answers
What is the best way to clean text? Paste it into a plain-text tool that removes invisible characters and odd spaces, then check the parts that are specific to your task, such as Markdown or line breaks.
How do I clean text copied from a website? Paste as plain text, then remove invisible characters and non-breaking spaces. Web pages are full of them.
Can a text cleaner damage my text? Only if the options do not suit the content. Straightening quotes and stripping Markdown are the two that change visible characters, so turn them off for text where those characters matter.