The paragraph that looks fine but isn't
You copy three paragraphs out of a PDF and paste them into your manuscript. They read fine. Then the spell checker flags "infor mation," your search for "office" misses every instance on the page, and the CSV import puts half the fields in the wrong column. None of this is visible on screen, because the damage lives in characters you cannot see.
Text that travels between programs picks up stowaways: soft hyphens, ligatures, zero-width spaces, non-breaking spaces, smart quotes, and leftover HTML fragments when you enable that option. The cleaner above strips or normalizes them in a single pass, entirely in your browser. Nothing you paste is uploaded anywhere, which matters when the document is a client contract or an unpublished chapter.
Where dirty text comes from
PDFs are the worst offenders. A PDF stores text as positioned glyphs, not as reflowable paragraphs. When you copy, you get hard line breaks where the page visually wrapped, plus hyphenated words split across lines. Many PDFs also embed soft hyphens (U+00AD), invisible markers that tell the renderer where a word may break; copy the text and they travel along inside your words. Ligatures are the other trap: "fi" and "fl" are often stored as single combined glyphs (U+FB01, U+FB02), so a pasted "office" contains a character that is not f-i, and no search will ever match it.
Word processors add typographic politeness. Word and Google Docs quietly swap your straight quotes for curly ones, turn hyphens into en and em dashes, and insert non-breaking spaces where they think you need help, such as between a number and its unit. Lovely in print. Poison downstream.
Web pages bring markup. Copy from a browser and you may collect HTML fragments, non-breaking spaces sprinkled by the site's CMS, and zero-width characters used for tracking or as anti-scraping traps.
Scraped and exported data mixes all of the above, plus control characters from old databases and byte-order marks (U+FEFF) at the start of UTF-8 files.
The invisible characters that break things
| Character | Code point | Typical source | What it breaks |
|---|---|---|---|
| Zero-width space | U+200B | Web pages, some editors | Search, string comparison, passwords |
| Non-breaking space | U+00A0 | Word, web CMS | CSV columns, scripts expecting normal spaces |
| Soft hyphen | U+00AD | PDFs, ebooks | Spell check, search, word counts |
| fi/fl ligature | U+FB01/FB02 | PDF copy-paste | Any search for words containing fi or fl |
| Byte-order mark | U+FEFF | UTF-8 file exports | Parsers that choke on the first field |
| Control characters | U+0000 to U+001F | Legacy databases, logs | JSON and XML validity, terminal output |
One zero-width space inside an Excel formula or a JavaScript string can cost you an hour of debugging, because everything on screen looks correct. This is the practical reason character encoding guides tell you to fear what you cannot see.
What each option does under the hood
Every toggle is one specific transformation, not magic:
- Strip HTML tags removes anything between angle brackets and decodes entities (
&becomes&, becomes a space). Use it on web pastes; skip it on text that legitimately contains angle brackets, like code or math. - Remove control and invisible characters deletes zero-width spaces, soft hyphens, ligature substitutions, byte-order marks, and control characters, while leaving tabs and line breaks alone.
- Normalize smart quotes converts curly quotes and apostrophes to straight ones. Code editors, CSV files, and legacy systems expect straight quotes; book publishers usually want the curly ones, so know which camp your text is heading to.
- Normalize dashes converts en and em dashes to hyphens. The three dash marks have distinct jobs, covered in our dash field guide; this option exists for systems that accept only one.
- Collapse whitespace replaces runs of spaces and tabs with single spaces. For finer control, including blank-line removal, the dedicated remove extra spaces tool goes deeper.
- Trim lines and remove empty lines strip edge whitespace and delete blank lines, which is most of what pasted lists need.
- Remove URLs and emails strips web addresses and email patterns from text headed for publication.
- Remove punctuation deletes every punctuation mark, leaving raw words. It is built for word-frequency analysis, often paired with the text sorter.
The paste, clean, verify workflow
Cleaning is a three-move habit. Paste the dirty text. Clean with the smallest set of options that fixes it. Verify: search for a word you know should be there (does "office" match now?), check the line count, or paste into the destination and confirm the import behaves. Verification catches whatever your option choices did not cover, because no tool anticipates every source's weirdness.
Keep the uncleaned copy until verification passes. Cleaning is destructive by design: it deletes characters, and if it deletes the wrong ones (punctuation, meaningful tabs in code), the original is the only way back.
Who cleans text, and why
Students and researchers paste constantly from PDFs and library databases. Cleaning at extraction time keeps quotes and citations free of hidden splits that corrupt later searches.
Developers and data people sanitize input before parsing: CSVs with non-breaking spaces, JSON exports with control characters, user submissions with zero-width characters that break equality checks.
Editors normalize pasted material from collaborators so that revision comparisons show real edits instead of invisible character swaps.
Marketers clean scraped keyword lists, then dedupe them with the duplicate line remover.
Limitations, stated plainly
The cleaner works on patterns, not judgment. It cannot know that a tab in a Python file is indentation rather than junk, that a non-breaking space between "10" and "kg" was deliberate typography, or that curly quotes in a finished novel should stay curly. Ligature repair covers the common fi and fl cases, not every font's custom glyph combinations. And if your text is mojibake ("café" showing as "café"), that is an encoding misread, not a dirt problem; the fix belongs at the source, as our encoding article explains. When in doubt, clean less. Every option you skip is a decision you can still reverse.
Frequently Asked Questions
Is my text uploaded anywhere?
No. Every transformation runs in JavaScript inside your browser, and nothing you paste leaves your device. That makes the tool safe for contracts, manuscripts, and personal data.
Should I convert smart quotes to straight ones?
It depends on the destination. Convert for code, CSV files, plain-text systems, and most web forms. Keep curly quotes for prose headed to print or a publisher, where typographic convention expects them. When unsure, find out what the receiving system was built to accept.
Why does copied PDF text have strange line breaks?
PDFs store text by visual line position, not as paragraphs, so every wrapped line becomes a hard break when copied. Clean the invisible characters first, then rejoin lines: the find and replace tool can swap single line breaks for spaces while leaving real paragraph breaks intact.
What is the difference between this and remove extra spaces?
This tool targets characters and normalization: HTML, invisibles, quotes, dashes, with whitespace collapsing as one option among many. Remove extra spaces goes deep on whitespace alone. Use the cleaner for mixed damage and the space tool when spacing is the only problem.
Can cleaning text ever lose something I need?
Yes, which is why the verify step exists. Removing punctuation strips apostrophes from contractions, collapsing whitespace destroys code indentation and aligned tables, and removing empty lines flattens paragraph breaks in plain-text formats. Clean with the fewest options that solve the problem, and keep the original until you have confirmed the result.