A customer sent a testimonial you were proud to publish: "Café Sevilla completely changed how I think about flan." You copied it from the email, pasted it into the testimonial page, hit publish, and watched it appear as "Café Sevilla completely changed how I think about flan."
Your first instinct was that the paste went wrong. It didn't. Nothing was lost and nothing was corrupted; every byte arrived perfectly intact. The trouble came later, when two programs disagreed about how to read those bytes. The disagreement has a name (mojibake), a cause (character encoding), and a fix that takes seconds once you understand the mechanics.
What your computer is actually storing
Computers store numbers, always have. Every character you type is a number in disguise, and an encoding is the shared agreement about which number means which character. That agreement stays invisible until two programs hold different copies of it, which is the entire story of the é above.
Walk through the crime scene. Your customer's "é" left the email as two bytes, 195 and 169 in decimal (C3 and A9 in hexadecimal), because the email was written in UTF-8, today's dominant encoding. Your CMS read those two bytes expecting Latin-1, an older scheme that spends exactly one byte per character. In Latin-1, byte C3 is "Ã" and byte A9 is "©". Two bytes went in, two characters came out, and "é" became "é."
The text did not break. It was mistranslated, one byte at a time, by a program speaking the wrong language. That is why mojibake gremlins arrive in recognizable pairs and triplets: every UTF-8 character beyond the basic set takes two, three, or four bytes, and each misread byte becomes its own wrong character. When you see "é" or "’" in the wild, you are looking at UTF-8 being read as Latin-1 or its close cousin, Windows-1252.
ASCII: the 128-slot empire
To understand why two encodings were ever fighting over your flan testimonial, meet the old one. ASCII, standardized in 1963 out of telegraph codes, fit everything into seven bits: 128 slots holding English letters, digits, punctuation, and a set of control codes (line feed, carriage return, a bell) inherited from teletype machines. For decades it ruled, because for decades computing's center of gravity was English.
But a byte holds eight bits, and those spare 128 slots became a battleground. Every language community filled them differently: Western Europe got Latin-1 with its accented letters, the Soviet bloc used KOI-8 for Cyrillic, Japan worked with Shift-JIS. These were "code pages," and the same byte meant different characters in each one. A document was readable only if you already knew which code page it came from, and the document never said. Email crossing a border in the nineties routinely arrived as soup. ASCII's smallness was its genius, and its smallness was also its ceiling.
Unicode's catalogue and UTF-8's trick
Unicode solved the dictionary problem. Instead of many regional lists, one catalogue assigns every character in every writing system, living or historical, a unique number called a code point. The é above is U+00E9. The Greek letter π is U+03C0. A snowman is U+2603, for the record, and the everyday emoji live up around U+1F600. The catalogue is only numbers, though. Something still has to store those numbers as bytes, and that is where UTF-8 enters.
UTF-8's trick is the reason it won: the first 128 code points, all of ASCII, are stored as one byte each, byte for byte identical to old ASCII. Everything else expands, to two bytes for most accented letters, three for most symbol and CJK characters, four for emoji. The consequence was enormous. Every existing ASCII document became valid UTF-8 overnight without changing a single byte, so English-language systems could adopt it for free. It was designed in 1992 by Ken Thompson and Rob Pike, sketched on a placemat in a New Jersey diner, which is either the most or the least glamorous origin story in computing. Today nearly the entire web declares UTF-8, while UTF-16 survives mainly inside individual systems, Windows internals prominent among them.
When smart quotes attack
Here is where this stops being trivia, because writers live directly in the blast radius. The typographic characters your word processor inserts automatically, the curly quotes, the curly apostrophe, the ellipsis, the em dash, do not exist in ASCII. Each is three bytes of UTF-8, so each becomes a three-gremlin cluster when misread:
| You typed | You see | UTF-8 bytes | Misread as |
|---|---|---|---|
| é | é | C3 A9 | Latin-1 |
| ' (curly apostrophe) | ’ | E2 80 99 | Windows-1252 |
| " (curly opening quote) | “ | E2 80 9C | Windows-1252 |
| … (ellipsis) | … | E2 80 A6 | Windows-1252 |
| — (em dash) | â€" | E2 80 94 | Windows-1252 |
That euro symbol squatting in the middle of every gremlin is the fingerprint of Windows-1252, by the way, because Latin-1 proper has no euro sign at that position.
The pain is specific and real. Spreadsheets open UTF-8 CSV exports and guess the encoding, which is how a mailing list full of Josés becomes a mailing list full of Josés in one double-click. Old forms reject "unusual characters" in people's names, which is how Muñoz gets bounced or quietly becomes Munoz. And when phone keyboards began autocorrecting straight quotes into curly ones, code pasted from a phone stopped compiling, because a curly quote is not a string delimiter. None of this is your text being fragile. It is old software meeting new characters at an unmarked border.
How to stop seeing é
Five rules cover almost everything.
- Save as UTF-8 everywhere you can reach the setting. Your editor's save dialog, your CSV export, your database connection, your CMS import. Recent versions of Windows Notepad even default to UTF-8, which was untrue for most of that program's long life.
- Declare it where text is served. An HTML page should carry
<meta charset="utf-8">near the top of the head. When a page does not declare, browsers guess, and guessing is the origin story of most mojibake. - Inspect every paste that crosses a system boundary. Email to CMS, word processor to spreadsheet, PDF to anywhere. If Ã, Â, or ’ appears, undo, and repair at the source instead of publishing the gremlins.
- Downgrade deliberately when you must. If a pipeline genuinely cannot accept Unicode, swap curly punctuation for straight equivalents first, with a text cleaner or a find-and-replace pass. A planned downgrade ages better than an accidental one.
- Play with the machinery once. Paste a sentence into a text to ASCII converter and watch the numbers appear; run numbers back through an ASCII to text converter and watch them become words again. Encoding stops being scary the moment you have seen the gears.
One diagnostic for your toolkit: if a repair attempt makes the gremlins multiply ("©"), you converted in the wrong direction. Undo, and go the other way. When text breaks, the useful question is never "what did I do wrong" but "which two systems disagreed."
Frequently Asked Questions
Is UTF-8 the same thing as Unicode?
No, and the confusion causes half of these bugs. Unicode is the catalogue, assigning every character a number. UTF-8 is one way of storing those numbers as bytes. UTF-16 and UTF-32 store the same catalogue differently, which is why all of them get called Unicode.
Can mojibake always be reversed?
Usually, because the bytes generally survive intact, and that is exactly what makes re-encoding work. It stops being reversible once the gremlins have been deleted by hand or re-saved through the wrong encoding twice. Fix it once, correctly, at the source.
Should I use smart quotes and em dashes in web writing?
On a page that declares UTF-8, they are safe, and they look better than straight punctuation. The risk sits downstream, in spreadsheets and old systems with strict validation. And if you are going to use an em dash, use it for the right job; our dashes field guide covers the territory.
Why do emoji sometimes show up as empty boxes?
The encoding delivered the character just fine; the font cannot draw it. An empty rectangle, which typesetters call tofu, means the code point arrived and the installed typeface has no glyph for it. It is a font problem wearing an encoding costume.
How can I tell what encoding a file is in?
Often you cannot, because most files never say. A file containing only ASCII-range bytes is indistinguishable from UTF-8, because those are the same bytes. In practice, you paste and look for gremlins, and if they appear, you re-encode and look again.
One setting, checked once
Somewhere in every tool you publish through, there is an encoding setting, and it almost certainly offers UTF-8. Find all of them today: the editor, the export dialog, the database connection, the CMS import. Set them, forget them, and glance at every future paste that crosses a border. The é class of bug is entirely optional, and you just stopped opting in.