URL Percent Encoding Explained
You know that spaces turn into %20, but what exactly is happening behind the scenes? Percent-encoding is an elegant mathematical solution to a fundamental limitation of early internet protocols.
The ASCII Limitation
When the standards for the World Wide Web were established by Sir Tim Berners-Lee and the IETF, they dictated that URLs must only be written using the US-ASCII character set. This consists of the letters A-Z, numbers 0-9, and a handful of safe punctuation marks.
This presented a problem. How do you link to a file named "résumé.pdf", which contains non-ASCII characters?
The Percent (%) Escape Mechanism
The solution was an escaping mechanism. A reserved character (the percent sign %) was chosen to signal to the browser: "The next two characters are not literal letters, they are a hexadecimal code representing a character."
The Math: Decimal to Hexadecimal
Let's take the exclamation mark (!). In the ASCII table, the exclamation mark is assigned the decimal number 33.
To convert 33 to a two-digit hexadecimal number (base-16):
- 33 divided by 16 is 2, with a remainder of 1.
- Therefore, the hex value is
21.
Add the percent sign, and the encoded value for an exclamation mark is %21.
UTF-8 and Modern Encoding
For characters outside the ASCII table (like Chinese characters or Emojis), the system uses UTF-8. UTF-8 represents these complex characters using multiple bytes. Percent-encoding simply takes each of those bytes and encodes them sequentially. This is why a single Emoji might result in a long string like %F0%9F%9A%80 (Rocket Emoji).