Charsets and encoding

From A Technological Wiki
Jump to navigation Jump to search

What is a character?

A character is a symbol that humans understand - like letters (a, ñ), numbers (3), or punctuation (?).

Computers, however, only understand binary numbers (0 or 1).

So, to represent characters in a computer, we need a mapping between characters and numbers.


What is a charset (character set)?

A charset (short for "character set") is basically a table that assigns a unique number (a code point) to each character.

For example, in the ASCII charset:

Character Decimal Code Binary Representation
a 97 01100001
b 98 01100010
A 65 01000001

So when you type the letter "a", the computer actually stores the number 97 in memory.

Here is the full ASCII charset table


What is encoding?

An encoding defines how the numbers (code points) from a charset are stored in bytes.

Think of it like this:

  • Charset = Dictionary (maps letters to numbers)
  • Encoding = Packaging method (decides how to store those numbers in bytes)

Example with "a"

Let’s look at "a" in several encodings:

Encoding How "a" is stored (in bytes) Explanation
ASCII 01100001 → 0x61 Only one byte (97 in decimal). ASCII supports 128 characters.
UTF-8 01100001 → 0x61 Same as ASCII for English letters. UTF-8 is backward compatible with ASCII.
UTF-16 00000000 01100001 → 0x00 0x61 Uses two bytes for basic Latin letters.
UTF-32 00000000 00000000 00000000 01100001 → 0x00 0x00 0x00 0x61 Always uses 4 bytes per character.

So for the same “a”:

Charset Encoding Bytes Stored
ASCII ASCII 0x61
Unicode UTF-8 0x61
Unicode UTF-16 0x00 0x61
Unicode UTF-32 0x00 0x00 0x00 0x61

Why do we need Unicode?

ASCII only covers 128 characters - perfect for English, but not for other languages.

Unicode was created to represent all the world’s characters (like ñ, 你, or 😊).

Unicode assigns a unique number (code point) to every symbol.

For example:

Character Unicode code point Decimal
a U+0061 97
ñ U+00F1 241
😊 U+1F60A 128522

UTF-8, UTF-16, and UTF-32 are different ways of encoding these Unicode code points.