What I Think Claude Shannon Meant

I don't actually know much about Claude Shannon. No formal math background, never took a course on information theory, and I mostly know his name because it floats through every conversation about computers, AI and communication. Still, I have the feeling I understand his idea on an instinctive level. Or at least, I think I do.

This piece is a deliberately early moment. I'm writing down what I sense now, before I've read the books. I'll come back in a few months and see how much of it held up.

What I think he said

Information is uncertainty falling away. A message is only worth something if it tells you something you didn't already know. A yes/no question whose answer you already have carries zero information. A question you have no clue about gives you one bit.

On top of that, everything is reducible to those kinds of questions. Text, sound, images, all of it. Underneath every message sits a sequence of binary choices, and the smarter you choose that structure, the less you have to send to convey the same thing.

And there is always noise. A channel distorts whatever you push through it. But there's a mathematical ceiling to how much real information you can still get through that noisy channel, and with the right encoding you can get arbitrarily close to that ceiling without the message falling apart.

That's what I think the core is. Maybe it's too simple, maybe I'm completely off. Which is exactly why this post exists.

Why it grabs me

Building, for me, is often pulling signal out of noise. A product that removes one real irritation. A piece of writing that cuts through clichés. A dashboard that shows the three numbers that matter and keeps the rest out. The idea that there's actual math underneath that, a fundamental law that says where the limit lies, feels good. Not because I need the math to keep going, but because it pins down somewhere that the problem is real.

What I'm going to read

Two books on the stack:

• The Information by James Gleick
• A Mind at Play by Jimmy Soni and Rob Goodman

The first is supposed to be the broad story (from drums to bits), the second a biography of Shannon himself. I'm hoping they give a good mix of the idea and the man.

Update to follow once I've finished both. Then we'll see how much of what I wrote above still stands.