Curriculum · Unit 1 · Can a machine think?

Before there were computers, how did people imagine thinking machines?

Long before chatbots, a handful of people worked out the pieces on paper: machines that follow instructions, switches that do logic, a way to measure information, and a game for judging whether a machine thinks. One of them even noticed that English is surprisingly easy to guess one letter at a time, and that is the seed of every language model.

Nib says: Press Next to walk the idea from 1843 to 1956, or Trace it to watch the whole trip. In 1937 you can flip the switches yourself. Then scroll down and play Shannon's guessing game: guess the next letter, and count how many tries it takes.

    Play Shannon's guessing game

    In 1951 Shannon had people guess a hidden sentence one letter at a time and wrote down how many tries each letter took. Your turn. A space counts as a letter, just as it did for Shannon.

    Tip: on a computer you can just type. The small number under each letter is how many tries it took; ✕ means Nib revealed it after three misses.

    The idea in plain words

    The first program was written for a machine that was never built. In 1843 Ada Lovelace translated an article about Charles Babbage's Analytical Engine, a mechanical computer made of gears, and added notes much longer than the article itself. One note lays out, step by step, how the engine could calculate a series of numbers called Bernoulli numbers, which is why it's often called the first published program. She also guessed the engine could work on more than numbers, perhaps even music. But she was firm that it could originate nothing: it would only do what someone knew how to order it to do.

    Almost a century later, a young Claude Shannon noticed in his 1937 master's thesis that the on/off switches in telephone relays behave exactly like true/false logic. Two switches in a row make AND; two side by side make OR. Any logical rule can be wired up out of switches, and every chip in your phone still works this way. In 1948 he went further and showed how to measure information, counting it in bits, the number of yes/no answers you need to pin something down.

    In 1950 Alan Turing asked "Can machines think?" and then swapped the question for one you could actually test: the imitation game. A judge trades typed messages with two hidden players, a person and a machine, and tries to tell which is which. Turing took on Lovelace directly. He called her doubt "Lady Lovelace's objection," pointed out that machines surprised him all the time, and suggested a machine might be built to learn rather than be told everything.

    Shannon's 1951 guessing game is the quiet hero here. Once people had a little context, they guessed most letters on the first or second try, and he estimated English carries only about one bit per letter, far less than the nearly five bits you would need if all 27 symbols were equally likely. Text is full of patterns. A modern language model plays the same game at enormous scale: it is trained to guess the next token, and each guess is a probability, just like a person's hunch.

    By 1955 these threads were ready to be tied together. A proposal by John McCarthy, Marvin Minsky, Nathaniel Rochester and Shannon asked for a summer workshop at Dartmouth College to study how machines could use language, form ideas, improve themselves and be creative. That proposal coined the phrase "artificial intelligence", and the workshop met in the summer of 1956. The field had a name.

    Three things to keep

    Check yourself

    What did Shannon's 1937 thesis connect, and why does it still matter?

    It showed that circuits of on/off switches can carry out true/false (Boolean) logic: switches in series act as AND, switches in parallel act as OR. Every digital chip is built on that idea.

    In Turing's imitation game, what is the judge actually trying to do?

    Trade typed messages with two hidden players, one human and one machine, and decide which is which. If the judge can't reliably tell, Turing suggested we have little reason to deny that the machine is thinking.

    Why does Shannon's guessing game matter for chatbots?

    It showed that English is highly predictable: with some context, most letters are guessed in one or two tries. A language model is a machine trained to make that next-piece guess as well as possible, one token at a time.

    Go deeper