Reading a text without its key: naming the cipher, peeling off encodings, counting letters, breaking nomenclators, replaying the Enigma breaks of the 1930s and 1940s, checking anagrams.
Unicity Distance
Divides each cipher’s keys, counted from its rules, by the redundancy of the language: Shannon’s unicity distance (1949), past which only one reading fits. The redundancy is his F_N counted over the solvers’ books, or his .7 digits for English. The app’s substitution solver is tried on his sentences and on random letters of each length; his guessing experiment of 1951 is played on them too.
Terms
About this tool
History
Claude Shannon’s “Communication Theory of Secrecy Systems” (1949), declassified from a report of 1945, asked how much ciphertext a cryptanalyst needs before a solution can be trusted. He called that length the unicity distance and found it to be about H(K)/D, the entropy of the key over the redundancy of the language, which he took as .7 decimal digits a letter for English. In 1951 he measured the redundancy itself, from Fletcher Pratt’s tables of letters, pairs and triples, and from a subject who guessed sentences letter by letter.
How it works
Each cipher’s keys are counted from its rules — 26! alphabets, 26^d Vigenère keys, the invertible Hill matrices, the Playfair’s 25! squares less the 25 shifts of each, the Enigma’s rotor orders, positions, rings and plugs — and divided by D. D is F_0 − F_N, Shannon’s F_N counted over the public-domain books the solvers’ statistics come from, or his own .7 digits. The trial solves Shannon’s two sentences under random alphabets, and random letters of the same length, with the app’s simple-substitution solver, four times at each length. The guessing game sets his two sentences with his subject’s guesses as printed and bounds the entropy from the player’s by his equation 17.
How it was checked
The Hill keys are counted one matrix at a time; a Playfair square shifted round enciphers all 600 pairs alike and a transposed one does not; the Enigma’s left ring, moved with its position, enciphers 1,500 letters alike. English’s F_1 and F_2 come within 0.06 bits of Shannon’s table, and F_2 with the space equals his 3.32. His subject’s guesses, read from the page, give his printed tallies: 89 of 129 at once, and 79, 8, 3, 2, 2 and 8. In a seeded trial ten random letters read as English and the 186 letters of his sentences are read whole.
Limits
H(K)/D holds for a cipher that mixes well; substitution and transposition fall short of it, and a short cryptogram leaves the letters it lacks unknown even when its reading is certain. A key chosen from words has far fewer bits than those counted. The counted F_N fall a little short of the true entropy at N = 6 and say nothing of the longer structure Shannon’s guessing reached. His figures for English are given as printed, the ones that disagree with one another included.