The story
The name is the recipe. In Hebrew the alphabet runs from aleph (א) to tav (ת), and the second letter is bet (ב), the second-to-last shin (ש). Aleph swaps with tav, bet with shin: a-t-b-sh, Atbash.
The best-known examples sit in the Book of Jeremiah, whose prophecies are usually dated to around 600 BC, when Babylon was the great power in the region. Twice the text mentions a king or a city called Sheshach (Jeremiah 25:26 and 51:41). No such place is known. Run its letters through the fold, though, and ששך becomes בבל, Babel: Babylon. A few verses on, in 51:1, the phrase Leb Kamai becomes Kasdim, the Chaldeans who ruled Babylon. Most scholars accept these readings.
A secret, or a game?
Why disguise the name, when the same verse, 51:41, goes on to name Babylon openly? Nobody knows. It may have been caution, a scribe's play on words, or something added later by copyists who liked the trick. Ancient Hebrew writers enjoyed letter games of many kinds, and Atbash had relatives: in Albam, for instance, the alphabet is cut in half and the halves are laid side by side instead of folded.
Whatever the reason, Atbash is often counted as the first known substitution cipher, and it is the ancestor of every scheme on the platforms that follow in which one letter simply stands for another.
How it was broken
Atbash has no key at all. Anyone who knows the trick can read it, and anyone who suspects it can test the idea in a minute. Its only protection is that the reader does not think of it. For that reason it is better described as a code word game than as a way of keeping secrets.
And because it swaps letters in pairs, it is its own inverse: folding twice gets you back where you started. The same holds for ROT13, a Caesar shift by half the alphabet, which you will meet a few platforms on.