Asymmetric Keys
At a house sale on a Saturday afternoon, a book scout who buys for the online bookstore finds a private collection worth making an offer on: a few hundred old books, and a typed inventory that lists each one with the price the collector paid and the collector's home address. The scout wants to email the inventory to the bookstore that night. The collector has asked that nobody else see it.
The scout and the bookstore have never agreed on a key, and tonight they have no private way to set one up. Symmetric encryption would need one. Asymmetric encryption gets around that.
The bookstore's key pair
Long before that Saturday, the bookstore generated a key pair. One key went on the page of its website for scouts and sellers, where anyone can copy it. The other sits in a file on the owner's computer and has never been sent anywhere.
Which key does the scout use to encrypt the inventory?
The scout uses the bookstore's public key. Nothing but the bookstore's private key can undo that encryption, so once the inventory is encrypted, the email can pass through any number of mail servers and only the owner can read it. The scout doesn't even need a key pair of their own for this message. One would be needed only for the bookstore to send something back the same way.
A leaked private key
Suppose the bookstore sold its old office computer without wiping the drive, with the private key file still on it. Whoever ended up with that file could decrypt everything ever sent to the bookstore under that pair, including the collector's inventory if they ever got a copy of that email. Nothing the bookstore did to its own copy could take back the one that left with the computer.
Organizations that guard the most valuable private keys, such as banks and the companies that vouch for websites, keep them in a hardware security module. The device makes keys, stores them, and uses them inside its own tamper-resistant hardware. A key never leaves the device in readable form, so it can't be copied off a drive or sold with an old computer.

DigiNotar, 2011
The scout copied the public key from the bookstore's website. Nothing in the key itself says whose it is, so if an adversary had put their own key on that page, the scout would have encrypted the inventory for the adversary. Browsers face the same question with every secure site, and they answer it with certificates. A certificate authority, a company the browser makers trust, checks that whoever asks for a certificate controls the site's name, then signs a certificate tying that name to the site's public key. The signing is done with the authority's own private key, so anyone holding the authority's public key can check that the certificate came from it and hasn't been altered.
DigiNotar was a Dutch certificate authority. An intruder took control of servers on its network in June 2011, and on July 10 one of its systems issued a certificate good for any name directly under google.com, such as mail.google.com, carrying a public key whose private key the intruder held. DigiNotar's private keys sat in hardware security modules made by nCipher, so nobody could copy them out. The intruder controlled all eight servers that ran DigiNotar's certificate authorities, though. Once an employee has switched a key on with a smartcard and PIN, the module signs whatever those servers send it, and some of DigiNotar's keys were switched on all the time.
In all, 531 fraudulent certificates were issued. DigiNotar found some of them in a routine check on July 19 and revoked them, but it told neither the browser makers nor the public, and the google.com certificate stayed valid. What could the holder of that certificate do with it?
The holder of that certificate could pose as Google to anyone on a network it controlled, and most browsers would show no warning. In August, about 300,000 different internet addresses checked the certificate's status, more than 99 percent of them in Iran. Chrome caught it, because Chrome carried its own list of the few certificate authorities allowed to vouch for Google's sites, and DigiNotar wasn't on it. In late August a user in Iran posted about the warning in a Google help forum. Within days the browser makers removed DigiNotar from their trusted lists. The Dutch government, which used DigiNotar's certificates for its own services, took over its operations, and on September 20, 2011 a court declared the company bankrupt.
The same forged certificate, checked two ways in August 2011.
The bookstore's scouts don't rely on the web page alone. The first time a scout copies the key, the scout calls the owner, who reads out its fingerprint, a short hash of the public key, and the scout checks it against the copy. A key an adversary had swapped onto the page would give a different fingerprint, and the call would catch it before any inventory was sent.
Sources: Fox-IT, "Black Tulip" interim report on the DigiNotar breach (September 5, 2011); Google Online Security Blog, "An update on attempted man-in-the-middle attacks" (August 29, 2011); Mozilla Security Blog, "DigiNotar Removal Follow Up" (September 2, 2011); VASCO Data Security, "VASCO Announces Bankruptcy Filing by DigiNotar B.V." (September 20, 2011); Fox-IT, "Black Tulip: Report of the investigation into the DigiNotar Certificate Authority breach" (August 13, 2012).