A rare 19th-century postage stamp became the world's most valuable stamp Tuesday when it sold for a whopping $9.5 million at an auction house in New York.

Auction house Sotheby's had valued the tiny one-cent magenta from the colony of British Guiana at $10 million to $20 million. However, bidding for the memorabilia began at $4.5 million, and the piece was sold two minutes later to an anonymous bidder over the phone.

"With the premium the stamp has just sold for approximately $9.5 million, which means it has set a new world record price for a stamp," said Sotheby's vice chairman and the director of special projects David Redden, according to Yahoo! News.

The 1856 British Guiana One-Cent Magenta hasn't been publicly displayed since 1986. It is also the only major stamp not in the British Royal Family's private Royal Philatelic Collection.

Redden said the sale was "a truly great moment for the world of stamp collecting."

"That price will be hard to beat, and likely won't be exceeded unless the British Guiana comes up for sale again in the future," he added.

"You're not going to find anything rarer than this," said Allen Kane, director of the Smithsonian National Postal Museum. "It's a stamp the world of collectors has been dying to see for a long time."

The last owner was John E. du Pont, a convicted murderer and American multimillionaire. Du Pont was an heir to the du Pont chemical fortune. He died in prison in 2010 after being convicted of fatally shooting a 1984 Olympic champion wrestler in 1996. The stamp was sold by his estate.

The record for the most expensive stamp had been held by an 1855 Swedish stamp, which sold for $2.3 million in 1996.

The British Guiana stamp is "the largest shining star in the very distance in the great universe of collecting," Redden told AFP in February.