Some members argued the city of about 500,000 was too small to deal with an event of such magnitude despite staging successful Asian Games in 2006.
Doha also planned to stage the Games in October instead of the usual summer months, due to the soaring desert heat.
"The weather was the main reason we were left off the shortlist," Doha bid official Aneesa Al Hitimi told Reuters.
An IOC official said the decision for the shortlist was unanimous among the 14 Executive Board members and Rogge, despite a 30-minute delay in the announcement that momentarily raised questions about whether there was a consensus.
IOC Communications Director Giselle Davies said Doha's decision to plan for Games in October had been a major drawback.
"The IOC Executive Board unanimously decided not to grant this exception as it conflicts with the international sporting calendar and would therefore be bad for the athletes and for sports fans," Davies said.
"Does this mean Doha or another country from the Middle East could never host the Olympic Games? No, not necessarily, especially given the IOC's desire to see the Games held around the world," she added.
The decision sparked wild celebrations among Chicago bid officials gathered at a downtown Chicago office with their cheers drowning out the other short listed cities.
