Pandit was replaced immediately by Michael Corbat, 52, a Citigroup lifer who had been CEO of its Europe, Middle East and Africa division. Corbat joined the bank in 1983, just after graduating from Harvard.
The Wall Street Journal reported that the departures followed a clash between Pandit, 55, and the company's board over strategy and business performance.
In a conference call late Tuesday with financial analysts and reporters, Corbat and Citigroup chairman Michael O'Neill remained vague about the sudden change.
"What happened is that Vikram submitted his resignation and we accepted it," O'Neill said. Corbat said the changes do not reflect any desire to change the strategic direction at Citigroup, the third largest bank in the United States, with $1.9 trillion in assets.
If Pandit's disagreements with the board were recent, his trouble with shareholders had been brewing far longer. They rejected his 2011 pay package in a nonbinding vote this spring.
Since joining the bank in December 2007, Pandit has made at least $56.4 million, according to data compiled for The Associated Press by Equilar, an executive pay research firm.
Many shareholders were also frustrated by Pandit's failure to boost Citigroup's stock price, which was decimated during the 2008 financial crisis and remains far below where it was when Pandit took over.
Citi's stock is by far the worst-performing major bank stock over the past five years, having lost 91 percent of its value, versus a 6 percent gain for Wells Fargo & Co. and a 2 percent gain for JPMorgan Chase & Co.
During his five-year tenure, Pandit sold businesses, sought and then repaid multiple federal bailouts, and helped right its balance sheet after billions in losses on bad mortgage investments made before he took the helm. It was a sharp departure from his empire-building predecessors.