Charles Plosser, president of the Fed's Philadelphia branch, told an audience Tuesday that the Fed's effort to support the economy would likely fall short of its goals.
The speech probably startled some investors who had faith in the Fed's latest plan, said Jack Ablin, chief investment officer at Harris Private Bank. The plan includes buying $40 billion in mortgage bonds each month until the economy improves.
"So many investors have bought into the illusion," he said. "And it was like Plosser pulled up the curtain on the Wizard of Oz."
The Standard & Poor's 500 index lost 1.05 percent to close at 1,441.59. The Nasdaq composite index dropped 1.36 percent to 3,117.73. The Dow Jones industrial average lost 101.37 points to close at 13,457.55.
Earlier this month, stocks enjoyed one of their biggest rallies of the year after Mario Draghi, the president of the European Central Bank, laid out a plan to buy unlimited amounts of government bonds to lower borrowing costs for Europe's debt-burdened countries.