But he's still very much missed.
"I'm not going to lie," the eight-time Pro Bowl cornerback said in a conference call with Washington reporters. "Everywhere I go outside of Denver, the only thing I get: 'I wish you were still with the Redskins.' That's all I get. Even in this city, there's Redskins fans everywhere.
"I know the fan base is bigger, stadium's bigger, a lot more tradition, but I still love being a Bronco."
It was in 2004 that Bailey left the wackiness of Washington for the relative stability of Denver, and he'll return for the first time when the Broncos visit the Redskins on Sunday.
"It was crazy when I left" he said when talking about the Redskins, and the craziness that continues to this day makes it easy to contrast his new franchise with his old one.
"You kind of know what to expect every day," Bailey said. "That's a good thing. When I left there, it was a swinging door. When I got here, it wasn't like that."
Bailey was traded in a rare superstar-for-superstar swap that's a source of debate to this day. With negotiations going nowhere for a new contract, owner Dan Snyder sent the best cornerback in the game and a second-round draft pick to the Broncos for running back Clinton Portis. Portis has since piled up the yards and is within striking distance of the Redskins all-time rushing record, but was he worth both a shutdown cornerback and a high draft pick?
Bailey laughed when asked which team got the better of the trade, then gave a diplomatic answer.
"Honestly," he said, "I think they got what they wanted, and the Broncos got what they wanted. We're both still playing at a high level."
Interestingly, Bailey and Portis haven't met on the field since the trade. Bailey missed the team's meeting in Denver in 2005 with a hamstring injury, and Portis probably won't play Sunday after suffering a concussion in last week's loss to Atlanta.
"It's funny how things happen," Bailey said. "At least he got to play at his old place, and I get to play at mine."
Bailey will also get a chance to see longtime close friend Fred Smoot. It seems ages ago that Bailey and Smoot were supposed to be the tandem that would rule the Redskins secondary for a decade.
Bailey was a first-round pick in 1999, and Smoot came along as a second-rounder in 2001. The Redskins ranked sixth in passing defense in 2001 and fifth in 2002.
"You can start building something special right there," Bailey said, "because good cornerbacks are hard to find and they had two good ones. I didn't understand why we couldn't keep us together."
Smoot left after the 2004 season but returned after two seasons with Minnesota. He's now a bit player in the Redskins defense, getting a handful of plays a game. He was once one of the team's most media-friendly players, but he rarely grants interviews these days and agreed to talk this week only because the subject was his old Broncos buddy.
"I do miss playing with him," Smoot said. "Those were some of my golden years of playing. At the time, I don't think a lot of people realized, me and him had one of the best young tandems out there. I think if it wasn't for Troy Vincent and Bobby Taylor, both of us could have got a little Pro Bowl action then, but we weren't winning at the time and the Eagles were dominating."
But when it came time for Bailey to get a new contract, the usually free-spending Snyder decided this was one time he wasn't going to foot the bill. Bailey worked out a seven-year, $63 million deal with the Broncos, and the trade was on.
"I was all for going back there, but it didn't work out that way," Bailey said. "Financially it did me good because I got what I wanted as far as my contract. I came to a good franchise. That worked out great. I was going to a place that was real stable for a number of years."
While the stability has been nice, the trade hasn't brought Bailey much postseason success. The Redskins and Broncos have each won only one playoff game since the trade. This year, however, Denver is off to a 6-2 start under new coach Josh McDaniels, giving Bailey a chance to rub it in with Smoot, whose Redskins are 2-6 and have lost four straight.
"We're going to talk about what team's up and what team's down," Smoot said. "Right now, he's at me."
NOTE: S LaRon Landry and DT Albert Haynesworth said they have been fined for their roles in the sideline altercation in the second quarter of the loss to the Falcons. Neither revealed the amount. CB DeAngelo Hall is also a candidate for a fine, as well as some Atlanta players and coaches. The NFL typically doesn't comment on fines until Friday.