E! Online is reporting that the two were spotted together last week in Swift's Nashville hometown dining with the country singer's parents.
Also, photos, said to be of the couple, making the rounds in Nashville were going viral Friday on gossip and social media sites.
The 22-year-old singer has already met Kennedy's family: Swift, an acquaintance of Ethel Kennedy's daughter, Rory Kennedy, asked her for an introduction to the matriarch. That led to Swift's appearance at a showing of Rory's documentary about her mother, "Ethel," at the Sundance Film Festival, The Boston Globe reported.
The Kennedy family isn't confirming whether love's brewing between the two -- Rory Kennedy would only say that Swift is a "great friend of all of ours."
Last month, the pair were seen eating in Westchester's Marcella's Pizza, a popular white-tiled pizzeria in the center of Mount Kisco. Witnesses said Swift and Kennedy ordered a couple of slices of cheese pizza -- New York style, with an ultrathin crust. They also have been seen canoodling in Cape Cod, near the Kennedy family homes in Hyannis Port.
Conor Kennedy, a scion of the storied, yet tragic, political family, has experienced his own anguish. His parents, Robert Kennedy Jr. and Mary Richardson Kennedy, were locked in a bitter divorce and custody battle in the past year. On May 16, his mother hanged herself at her Bedford home.
In early July, Conor Kennedy was named administrator of his mother's estate just days after turning 18. He has three younger siblings.
Swift's publicist did not immediately return a call for comment.