The search for the man, Raymond Roth, 47, started about 3:30 p.m. Saturday east of the beach's Field 6, where he was last seen, police said.
Roth was reported missing by his wife and son.
"The son said that he swam straight out and he lost sight of him," said Capt. Bruce Marx, commanding officer of the New York State Park Police on Long Island, which has jurisdiction over Jones Beach State Park.
"We did a large-scale search on the beach and in the water and just couldn't locate him," Marx said. Roth was swimming in an area that isn't protected by lifeguards, Marx said.
Karyn Smith, 18, of Freeport, a Jones Beach lifeguard, said the lifeguards joined hands and combed the shoreline for Ross when he first went missing.
It's her first summer as a lifeguard, she said, and this is the first emergency she's witnessed. "It's definitely not something I want to see again," she said.
At Field 6 last night, helicopters and divers continued to search as distraught family members stood vigil. They declined to comment.
By 9:30 p.m., all ambulances had left the scene, but the family was still waiting.
During the initial search, police issued a court summons to a lifeguard on a charge of obstructing government administration as authorities questioned beach staff, Marx said. "He clammed up" when cops questioned him, Marx said, adding: "This guy was telling other lifeguards not to answer."