Faster deterioration of three parts of the brain was seen in mostly older adults who had poor sleep quality, though not necessarily too little sleep. Sleep difficulties included having trouble falling asleep, waking up during the night or waking up too early.
However, it isn't clear whether poor sleep causes the changes in the brain, whether a shrinking brain causes poor sleep, or whether a bit of both is occurring.
"We spend roughly a third of our lives asleep, and sleep has been proposed to be 'the brain's housekeeper,' serving to restore and repair the brain," said lead researcher Claire Sexton, a postdoctoral research assistant at the University of Oxford in England.
"It follows that if sleep is disrupted, then processes that help restore and repair the brain are interrupted and may be less effective, leading to greater rates of decline in brain volume," she explained.
But it's just as likely, she said, that the deterioration in the brain also contributes to difficulty sleeping.
"It may be that greater rates of decline in brain volumes make it more difficult for a person to get a good night's sleep," said Sexton, adding she suspects the problems run in both directions.
While a visiting research fellow at the University of Oslo, Norway, Sexton and her fellow researchers gave brain scans to 147 Norwegian adults, average age 54, at the study's start and an average of 3.5 years later.
At the time of the second scan, the adults also filled out questionnaires about their sleep quality, including how long and how well they slept, how long it took to fall asleep, how much time in bed was spent actually asleep, how often they woke up, how sleepy they were during the day and whether they used sleeping medications. Participants took an average of 20 minutes to fall asleep and slept an average of seven hours a night, the researchers found.
After making adjustments for differences in the participants' physical activity, weight and blood pressure -- which have been shown to affect sleep quality -- the researchers compared changes in participants' brain scans and reported their findings online Sept. 3 in Neurology.
In those with poor sleep quality, the researchers saw shrinkage in one part of their frontal cortex and some atrophy, or deterioration, throughout three other parts of the brain, including parts involved with reasoning, planning, memory and problem-solving.
The study didn't test participants' thinking skills, so it couldn't prove that poor sleep or brain shrinkage was linked to poor memory or difficulty thinking. However, past research has found links between declining memory and decreases in brain volume.
"We often correlate brain shrinkage with losing brain tissue, and assume that that isn't advantageous as you get older," said Anton Porsteinsson, director of Alzheimer's disease care, research and education at the University of Rochester School of Medicine and Dentistry in New York.
"Sleep disturbance is such a common symptom among the general population, and it often becomes worse as you age," he said. "There is growing data to suggest that sleep disturbance may be a risk factor for poor outcomes in terms of brain cells and other medical issues as well."
The correlation was only with poor quality of sleep, not shorter sleep. The reduced brain size in poor sleepers was seen across all ages, but the correlation was stronger among adults over 60, the study found.
"What this study signals to me is that [good bedtime habits] and good sleep matters," Porsteinsson said. "Whether that has to be natural sleep or whether we can use medications to enhance sleep has not been answered, but it's probably best to improve your natural sleep patterns."
Sexton made several suggestions for those hoping for better sleep. Besides talking to a doctor about sleep problems, she recommended having a bedtime routine and going to bed at the same time each night.
Other tips include removing gadgets such as smartphones and tablets from the bedroom, not checking emails right before sleep, being physically active during the day, avoiding caffeine late in the day and spending time outside in the sunlight each day.
The U.S. National Library of Medicine has more about improving your sleep.