New research claims to provide the best evidence to date that the late-day anxiety and agitation sometimes seen in older institutionalized adults, especially those with dementia, has a biological basis in the brain.

The findings could help explain "sundowning," a syndrome in which older adults show high levels of anxiety, agitation, general activity, and delirium in late afternoon and evening, before they would normally go to bed.

"It's a big problem for caregivers. Patients can get aggressive and very disruptive," says Tracy Bedrosian, lead author of the study and a doctoral student in neuroscience at Ohio State University. "There have been a few clinical studies documenting sundowning, but until now, there hasn't been research in animals to see what's going on in the brain to explain this."

The new study found that aged mice showed significantly more activity and more anxiety-like behaviors in the hours before they would normally sleep when compared with middle-aged mice, just like sundowning in humans.

In these aged mice, the researchers found changes in parts of their brain associated with attention, emotions, and arousal, all of which could be associated with the behavior seen in sundowning.

In addition, mice that were genetically engineered to have an Alzheimer's-like disease also showed more anxiety before sleep than did other mice.

"Some people have argued that sundowning could be explained just by a buildup of frustration of older people who couldn't communicate their needs over the course of the day, or by other factors," says Randy Nelson, co-author of the study and professor of neuroscience and psychology at Ohio State.

"But our findings suggest there is a real phenomenon going on here that has a biological basis."

The study will appear in the online early edition of the Proceedings of the National Academy of Sciences.

In the first experiment, researchers compared middle-aged adult mice (7 months old) with aged mice (29 months old) that would resemble humans in their 80s.

Results showed that the aged mice were significantly more active than middle-aged mice in the two to three hours before they would normally go to sleep.

"The middle-aged mice had a distinct pattern of activity, with three peaks of activity during their waking hours," Bedrosian says. "But the aged mice had a flattened rhythm in which they showed the same level of activity throughout their active period."

That means that in the evening, when the middle-aged mice would slow down compared with their peak activity levels, the aged mice kept going.

The mice also were tested for anxiety-like behaviors at two different times during their waking hours. The mice were placed in a maze where they were allowed to explore open areas, which are more anxiety-producing, or hide in enclosed areas.

The middle-aged mice showed consistent levels of anxiety at both times of the day. However, the aged mice showed more anxiety when tested soon before they would have gone to sleep, which is consistent with sundowning, Bedrosian says.

There were also differences in the brains of the aged mice when compared with the middle aged mice. The researchers looked specifically at the cholinergic system, a neurotransmitter system involved in memory and learning, because loss of function in that system is associated with dementia and many of the changes associated with aging.

Findings in aged mice showed greater expression of a certain enzyme—acetylcholinesterase— before sleep than earlier in the day. High levels of this enzyme are associated with anxiety and agitation.

However, in the middle-aged mice, there were no time-of-day differences in the expression of this enzyme.

Nelson noted that drugs used to control levels of acetylcholinesterase are sometimes used on dementia patients, although there has been no research evidence that it actually had an effect on sundowning.

"These drugs were prescribed for other purposes, but it also seemed to calm patients down. Now we have some evidence on why it works," Nelson says.

The researchers also found differences in expression of two other proteins in the brains of the aged mice that are associated with behavioral disturbances.

"All of these results converge to suggest there are changes in the cholinergic systems of aged mice that may be contributing to the anxiety and agitation symptoms that we documented," Bedrosian says.

In another experiment, the researchers used mice that were genetically engineered to develop Alzheimer's-like disease in their brain.

They were compared, at nine months of age to similar wild-type mice of the same age.

The Alzheimer's-like mice showed more anxiety-type behaviors when tested before they would normally sleep than they did when tested earlier in their waking period. That is consistent with sundowning in humans, the researchers say.

However, the wild-type mice showed no differences in anxiety levels based on the time of day they were tested.

Nelson says one of the theories about sundowning is that it is tied to disruptions that often occur in the biological clocks of older people, where their sleep-wake cycles are fragmented.

To test this theory, the researchers also treated the aged mice with melatonin for four weeks in order to help consolidate their circadian rhythms. However, this treatment didn't work to reduce anxiety issues in the mice.

Nelson says melatonin alone might not work because it doesn't deal with the disruptions in the cholinergic system that was identified in this study.

"We need to study whether treating cholinergic dysfunction alone or in combination with melatonin treatment will help deal with sundowning symptoms," he says.