The prolonged economic downturn has become the top worry for most wealthy investors, according to new research from Spectrem Group LLC.

Four out of five millionaire investors participating in the study cited the sluggish economy as a concern in the first quarter of 2012, compared with 70 percent in the same period in 2011. Similarly, 81 percent of mass affluent investors, whose net worth is between $100,000 and $1 million, cited it as a concern in the first quarter, versus 74 percent in 2011's first quarter.

Only ultra-high net worth investors, with at least $5 million in assets, didn't list the slow pace of the recovery as their chief concern. Instead, the national debt, cited by 82 percent of the wealthiest investors, and the contentious political environment, listed at 80 percent, beat out the lackluster economy. Even so, the economy came in third, with 77 percent of ultra-high net worth investors naming it as a concern, up from 69 percent a year ago.

"With the unemployment rate remaining high, the continuing volatility in the stock market and gasoline prices, and a high-stakes presidential election looming, wealthy investors are anxious about the country's outlook," says George H. Walper Jr., president of Lake Forest, Ill.-based Spectrem Group. "You can expect a strong turnout among these voters."

Anxieties among the affluent about maintaining their own financial position and having enough money for retirement also increased from a year ago. In addition, a strong majority worry about tax increases: 70 percent of ultra-high net worth investors; 65 percent of millionaire investors, with net worth of $1 million to $5 million; and 64 percent of mass affluent investors.

But tax concerns aren't translating into major investment changes. Just 40 percent of the wealthiest, 28 percent of millionaires, and 25 percent of mass affluent investors are making investment changes because of possible tax increases.

"Overall, wealthy investors are more confident about their own finances, with roughly half optimistic that their personal financial situation will improve in 2012," Walper notes.

Among wealthy investors expecting their financial situation to grow stronger in 2012 are 55 percent of the ultra-high net worth, 52 percent of millionaires, and 49 percent of the mass affluent.

More insights from Spectrem Group's first-quarter investor reports are available at MillionaireCorner.com.

Spectrem Group conducts monthly surveys with wealthy investors on a variety of topics. In the first quarter, Spectrem surveyed 1,319 mass affluent, 1,303 millionaire, and 482 ultra-high net worth investors, with a focus on investment attitudes and behaviors.

Spectrem Group is a strategic consulting firm specializing in the affluent and retirement markets.

MillionaireCorner.com is a source of information for and about investors.