Higher-income people are worried about the economy