The Dow Jones is at record highs, but that isn't a sign of economic recovery for everyone. Only 17 percent of Hispanics in the United States have any money in the stock market, according to a new poll from the Pew Research Center.

That figure includes retirement accounts like 401Ks that come with many corporate jobs or positions at large companies. While Hispanics aren't alone in being left behind by the stock market, they are the most affected.

Among African Americans, 28 percent have stock market investments, while 55 percent of whites have money invested in the market. Much of the disparity is due to income differences among the groups.

A full 80 percent of households making more than $75,000 per year and 77 percent of people with college degrees have stock market investments, while only 15 percent of households making less than $30,000 a year and 25 percent of people without college degrees do. Hispanics fall disproportionately into the last two groups, as higher education and income are more difficult to access, particularly for immigrants, both legal and undocumented.

But the recovery has passed over most Americans, even many of those with stock portfolios. While the net worth of the 7 percent of the wealthiest Americans rose 28 percent between 2009 and 2011, the net worth of the remaining 97 percent actually fell 4 percent.