What’s the biggest danger facing the market today? It might be your largest holding.

Think about past market dangers like the housing bubble or the technology boom. As different as those events were, they had something in common.

That common thread? The fact that nearly all investors got onto one side of the trade.

For tech stocks, it meant a frenzied buying of initial public offerings (IPOs). It didn’t matter if a company needed, on paper, five years to get to profitability. As long as you got in first, you could flip shares later to someone else for big profits. The big tech moves of the 1990’s could have made everyone a day trader, if it had lasted long enough.

In the case of housing, a normal boom became fueled by ultra-low interest rates, and gradually relaxed lending standards. When buying a house, fixing it up, and flipping it in a profit within the space of a few months became more profitable than other jobs, careers changed and priorities shifted.

Where there’s big profits, or at least safe, consistent profits, you’ll find a loaded side of a trade. That’s why I’m skeptical about the bond market—bonds are overpriced and don’t offer enough yield. They’re finally susceptible to some weakness now that the Federal Reserve is raising interest rates.

But that’s not the biggest danger facing investors today. At least bondholders get their principal back. Stock investors aren’t always so lucky. And markets look particularly vulnerable for one reason: there’s a lot of money going into one trade.

That trade? The index.

Think about it. It’s what drives everything. Financial news doesn’t talk about every individual name every day. It leads with the index—the forest, not the trees. And right now, the indices are dropping yellow flags. The biggest is overall valuation.

But what happens when your biweekly paycheck goes into your 401(k)? Chances are it goes into an index fund. And that makes sense. You were sold on the possibility of earning the market’s return with a low fee. There was no risk of seriously underperforming the market. Over time, you’ll do pretty well.

I get it—I’ve interviewed Jack Bogle, creator of the index fund. It makes sense for most individual investors who don’t have the time or resources to research and monitor a portfolio of individual stocks.

But that’s lazy intellectual thinking. Index investing isn’t as riskless as it looks. Most investors don’t even know that indices are weighed by market cap. The bigger the company, the more weight it has in an index fund. That means everyone owning an index has their biggest position in mega-cap Apple (AAPL), whether they want to or not.

What’s more, an employee’s biweekly paycheck flowing into the fund gives it more capital to put to work in the biggest names first. If you also own Apple shares outside an index, you may end up having a far larger concentration of your portfolio in a name than you might necessarily like. In other words, investors might be running less diversified portfolios than they think they have.

That’s not necessarily a problem on the surface. But the best returns over time come from smaller companies striving to make it big, not the big companies themselves. And an index is subject to change. Most of the companies in the original Dow 30 have gone away.

For a larger index like the S&P 500, the bottom 400 names are constantly jockeying for position. Names can, and do, get added and dropped all the time. Getting onto the list is a privilege, and a profitable one if you’re in early.

On average, a stock being added to the S&P 500 Index gains 9 percent, according to NYU professor of finance Jeffrey Wurgler. That means that, in an overvalued market, stocks that are also in an index are likely to be even more overvalued. It’s where money is passively going, whether it’s a good investment decision or not.

In short, index investing is a lot messier than people think. For most individual investors, these aspects get glossed over under the rationale that you’re still getting stocks on the cheap and you won’t significantly underperform the market. That ignores the simple fact that weighting stocks by market cap creates a mega-cap bias.

But to me, the real danger comes back to the fact that index investing has become so popular—it’s a one-sided trade. From its first inception in 1975, index funds now account for over 30 percent of all market trades. That beats out institutional investors, individual investors, and hedge funds.

What happens when stocks fall? Many funds might employ a stop-loss method, but the amount of money in each stock is so great that selling risks creating a huge wave of new selling because there are so few willing to buy on the other side of the trade. Or, in a truly passive index fund, investors have to take their lumps and are told to think about the long-term instead.

I’m not a fan of one-sided trades that get all the attention. It means there are areas that are out of attention. Sure, you could buy beaten-down individual names. Or you could get away from stock-focused ETFs and diversify with real estate investment trusts (REITs). Or you could get into smaller companies that won’t make it to the big-league indices for a long time.

When you’re looking to buy outside a retirement plan or index fund, think different. Get away from the index. That’s why I’m a fan of buying small gold mining companies right now. You won’t see them in a big index. Gold could benefit from today’s political uncertainty. And they’re cheap enough that you can sell covered calls against the position right away to generate some income while waiting for the trade to play out. That’s something I’ve done with small-cap mining company Yamana Gold (AUY). That way, I can make money whether it goes up or not.

At this point in the market rally, avoiding overvaluation is the name of the game. And any stock that’s in an index has a built-in overvaluation to it. Tread lightly.

Andrew Packer is a Senior Financial Editor with Newsmax Media. He currently writes the Insider Hotline investment advisory, serves as investment director for the Financial Braintrust, and writes the monthly newsletter Crisis Point Investor.