Why Don’t Professional Investors “Beat the Market”?

My strategy for investing is really simple.

I ignore all of the talk about individual stocks and mutual funds that you’ll find in places like CNBC or in Money. I don’t spend time reading the Wall Street Journal or hunting down the “perfect” investment.

Instead, I just put my money in broad-based index funds that charge low expenses and no fees and walk away from it.

Why? I’m perfectly content to match the stock market and real estate market as close as possible. As the overall real estate market and stock market go up and down over time, so does the value of my investment.

This surprises some people. “Why don’t you try to beat the market?” is a pretty common question that I hear.

Here’s the truth: if it were easy – or even possible – to consistently beat the market, I would do so, but the reality is that no one can consistently and predictably beat any sufficiently large market over an extended period of time. It just doesn’t work.

I’m going to use the stock market here as an example, but what I’m writing about is true for the bond market or the real estate market or any other avenue for investing where no one person has enough capital to corner or manipulate the market. (Yes, there are small markets – some collectibles and other things – where people can corner and manipulate the market, but those markets offer completely different risks (like the entire market dying off), so we’re not going to include them here.)

So, why can’t professional investors – people who spend their lives investing in the stock market – consistently beat the stock market? They have more knowledge and more time than anyone else, so why can’t they do better than average, either?

Here are a few reasons why.

They Do “Beat the Market” But Not in a Way Accessible to Individual Investors

There are some large-scale professional investors who can get a better return for their money than the overall stock market would provide. You’ve probably heard of some of them, like Warren Buffett or Carl Icahn.

For them, buying stocks on the open market is just the first step in their investment process. They tend to buy a lot of stock in a particular company, one that gives them some significant voice or control over the management of the company, and then they manipulate the business of that company to benefit themselves.

They might give the stock a short term boost (so they can sell it and earn a quick turnaround). They might make the company start issuing more dividends (so they can collect lots of money that way). They might make the business sell off assets at a discount to benefit other businesses that the person controls. There are a lot of things that a person can do if they gain controlling power over a large business that are just outside the reach of the typical investor.

By doing this, they might earn a better return than the overall stock market, but they’re not really “beating the market.” Instead, they’re just manipulating the specifics of a business for their own ends.

Don’t get me wrong – I don’t have a problem with this. When companies make the decision to start selling shares of their company, they absolutely run the risk of someone coming in and taking control, and investors have the right to have a say in how a business is run and what kind of financial benefits are paid out.

It’s just a method that’s completely outside of the hands of almost all individual investors. Unless you have a billion dollars on hand to manipulate control of a large company, this pathway isn’t really open to you.

They Can’t Predict the Future

No one can predict the future. No one can predict when an earthquake might hit somewhere and completely devastate a pile of businesses or a whole industry. No one can predict when someone in a company’s R&D department might discover something that shakes the whole industry. Those things simply cannot be predicted.

Because of that, someone might be holding the safest stock in the world but when someone else in that industry innovates, that “safe” stock is going to tank. When a disaster strikes, that “safe” stock is going to tank.

On the other hand, the person who, for some reason, invested in a smaller company because the CTO had a killer presentation might be sitting on the company that innovated, or the company that didn’t have their headquarters and data center on top of a huge earthquake.

If you try to hedge your bets against all of these things, do you know what happens? Eventually, you wind up just matching the market. When you hedge as much as you can against everything that could happen to every company, you eventually wind up with a small investment in everything, which ends up effectively being the same thing as the broad-based index fund described above.

If 1,000 Investors Invest, One Should Randomly “Beat the Market” Ten Years in a Row

But what about stories of investors like Peter Lynch who managed to beat the market fourteen years in a row while managing the Magellan Fund? Isn’t he proof that someone can beat the market consistently?

Not to downplay Lynch’s skills, but a big part of that is sheer luck. Someone has to “win” and it happened to be Lynch.

Let’s assume, just for a moment, that everyone managing a mutual fund is very, very skilled – effectively equal in skill. Some might be a little more skilled than others, but these people are mostly so sharp that they balance each other out.

Given that they’re all so skilled, they’re all going to be effectively driving the market together. If all of the people investing in the stock market are very, very skilled, the stock market is going to essentially match what these guys and gals do as a collective whole.

However, not all of them own the same investments. Some invest in Company A while others invest in Company B and so on. Each investor has their own individual portfolio that’s nothing like the investments of the others.

This would mean that, in a given year, half of the fund managers would beat the market and half of them would not (it’s essentially impossible to perfectly match the market). Some of them will have made good guesses about what happens in a particular year, while some do not.

As I noted above, no one can predict the future. No matter how good you are, unforeseen events are going to sometimes benefit a company and sometimes damage it, and that’s going to affect the value of a stock investment in that company.

So, in a particular year, some set of companies will do better than the market and some set of companies will do worse. Because of that, some of the investors will randomly own more stocks in the good companies and beat the market, while others will randomly own more stocks in the bad companies and fail to beat the market that year.

The next year, who knows? The whole game starts over again. The next year will bring a different set of winners and losers, as everyone changes around their portfolios and different companies have different unexpected events.

So, let’s say we have 1,000 investors. During year one, half of them will beat the market. That leaves 500 winners. During year two, half of those guys and gals will beat the market again, meaning you’ll have 250 people who beat the market in years one and two. After year three, 125 people beat the market all three years. The next year, 63 people beat the market all four years. The next year, 32 people beat the market all five years. After that, 16 people, then 8, then 4, then 2, then just one person beats the market over ten straight years.

That one person still left standing, the one person who beat the market ten years in a row, is going to look like a genius and money and opportunities are going to flock to that fellow. However, as smart as that person is, part of his or her success is just that he or she was able to be the one frog that safely hopped across the interstate without getting hit by a big unexpected event.

Yes, there are some differences in the skills of fund managers, but when you recognize that pretty much everyone running the funds are incredibly skilled and that the market is littered with nonstop random events and that no one can predict the future, you end up with fairly random winners and losers.

They Charge High Fees

If a mutual fund charges a 1% annual fee, that means that for an individual investor who puts money in that fund to be able to beat the market, the fund has to be beating the market by more than 1%.

In other words, it’s not enough for a fund to just barely beat the market – the fund has to beat it by a lot or else individual investors are going to trail behind the market.

Let’s compare that to an index fund, like the Vanguard Total Stock Market Index. This fund simply tries to match the contents of the overall American stock market as close as possible – it essentially is the market.

The admiral shares of that fund have an expense ratio of 0.05%, which is the sum of all of the fees that someone investing in this fund pays to Vanguard each year. If we use that as a baseline, that means that someone with their money in VTSMX doesn’t beat the market, but only trails it by a tiny bit – 0.05% to be exact.

Now, if you have a regular mutual fund that has an expense ratio of 1%, that means in order to match this baseline, it has to be beating the stock market by a substantial amount each year. If the stock market grows by 7%, the mutual fund has to grow by 7.95% just to match the Vanguard Total Stock Market Index.

That’s a tall order. It’s quite easy to match the market, but to beat it by almost a full percentage point is very difficult. In other words, it’s the fees themselves that make it very difficult for a fund to “beat the market” consistently for the people invested in it.

They “Window Dress” by Dumping Short Term Losers to Look Good at the Moment

At this point, we’re going to look at things that mutual fund managers have to do in order to attract and keep clients, which is the lifeblood of the mutual fund industry.

One trick that many funds like to use to make their portfolio look good is that they’ll list their top ten holdings and what those holdings did over the last year.

Usually, that top ten list looks amazing, and that in turn will make you think that this fund is primed to make a mint.

This is actually a trick called “window dressing.” Near the end of a year, funds will often look at their top holdings and sell off everything that didn’t yield at least a somewhat good return. If their fourth biggest holding had a bad year, for example, they’ll sell it off – or at least sell it enough to knock it off their “top holdings” list – and they’ll use that money to buy something that had a great year to get it into their “top holdings” list.

And just like that, their list of “top holdings” looks amazing. Who could argue with a list that contains tons of individual stocks that beat the market last year?

This isn’t fraudulent, but it is misleading, particularly to armchair investors who do a little research but don’t have the time to deeply research investments. They see the holdings of a fund and they look like a bunch of winners, so why not invest?

It also gives the overall appearance as though the fund manager can really pick winners, when in fact that appearance is an illusion.

They Need to Appear to Be Following Trends

If a particular industry or business is seen as being “hot,” many people will want to know if the fund that’s investing their money is investing in this “hot” company or “hot” field regardless of whether or not the fund’s manager actually thinks it is a good idea or not.

If the fund manager can’t state that they’re holding at least some hot stocks or have at least some money in whatever the hot industry is at the moment, that fund manager looks “out of touch” and is very likely to lose some investment dollars from potential customers and even from other customers who might pull their money out and move it elsewhere.

Again, this is all about attracting new investors and keeping investors in the fund. Professional investors who manage funds need to have investors in that fund in order to make money. Without those customers, they don’t earn money from the fees and they don’t have money to invest, either.

They Spend Time Marketing and Selling, Not Just Investing

If you watch CNBC for long, you’ll notice that the network has a constant array of fund managers and analysts on there talking about whatever stock is hot at the moment and giving their viewpoint on that stock, usually done in a “hot take” format where they’re encouraged to have sharp and entertaining opinions.

None of that helps that person be an effective fund manager in any way. Sure, it makes for interesting television, but it’s time and skills spent on being an entertaining presenter on television and offering up “hot takes” on stocks is completely separate from the hard analytical work of being a good fund manager.

The same thing is true whenever you see or read or hear an interview with a fund manager or an article from a fund manager. It’s effectively marketing. They’re not doing anything that actually furthers their investment planning. They’re merely trying to attract new investors to their fund.

Again, that’s not a bad thing – they need to do this. The truth is, though, that professional investors – those who invest other people’s money for a living – do not spend all of their time on investing. They spend time marketing and selling, too.

Final Thoughts

When you look at all of these factors together – the randomness of investments, the high fees, the need for marketing – it’s really surprising that any professional investors beat the market for any period of time. Whether you attribute it to luck or skill (I say it’s a mix), Peter Lynch’s track record is just mind-boggling when you consider all of these obstacles that were in his way.

It’s also why I am far happier investing in a very low cost index fund that just invests in everything automatically than in putting my money into a mutual fund run by a professional investor.

The professional investor isn’t a bad person in any way. He or she just happens to already have several strikes against them, as described above. They have the albatross of high fees around their necks. They have to follow hot trends and do marketing tricks to attract more business. They’re also not hedged perfectly well against the unknown – in fact, they can’t be if they don’t want to be beaten every single time by index funds.

Professional investors and fund managers often have the deck stacked against them from the start, at least from the perspective of individual investors like ourselves who might let those people manage our money. I’d rather just minimize the middlemen, go for low fees and huge diversification, and ride the market myself.

Trent Hamm

Founder & Columnist

Trent Hamm founded The Simple Dollar in 2006 and still writes a daily column on personal finance. He’s the author of three books published by Simon & Schuster and Financial Times Press, has contributed to Business Insider, US News & World Report, Yahoo Finance, and Lifehacker, and his financial advice has been featured in The New York Times, TIME, Forbes, The Guardian, and elsewhere.