The S&P 500’s Dirty Little Secret
How Quiet Changes in the Methodology Are Fundamentally Transforming the Index
Imagine you live on a little island. On this island, there is an Italian restaurant that you adore. Once a week, you go to this place and order chicken cacciatore. It’s made with the best ingredients and the quality is top-notch. It’s so good that, over many years, more and more customers are attracted to the restaurant as high rises are built on the sandy beaches of your formerly quiet home.
The increased demand for chicken cacciatore, coupled with the limited local agricultural supply of available ingredients, causes a problem for the owners of the restaurant. Scalability being nearly impossible, they decide to quietly tweak the recipe, knowing a majority of customers won’t realize what happened. “It’s such a small change, it won’t matter, right?” they ask themselves. They substitute the heirloom tomatoes with mass produced tomatoes. No problem.
A year or so goes by and they can’t get sufficient amounts of the high quality olive oil they had been using. They drop by a grade. It’s so subtle, it shouldn’t matter.
Another few years go by and they reduce the quality of the red wine. Again, it’s not by much, but it will allow serving a much bigger audience with greater availability from suppliers.
Now, the chicken cacciatore is the most famous meal on the island. People come from all over the world to eat it. How can they keep up with demand? One more compromise is made. They go with mass purchased spices, instead of the higher quality, more expensive spices.
Slowly, surely, little by little, the recipe changes for the sake of profits until not a single ingredient in the dish people are eating today is the same as the dish that made the place famous. At some point, customers begin to think, “Yeah, it’s good”, but the business doesn’t suffer immediately as it is riding on the fame of the former glory and it still remains one of the best restaurants on the island.
That is the S&P 500.
Three Examples of How the S&P 500 Index Methodology Has Been Quietly, But Substantially, Changed Over the Past 13 Years
When someone talks about a stock market “index”, what they are referring to is a list of rules. These rules determine which individual stocks are purchased – all equity portfolios are made up of individual stocks whether you realize it or not; a mutual fund is a legal structure, not an investment – and in what proportion, substituting the composition of the rules for human judgment on a case-by-case basis. This, ideally, leads to less emotional decisions, improving performance.
The original S&P 500, one of the greatest inventions in the history of capitalism, sought to value a basket of individual stocks representing the biggest, most important market capitalization-weighted firms doing business in the United States. It included foreign companies such as Unilever and Shell; empires with incredible profits, gushing massive dividends, and operating histories that spanned multiple centuries rather than mere decades.
In 2002, the people entrusted with the S&P 500 methodology decided to change the rules. For the first time in generations, they unilaterally kicked all foreign companies out of the index, forcing index fund managers to sell the stocks, trigger capital gains taxes in many cases, and deny their fund owners the opportunity to profit from these international titans, many of which had been instrumental in historical S&P 500 returns. This decision generated a lot of activity. And, of course, fees. Now, if an investor wanted foreign-domiciled companies, he or she would have to buy another index fund that focused on off-shore enterprises, the fund company, obviously, paying an additional licensing fee to Standard and Poor’s. (I’m sure that didn’t influence this stupid decision. Surely they’d be above such self-interest at investor expense, right? Right? I mean, this was in the midst of the corruption period when they were all but selling their imprimatur, labeling junk securities as investment grade, helping lead directly to the worst economic catastrophe since the Great Depression. That was probably just coincidence.)
Not satisfied, barely a few years later in 2005, Wall Street went at it again, changing the S&P 500 rules in a profound way that many main street investors won’t understand but that will almost assuredly enrich insiders at their expense. They implemented something known as “float-adjustment”, which changed how index funds that track the S&P 500 are forced to weight individual components. It sounds innocent, boring, and technical, but had it been in place in the past, index funds would have compounded at far lower rates.
A research report from Horizon Kinetics (PDF) uses Microsoft as an illustration. Had these rules been in effect during the 1990’s, what would it have done for index fund investors?
When added to the S&P 500, the share price was about $2.41. At January 1999, when the stock was $44.75, insiders still owned 31% of the shares. By September 1999, near the peak of the “great technology bubble,” when the shares were $47.50 (and when Microsoft’s weight in the S&P 500 exceeded 4%), inside ownership had been reduced to 26%, and by early September 2000, the very threshold of the collapse of that bubble, when the stock was $35, inside ownership had dropped to 19%. Today, 13 years later, the shares are lower than that. Whether by fortune or perception, insiders were dramatically reducing their holdings going into a decade-plus period of decline and stagnation. What did outsiders do? Had the float-adjusted index weighting method been in place at the time, as insiders sold, such that the float increased, the Index rules would have increased the Microsoft weighting, and mutual funds and other index investors would have been buying more—more of what the insiders were selling.
We’ve talked about Microsoft shares; the absolute insane, once-in-a-lifetime earnings multiple applied to what was then the largest technology firm on planet Earth. Under the new S&P 500 rules, index fund investors would have been lining up, handing over their cash so Bill Gates could sell his stock at the height of the tech bubble to them. It’s asinine.
The authors aptly conclude, “It’s not your grandfather’s S&P 500”.
The more cynical among you are likely to think this is an example of collusion; a way for the rich to get richer at the expense of everybody else by modifying the rules, quite literally raiding the retirement money of the masses so they can create a much more liquid and supported market when they begin systematic sales programs of their own holdings.
The more charitable of you are likely to think this is simply a way for the index fund industry to accommodate the staggering amount of money that has been thrown its way; so much money that it is impossible to invest ideally under the old rules given the amount of float available in specific businesses.
But, wait! There’s more! Last year, the S&P methodology was modified again to permit the inclusion of mortgage REITs. These special types of securities don’t actually represent equity ownership. They don’t even represent real estate ownership. Rather, they are bundles of debt portfolios – fixed income investments! Somehow, in the mind of Wall Street, it makes perfect sense to sell investors de facto bonds when they think they are buying stocks.
Had the New S&P 500 Index Rules Been In Place, Historical Index Fund Returns Would Have Been Substantially Lower
The most damning part of all of this is the fact that had the new rules been in place throughout the lifetime of the S&P 500 – the removal of some of the highest-returning, super compounders like Shell and Unilever, the underweighting of high growth companies until the owners sold out, often at stock market peaks, the inclusion of de facto fixed income securities through REIT conduits – the historical compounding rate would have been lower.
The old, mathematically-almost-guaranteed-to-be-higher-returning methodology is gone. Yet, not a single one of the major S&P 500 index funds offers an adequate disclaimer on the page showing the charts and figures with historical total return performance explaining that the product the investor is buying today would have resulted in significantly different results had those same rules been used in the past.
Anywhere other than Wall Street, they’d call this type of behavior fraud.
The Secret of the Index Funds That Nobody But Academics Seem to Care About
One of the most fascinating things in the academic research looking at stock market returns over the past century or more is that John Bogle’s original thesis holds truer than the index funds admit: Turnover matters. In nearly every case, under nearly every valuation scenario, when you stretch the performance period out to 25 years or more, a basket of a given index bought and held on the date it was acquired, with absolutely zero subsequent changes, ended up outperforming the index itself. Activity is frequently the mortal enemy of good returns.
Add in the propensity of smaller capitalization stocks to grow faster relative to their size and you get another truth: The market capitalization-weighted approach to the S&P 500 largely facilitates management companies earning fees rather than giving investors the best chance at good returns for one simple, practical reason. Namely, if all the money in S&P 500 index funds were invested on an equal-weight basis, the index would break because the smaller components don’t have enough market share outstanding to accommodate the money. The market capitalization-weighed approach isn’t used because it’s better, it’s used because it’s able to facilitate higher assets under management and, thus, fees for the money management firms.
When you dive into the historical record and begin looking at the way all of this interacts, the implications become fairly clear. If history is any guide, and the forces, such as reversion to the mean, that made John Bogle’s original insight so profound continue to exert their presence over the coming quarter-century as they have for most of American history, a relatively young, affluent individual with at least $500,000 who wanted to take up index fund investing could almost assuredly beat the index fund itself over the next 25+ years by creating a modified, equal-weight, S&P 500 portfolio of directly held individual stocks, including the foreign companies that were removed. Such an “organic”, if you will, index fund account would offer several significant advantages that serve as the engine of this relative out-performance:
- Lower Cost: It could be run at a lower cost than the sponsored index fund, with operating expenses coming in at or near 0.00% in many years should the investor want to do it himself or herself. Alternatively, should the investor wish to outsource the task and maintenance to a good asset management firm so they didn’t have to deal with the considerable work involved, it likely could be achieved for an investment advisory fee of 0.25% to 0.75% (differences in cost could include preferences such as whether or not dividends were reinvested into the component that distributed the dividend, whether tax-loss harvesting was done by the firm whenever the investor needed to raise cash by selling off equities, whether or not foreign stocks were included in the component list, or whether the portfolio was truly, entirely passive or had some element of annual rebalancing as part of the mandate). For a household with sufficiently high assets, that has a lot of appeal compared the pooled index fund charging 0.05%. To give a personal illustration, I would much rather own the direct index components, equally-weighted, with the handful of foreign blue chips reinserted while paying 0.50% per year than I would the VFINX while paying 0.05% per year because I’m convinced that the mathematical evidence indicates on an after-tax basis, the former should trounce the latter over a 25+ year period even after factoring in costs. It’s not even a question that is close to the line in my mind nor do I believe it should be for anyone who has even a modicum of arithmetic fluency and an understanding of equity markets. Choosing the latter would be a case of being penny-wise, pound foolish; adapting what I believe to be a demonstrably inferior methodology in an attempt save what, in comparison, is an immaterial difference in cost. In either case, even setup could be low-to-no cost because almost any mainline broker should be willing to negotiate discounted or free trades on those 500 initial transactions for that sort of opening deposit.
- Less Risk: It would have a lower risk profile since there would be much greater diversification on a dollar-weighted basis. If, for example, Apple went bust, again, like it did in the 1990’s following its first run-up, the total loss would be 1/500th of assets, not 1/26th of assets as it is under the market capitalization-weighted formula.
- Less Turnover: Unless the investor wanted annual rebalancing, with a total lack of methodology changes, it would require extreme passivity. It would make the sponsored index fund look hyperactive in comparison.
- Greater Tax Efficiency: As Bogle himself admits in his writings, and we have discussed in the past, holding the components in the form of individual stocks rather than through the legal conduit of an index fund (even if you are indexing, still), provides an opportunity for advanced tax harvesting techniques that can provide a decent boost to your returns over long stretches of time. For higher income families, and those with larger net worths, the difference can be especially meaningful.
Most of You Should Promptly Ignore All of This
What should you do with this information? Most of you should promptly ignore it. If the best fund you have available in your 401(k) plan at work is a low-cost S&P 500 index fund, consider making that the cornerstone of your retirement. It is almost assuredly going to outperform most money managers after their fees have been deducted. You’ll still likely get satisfactory returns over the coming decades provided you couple dividend reinvestment with regular, systematic purchases to balance out market booms and busts; a technique known as dollar cost averaging.
Otherwise, I’m once again with John Bogle in preferring the Vanguard Total Stock Market Index as I generally consider it superior to the S&P 500. For a minimum investment of $10,000, you can get the dirt-cheap Admiral class shares, ticker symbol VTSAX, with an expense ratio of 0.05% per annum. It’s not perfect, either, but it’s still very, very good. Throw on top of it something like the Vanguard Total International Stock index fund to get exposure to the entire global corporate sector outside of the United States and you have a solid foundation. This formula, for what it’s worth, is practically identical to the holdings I have in my family’s charitable foundation, which has certain restrictions that make index funds the most ideal choice.
Why write this at all? Over the past six to twelve months, I’ve had a lot of you contact me asking for clarification on my somewhat passing remarks about the fundamental problems I have with the changes in the S&P 500 methodology. More than a few have taken these remarks, erroneously, to mean that I – who have been one of the biggest proponents of index fund investing online for the past 14 years – somehow dislike indexing, which couldn’t be further from the truth. Rather, what I hate is that Wall Street, in its typical fashion, is slowly, surely, little by little, twisting and manipulating, bastardizing and corrupting one of the greatest tools ever devised for small, inexperienced investors (and more than a few wealthy, knowledgeable professionals who have no interest in managing their own finances but want to piggy back off Corporate America as a whole, which only makes sense if you are doing it through a tax shelter and not a taxable account). There is a better way to index, provided not a lot of people adapt it as the liquidity isn’t there to support widespread implementation. The surplus performance over 25 to 50 years should result in significantly more terminal wealth. It is meaningful. That is useful to know.
Now, if you’ll excuse me, I have to go prepare for the inevitable onslaught of inbox messages from people who write me angry emails about positions they assume I hold after only reading the headline, not the actual body of the text. Interestingly, in almost all cases, these fanatical indexers, who treat it like a religion rather than the tool that it is, are almost all reformed gamblers and speculators who lost huge parts of their net worth at some point in their lives. Like now-sober alcoholics who think everyone is tempted by the mere presence of whiskey or gin, they can’t tolerate any deviation from the official line: Put your faith in the index for it will never let you down, don’t even think to examine the assumptions because it will only lead to mistakes. The index is holy, its methodology shall not be questioned. It is beyond reproach, handed down from on high to mere mortals. It’s somewhat ironic this sub-optimal, non-critical approach will lead to better results given their temperament; that correcting the flaw in cognition could make them more likely to attempt to manage their own funds and suffer significant losses.
The only other place you see this sort of fanatical behavior is in Gold Bugs. Years ago, in the midst of the Great Recession when I said I’d much rather have 1.) silver, 2.) ammunition, 3.) food supplies, 4.) tobacco (for trade), and 5.) a working farm rather than gold were civilization to fall apart, you should have seen some of the messages I received. Many of them – oddly, every one to a person was a man, not a woman (there must be a reason for that) – were so offended, you’d think I’d insulted their mother. There’s something about it that attracts a certain obsessive psychology profile that never made sense to me.
Reader Comments (55)
Comments are presented chronologically, with replies indented beneath the comments to which they respond.
Andrew
March 6, 2015
When and how will the gap stop expanding?
What happens?
Joshua Kennon
March 20, 2015
Replying to Andrew
I have no idea. There are a few possibilities. My sneaking suspicion is that long before the major index funds themselves would break, Wall Street will pervert the entire notion by introducing dozens of other index benchmarks (e.g., "The Goldman Sachs Low-Cost Fundamentally Re-Weighted Annually Blue Chip Dividend United Kingdom Index". It word "index fund" attached will make people buy it while diverting assets from elsewhere.
Alternatively, if the methodology itself just gets worse and worse, you could have some extended period of under-performance where people just abandoned ship.
Ray W
April 19, 2015
Replying to Joshua Kennon
the problem isn't so much with the index itself, in fact one could argue that the changes made are highly intelligent and flatten volatility.... the problem is how the (heavily regulated) investment industry is forced to function and the difficulty of investing in an index fund if you don't have enough capital (or the time of day) to construct your own portfolio...
Erich
March 6, 2015
The 401k plan at my employer offers a S&P 500 index fund that has an expense ratio of 0.02% (through Vanguard, it is an index fund only offered to certain institutions). I'm thinking about allocating all of my 401k contributions to this fund but often wondered about the systemic issues with the S&P 500 index itself, such as the issues you addressed in your article. The decision I make with respect to my 401k allocation could have huge repercussions due to differences in the compounding of the fund I select and the one I do not as I am in my early 20's and (hopefully) have a long run ahead of me. Any thoughts?
Todd
March 6, 2015
Replying to Erich
Joshua, With every one , including Buffett calling for the S&P 500 index fund as a one stop shop. This will lead to a pyramid like result where the first ones in make most of the money. People will be buying it not for the equity holding but because everyone else is buying it. S&P 500 is the new hot stock!
Erich
March 6, 2015
Replying to Todd
Perhaps in the short run but the idea is to continue to dollar-cost average into the index over several decades while reinvesting the dividends. The S&P 500 index fund (low-cost) is not a "hot stock" but a tool for long term compounding of capital for small investors.
Todd
March 6, 2015
Replying to Erich
Erich, I also have about 20% of my total retirement money in the S&P 500 index fund but only as a hedge against myself. The rest is in American Funds, Dodge and Cox, Tweedy Browne Global and looking to starting up a Roth with Mars and Power Growth fund.
All are Value funds. Each one is like a leg on a stool if one breaks I shouldn't fall.
Todd
March 6, 2015
Replying to Todd
Erich, All 4 have beating the S&P 500 longterm.
purerich gold
July 17, 2017
Replying to Todd
We are writing to you this time to advertise the availability of our
main products, alluvial gold and diamonds in bulk quantities.
We have Gold Dust,Gold Bar ,scrap metals, scrap
coppers, aluminum, used rail, scrap batteries, HMS 1/2 & Rough Diamond
in stock for sale looking
for buyer worldwide to do long term business relationship with our
firm. As we
represent a large number of local surface mining, we are prepared to
provide quantities up to 400 kilograms of 22+.karat alluvial gold
monthly
Price $25,000 per kilo
PURITY...........97.85%
QUANTITY ... . 200 KILOS
(1st DELIVERY ORIGIN ... ..GHANA DELIVERY WITHING 5 WORKING DAY UPON
RECEIPT OF SHIPPING FUND
Name PURE RICH GOLD MINING CO.LTD
Email [email protected]
lnt90
March 8, 2015
Replying to Erich
Just my opinion, not the word of god lol - disclaimer check.
The S&P 500 is PERFECT for most people and small discrepancies such as described above will not matter long term for YOUR purposes. All else being equal and Russia does not invade us or some crazy thing, by dollar cost averaging and reinvesting the dividends you'll get the average return of the S&P 500 long run and a significant boost from the dividend reinvestment.
Even with the S&P 500 it never hurts to make sure you know what you own and be critical of it. But also understand that in my experience there is no such thing as the PERFECT financial tool there will always be trade offs, but the S&P 500 is the perfect balance for most people because the advantages for most people outweigh the downside. No individual company wipeout risk so you don't have to worry about a Radioshack, Lehman Brothers, Bethlehem Steel or Eastman Kodak setting you back. SUPER SUPER low expense ratio which is very important (Vanguard is the best at this, though there are other places) and this combined with reinvestment will be sure to have great success over time.
True while you can get superior (or inferior) returns going out on your own or in other investment areas you must be willing to dedicate some time to doing so - the S&P 500 does not require this, just keep throwing fuel into the fire and the magic of compounding and time will take care of the rest. Wish you many Happy Returns.
P.S. - even though D.C.A. will work great over long time periods - try to avoid paying super RIDICULOUSLY high Valuations on the S&P (The Dot COM Bubble type values) and double down when its super low like in 09. Bottom line - Minus paying like 50 PE's for the S&P - no matter if another 2009 happens or whatever just keep a level head and keep throwing fuel into the fire.
Joshua Kennon
March 20, 2015
Replying to lnt90
This is completely off subject but ...
... it's funny you say this because I've been looking over equity results for various European markets over the past 100-200 years from a report by Credit Suisse and it's just crazy how otherwise fine economies were decimated, investors totally wiped out due to invasion. It's still probably rational to invest under those circumstances if you can't flee in the middle of the night (if you are saved, you should make a ton of money, if you aren't, the money wasn't worth much, anyway, as even your land might be taken by the conquering forces), but look at a nation like Austria. Just total obliteration of equity investors.
Joshua Kennon
March 20, 2015
Replying to Erich
I can't tell you what to do but I'll tell you what I would say to my own family members in a similar situation: I don't think the systematic issues, at present, are sufficient to overcome the other advantages a low-cost S&P 500 index fund is likely to offer given that the probabilities are extremely high it is the best thing available in your 401(k) plan if you have a long horizon (which you should). If you like, you can always augment the methodology by making a few manual corrections yourself; e.g. buying a few shares of British dividend stocks such as Unilever outright as a sort of augmentation to it.
My concern is that I can easily see this whole indexing obsession falling into the same pattern that has played out countless times before with everything from securitization to derivatives as encapsulated by that old, famous proverb, "What wise men do in the beginning, fools do in the end."
innerscorecard
March 22, 2015
Replying to Joshua Kennon
"My concern is that I can easily see this whole indexing obsession falling into the same pattern that has played out countless times before with everything from securitization to derivatives as encapsulated by that old, famous proverb, "What wise men do in the beginning, fools do in the end.""
This is exactly my strong sense based on my "contextual inquiry" of individual investors from monitoring and participation of many forums, as well as conversations in real life. People don't understand what they are getting into - and it's a strategy that by definition cannot work for everyone. As you have said, some of the carrying capacity is already starting to be filled, with some changes to what made the strategy so robust in its earlier years.
Of course, in its current form, it is still the optimal strategy for the average person who have to choose something. We're obviously very far from the breaking point, yet.
asdfdsaf
March 6, 2015
Well something like gold, or a basket of foreign held stock and currency is useful in the case of a country collapse, revolution or fiscal confiscation. These kinds of events are more possible, but don't necessarily require a farm and silver.
In places like asia and such, gold is very popular. A big part of it comes from the fact people don't really have trustable banks, and when shit hits the fan somewhat you can wear your gold under your shirt and run.
LordSquidworth
March 8, 2015
Replying to asdfdsaf
Well good thing we don't live in Afghanistan then.
A farm likely has a better return than gold.
Joshua Kennon
March 20, 2015
Replying to asdfdsaf
Gold could be hugely problematic in terms of transacting day-to-day exchanges in a post-apocalyptic economy. Imagine a return to a barter economy and all you have are 1 Troy ounce coins, which seem to be the most popular weight. It's not very useful if you want to buy a couple of chickens and some corn. You could, in theory, shave off parts or cut off portions but then people aren't going to trust it as much.
No, seriously. You need a loaf of bread. The world has fallen apart. Your kids are starving. All you have are larger gold coins and bars. What are you going to do? Hope the other guy has silver to break change.
innerscorecard
March 7, 2015
Post of the year.
I mean it. I like the "fluffier" stuff on here too (pictures, food, videos), but I always jump for joy when I see a hard-hitting piece of incisive analysis like this from you, Joshua (often it comes in the comments rather than the site, so I have to occasionally monitor that instead of just relying on my RSS feed).
Sadly, this will surely never be read by the people that need to read it the most, as it resides on your personal blog with zero promotion (really digging the new link widget on your sidebar to your About.com articles, by the way).
A few points:
1. It does seem like the iron law of finance (or life) is that nothing works all the time forever. I guess it was not to be that "Buy the S&P 500 index" was going to be the one solution that you could tell all your relatives, and they could just do without thinking. As you say, returns over a long enough time period will still be satisfactory, but I think they will likely be very different from expectations (which are naturally formed from charts of past performance).
2. I really like the "ship of Theseus" analogy. I suppose you might apply that to an investment in a company itself (or any time you take ownership of something through some kind of structure which changes).
3. I've been thinking more recently about the wackiness of how companies that don't "screen well" (don't pay a dividend, have economic earnings far more than reported GAAP earnings that show up in screeners, or have some other weird corporate issues) will be less owned by individual investors who are supposedly trying to buy the whole economy, just because of how indexing works. ESPECIALLY in the case of "fundamental" indexes that are weighted on things like dividends or book value, to try to remedy the problems of market-cap weighted indexes. It's such an odd quirk that really shows that the market is structurally not efficient.
It appears the report of the death of actual security analysis (in favor of "factor tilts" and the like) was premature.
Joshua Kennon
March 20, 2015
Replying to innerscorecard
I think this is the key ...
If someone doesn't know why the S&P 500 works so well, and the mechanisms that cause it to work so well keep getting modified, you could be in for a lot of disappointment. I was influenced heavily by the dot-com boom in the late 1990's when I watched people dump money into the S&P and Nasdaq with these absolutely insane valuations, wondering what on Earth was wrong with them when the overvaluation was concentrated in tech and mega cap blue chips, with all of these other, perfectly normal businesses being reasonable priced. I really don't think a lot of people have any idea what that index fund they own actually is.
I saw someone on a forum the other day argue against investing in individual stocks because something along the lines of "one of them might go bankrupt!". I just wanted to shake my head because that is the entire reason low-cost, passive index funds work in the first place. They genuinely, honestly have no idea that they did own Kodak went it went bust because it's hidden behind this veil. They've mistaken the form - the basket through which their individual stocks are held - with the investment itself.
As for the not screening well, I hear you. It's been fascinating these past six months to really see how insane some of it is. You take a firm like Diageo - it looks so expensive at first glance but it's one of the only major decently priced, really incredible long-term blue chips out there. Nearly every time cash comes in to our, or any of our family member's, accounts, I tend to buy more. You look out 60 months at what the earnings in constant currency should be and it's just so much more reasonable than many other companies with this fantastic balance sheet that is so strong it would be very, very difficult for it to go bankrupt without trying to fail, and a huge share of the global market in which it operates. There's no way it would come in a value screen right now because it appears, due to a few quirks, like it's trading at 25x earnings with a 2.3% dividend yield. The actual p/e is lower than the S&P 500, though, when you make the accounting adjustments. What I see is a business that should, within 36-60 months, be offering a good probability of 9% to 10% earnings yield with a ~4% dividend yield.
Did you see the report out of Credit Suisse showing historical industry-based returns? It demonstrates this whole efficiency nonsense perfectly. Sin stocks - tobacco, alcohol, etc. - historically continue to outperform by considerable margins for long-term owners because people don't want to own them for moral reasons. Humans are complex. They are not these perfectly rational calculators that some Nobel prize winning theorists seem to believe, nor is everyone capable of making equally informed decisions with the same data.
MoreResearch
March 7, 2015
Joshua - 2 quick questions on your data so I can empower myself to research this more:
(1) What sources did you find these changes to the index in? So we can continue to monitor future changes.
(2) What data or analysis are you referencing in terms of the lowered performance? I would love to study that data more in depth.
Thank you
Todd
March 8, 2015
Replying to MoreResearch
Maybe Warren Buffett wants investors to focus on the S&P500 index because his Berkshire Hathaway Stock would be one of the benefactors of the index since it is a Large Cap Stock which would lead to stable stock price.
Joshua Kennon
March 20, 2015
Replying to MoreResearch
1. Standard and Poor's breaks out their methodology changes themselves and announces when they make modifications. Here is a link to the 62 page breakdown of the current methodology from the firm itself as of January 2015. I'm sure they're very nice people but they pull this crap all the time and the news stories largely go unnoticed. Did you see, for example, back in August of 2014 they revised their CLO ratings methodology so they could make riskier loans appear more attractive, bundle them, and earn higher fees? Second verse, same as the first.
Note that it's not just the S&P index products either ... the MSCI announced around the same time they were adapting the free float methodology [PDF].
2. There's no simple answer to this so you're going to have to dive into quite a few different places.
First, think about the math of diversification. When a specific company does well, it drags the portfolio up like gravity, more than making up for bankruptcies and sub-par returns of other firms, which is one of the primary appeals of index funds. The top 5% or 10% of returning firms act like this gravitational force, dragging the index along with it. This pattern is repeated in all broad portfolios (e.g., Charlie Munger once talked about how huge it was for making Berkshire Hathaway so successful; that if you took out just the top 3 or 4 investments, the whole place would have compounded at average rates of return). Wharton Business School's research on the returns of the S&P 500 individual components over the past few generations, some of which are detailed in Dr. Siegel's book (see the footnotes and appendices) The Future for Investors: Why the Tried and True Triumph Over the Bold and the New reveals that several of the kicked out firms, Royal Dutch Shell and Unilever among them, were among the highest returning components and had a disproportionate effect on the index returns, dragging up the overall compounding rate by way more than the failures lost. Had they not been in the roster, the index would have spread that money out among the other initial components, which didn't do as well.
Second, you're going to have to go back and look at the individual S&P 500 index weightings by year and make reductions in the weightings of the components with large insider holdings - the Wal-Marts, Microsofts, Googles, etc. of the world. This is going to be a tremendous amount of work unless you can somehow get access to WRDS, the Wharton Research Data Service, which should let you pull it up in no time, but if that isn't an option, you can quickly get an idea of how bad it would have been by considering that in almost all cases, the firms effected, which still had heavy insider ownership because they were growing so fast the founders wanted to keep their equity, were also these upward-pulling gravitational forces that disproportionately helped the index returns. In the 1990's, for example, Wal-Mart and Microsoft, which helped drive the index to historical highs and were some of the best performing components in history, would have been underweighted, lowering overall returns.
Those two things alone, historically, would have kicked out at least half a dozen of the best performing equities of the past century, with the money dispersed to the other index components. Mathematically, a lower return is the only possible outcome under the reconfigured rules as it doesn't all get dumped into the next highest component or anything like that.
You might find working papers like this one interesting from back in 2007/2008 [PDF]. It looked at companies with low float that were underweighted in the index relative to past rules. They include Campbell Soup, Wal-Mart, Hershey, Pepsi, Nike, Brown-Forman, Kellogg, AutoZone, Coca-Cola, Goldman Sachs, and American Express. They point out that this was good for the index itself as it removed pricing distortions caused by all of the assets that have been shoved into index funds. Too bad individual investors were worse off than what they could have gotten had they constructed the index directly.
thegoblinchief
March 7, 2015
Very insightful article. Realizing I need to start amassing a bookmark list of investing pieces. This is going in there!
Alex
March 7, 2015
It looks like the S&P 500 isn't the only index making changes... Joshua what are your thoughts on the replacement of AT&T with Apple in the Dow?
lnt90
March 8, 2015
Replying to Alex
About time they added in Apple - seriously its LONG overdue. That being said the Dow is just a tradition - nothing more. Just because a company gets added in or taken out means nothing in terms of which will succeed or fail long term. IMO I would take AT&T being here in 50 years still over Apple even without them in the Dow, but that's just me. Though Apple is a tremendous company, the turnaround it completed is the stuff of American legends.
General Electric was taken out of the DOW twice in the early 1900's but long run that meant nothing. Bethlehem Steel was in the Dow for decades but was a TERRIBLE long term investment - even without the bankruptcy. There is no connection between a company being a successful investment based SOLELY on membership of the Dow. Woolworth was taken out of the Dow in the late 90's, someone who sold all there Woolworth shares based solely on that reasoning along would have never bought it when the company transformed itself into a great investment opportunity in Foot Locker. Point is, there is a story behind every company, looking at membership of the Dow Jones will not tell you anything about it and will not tell which is a good investment or not.
I wish they would fix the Dow & make it more relevant - expand it to 40 companies and find a way to fix the price weighted problem. General Motors & Wells Fargo should be included also. It's incredible to look at the legendary companies that made it up over the eras, they tell a story about America. Sears Roebuck the Amazon of it's day, General Motors when it was the greatest car company in the world, U.S. Steel the first billion dollar company (The Apple of its day).
BTW: The AT&T in today's DOW is not the same AT&T that joined 100 years ago, I've come across MANY people that do not know about the 1984 breakup. In a strange way the old Ma bell is still in the Dow even without AT&T, thats because Verizon is still in there.
Eric
March 8, 2015
Replying to Alex
It's really odd considering AT&T is about to add DirecTV, and expand into Mexico/South America in a big way. I wouldn't be surprised if in 10 years AT&T has a bigger market cap than Apple. (I don't own either).
Joshua Kennon
March 21, 2015
Replying to Alex
I think it's a strategic move against the S&P 500, not necessarily because they think it will generate higher rates of return in the long-run.
Specifically: In my lifetime, from the day I've been born until today, the Dow Jones Industrial Average has beaten the S&P 500 by more than half a percentage point per annum, which adds up to a lot of extra wealth over 32+ years. The massive success of Apple has resulted in it becoming a hefty 4.02% of the net assets of the S&P 500 compared to something like >1% for AT&T. The DJIA already has Verizon in it weighted at a higher level than that so by kicking our AT&T and replacing it with Apple it removes what amounts to a relative short position for the DJIA compared to the S&P 500 so the performance doesn't diverge too much.
I'd bet money 25 years from now the DJIA bought and held with no changes that had AT&T a part of it would do better than the DJIA index fund. The academic evidence on this sort of thing always shows the same pattern and I'm not sure I'd want to bet against it but you'd be shocked how many people just don't understand the math.
I mean, look at this blog post, which took me less than a few seconds to find. He argues against buy and hold investing because, "Of all of the stocks that were part of the original Dow Jones Industrial Average, only General Electric is still a part of the index." Only it's complete nonsense.
American Cotton Oil? Yeah, that's now part of Unilever, one of the most profitable, successful investments of the past century.
American Tobacco Company? Sweet Lord it ended up becoming Fortune Brands, one of the most successful spin-off and compounding enterprises in the history of human civilization. It went through so many reorganizations and spin-offs that you ended up owning everything from home security systems to Jim Beam whiskey.
Distilling & Cattle Feeding Company? Yeah, that ended up part of British giant Hanson PLC before being spun-off as an independent business with a name change, and then taken private.
Chicago Gas Company? That was aquired by Peoples Energy, which later merged in Integrys Energy Group, which was acquired last year by Wisconsin Energy Corporation.
General Electric? Still around plus a bunch of spin-offs.
Laclede Gas Company? The largest natural gas company in my home state of Missouri. It may be a boring utility but in my lifetime, it's compounded at just shy of 9% assuming no dividends reinvested.
United States Rubber Company ended up finding its way into French blue chip Michelin.
National Lead? It's complicated but it is one of those situations like Eastman Kodak where you didn't lose money despite horrific losses on the surface if your family held it as part of the original index. The company itself, which worked on the atomic bomb for the government and was known for its Dutch Boy paints, paid out dividends and spun-off its Baroid division, which is now part of Halliburton.
North American Company? That was broken up by the SEC, and permitted by the Supreme Court decision in 1946 thanks to the earlier-enacted PUHCA, shattering a public utility empire that was straight out of the gilded age; a sort of Standard Oil of electric companies operating an incredible network of subsidiaries that spanned the country.
Tennessee Coal, Iron, and Railroad Company found its way into U.S. Steel.
U.S. Leather Company has the distinction of being the only original Dow component that went into liquidation. It final reorganization involved a distribution of cash and a one-for-one share exchange for Keta Gas & Oil Corporation, which was subsequently abused by a scoundrel named Lowell Birell, who hid his misdeeds in the financial statements as a sort of front. So, yes, this one was largely a failure.
Historically, you did very well. Of course, everyone now ignores the huge portfolio of shares you inherit spanning multiple continents and currencies because ... face it ... people are lazy. They just want to pull up a quote and get a quick answer but that's not how the real world works.
It seems weird to me that they are removing a nearly $200 billion giant, which has served the index well since 1916, but it's their prerogative.
jack's smirking revenge
March 7, 2015
What do you think about equal-weight index funds? Or even the "fundamental"-weighted funds? I ask because it would be very expensive and difficult for an individual investor to buy shares in every single S&P500 company and keep things balanced.
joe pierson
March 7, 2015
Replying to jack's smirking revenge
The DOW 30 is just as good as the S&P 500.
Joshua Kennon
March 21, 2015
Replying to jack's smirking revenge
Even the equal weight funds have adopted this free float methodology to avoid pricing dislocations, which is rational on the fund level though sub-optimal for the individual investor.
Mathematically, the combination that has worked out best for the past few generations (I have a couple thousand word draft I'm working on about the topic with excerpts from research at Wharton) has been to 1.) equal weight in the beginning, 2.) let the natural weights develop from there to minimize taxes, costs, and fees (no rebalancing). The outperformance has been significant.
If I were building some sort of closed portfolio my family had to hold for a very long time with no subsequent changes, I'd almost assuredly start out with equal weights rather than market capitalization weights. There'd be no contest.
Hexar
March 7, 2015
How do you feel about the Vanguard Dividend Growth Fund?
Joshua Kennon
March 21, 2015
Replying to Hexar
I'd have no major objection to it, though, personally, I'd have a hard time giving up 31 basis points when I could just as easily construct a direct dividend portfolio myself with no on-going expenses. It would add up over a multi-decade period to some real money if you were dealing with larger amounts (if the amounts were smaller, the convenience would make it an incredible bargain so I wouldn't worry about it). For a lot of folks, this is dangerous, though, because they can't remain rational when they see huge divergence in individual component performance or they are tempted to load up on a handful of businesses, increasing risk.
I know it's returned slightly less than its counterpart, but I'd be more inclined, were I to invest through such vehicles, to acquire something like the lower-cost (0.10%) Vanguard Dividend Appreciation Index Fund, instead, because it does it with 1.) less company-specific risk and 2.) relies on less human judgement, lowering manager risk, as it is based on the Nasdaq U.S. Dividend Achievers Select Index, which requires a business maintain ten consecutive years of dividend increases. It excludes limited partnerships and REITs, which I like because I'd prefer to manage those separately, as their own investment type since the risks and tax rules are different. Even if it shaved off return in the long-run, I'd sleep better at night with it but that's just me.
I'm not crazy about the periodic rebalancing of the latter for certain reasons but if you held it in a tax-advantage account, it won't matter as much (the index it tracks reshuffles the asset deck so that no one component represents more than 4% of assets with any excess "distributed proportionally across the remaining Index Securities" (Source PDF).
Here's the deal, though: What counts is total return. The SEC yield on the Vanguard Total Stock Market Index Fund Admiral Class, which trades under ticker VTSAX, is not that much lower than the two funds we've discussed. It includes far more businesses with different economic characteristics and growth profiles. Why not just buy it, instead?
If you forced me to put 100% of my net worth in a single domestic Vanguard fund, it would be either the Vanguard Total Stock Market Index Fund or the Vanguard Small-Cap Value Index Fund (the latter is much riskier in the sense that the businesses are inherently worse but, historically, the evidence is abundantly clear that small cap value equities outpace everything by a percentage point or two over 25+ year compounding periods because they are so distressed, they trade, at times, for far less than they are worth, allowing the reinvested dividends to work major. You also get the occasional supernova, which pulls the whole thing upward. In this case, you'd be buying them in a basket of 3,800+ stocks so the firm-specific risks are severely mitigated even though it drives me nuts the second largest holding is an airline. It's probably going to be a lot more volatile, though, especially if we hit a Great Depression.) If I were over 45 years old, I'd probably go with the first one. It's basically America, Inc. If the country prospers, it should prosper. I may be tempted to go with it, anyway, given that it's generally stronger so I could consider the relative point or two given up as an insurance premium against a 1-in-600 year event.
Internationally, I think Vanguard has done the world a great service with its Vanguard Total International Stock Index Fund. Investors who know nothing about foreign accounting rules, currency translations, etc. can get exposure ranging from Switzerland to Taiwan all for 0.14% in expenses per year. It holds 5,300 stocks in 46 countries. If I were a doctor who knew nothing about finance, I'd dump 25% of my portfolio into it year after year, decade after decade, dollar cost averaging my way into things like Nestle and Bayer, Novartis and Royal Dutch Shell.
Note: None of this is a recommendation. I'm simply thinking, or typing, as if you were sitting here and we were talking casually. I have no idea what is appropriate for you.
Hexar
March 7, 2015
Also, what do you think of the arguments here?
http://www.marketwatch.com/story/why-vanguard-total-stock-market-isnt-the-best-fund-in-the-fleet-2014-12-03?page=1
Joshua Kennon
March 21, 2015
Replying to Hexar
Academically the author is absolutely correct in asserting that the data shows small cap value equities have crushed all other equity classifications over the past near-century. Ibbotson & Associate's yearbooks have highlighted this buried hundreds of pages into the data sets year after year and the return differential is substantial when compounded out over time. It cannot be disputed.
That is the exact reason I said in another response on this page:
If the sole consideration is 25+ years with absolute disregard to volatility, the structural advantages of the small cap value index give it the highest probability of the best long-term returns if you believe the next century will look like the last century; that this is a "normal" that will persist. You cannot deviate, though. For it to work, even if you suffer through year after year of godawful returns, you have to just keep dollar cost averaging into it as if it weren't happening, reinvesting the dividends, too. If you lose heart, you'll be screwed. It's not psychologically appealing to a lot of investors.
Give me a second and I'll go take a picture from the Ibbotson SBBI return calculation yearbook. The most recent one I have handy is from 2011 so the returns are much higher due to the market skyrocketing in the meantime but you'll get the point.
Hold on ....
Okay, there. It should be attached. It's Graph 8-5 on Page 110. Between 1927 and 2010, $1.00 invested in each index strategy resulted in the following terminal values:
* Fama-French Large Growth Stocks = $1,078.18
* Fama-French Small Growth Stocks = $1,475.79
* Fama French Large Value Stocks = $6,384.81
* Fama French Small Value Stocks = $59,017.27
Bill Larson
March 8, 2015
Joshua returns from vacay & fluff w a haymaker
Adam
March 8, 2015
Joshua back and laying down the gauntlet.
jonnymack
March 9, 2015
So much awesome information. Thank you! This is so very helpful to me and I would have never picked up on these differences.
Brendan
March 9, 2015
I'll have to take another read-through of this article just to fully get into it, but certainly an eye opener despite prior articles that delved into what makes up various indexes, and how to interpret them.
innerscorecard
March 10, 2015
Can you write something about negative interest rates in Europe?
Joshua Kennon
March 21, 2015
Replying to innerscorecard
I'd like to when I have time. It's certainly an interesting development.
Anon
March 10, 2015
I always notice and then stop eating at those restaurants.
Fin
March 11, 2015
Thinking of building a portfolio of individual S&P 500 stocks, has anyone distinguished the effect of reinvesting dividends in the individual stocks that generate the dividends vs. investing the dividends across the entire index? I am trying to think through how this would work in a brokerage account. Thanks!
Zaphod
March 16, 2015
Replying to Fin
Thats certainly an interesting idea, if you could run a back test for even a short period of time it would be neat to see what happens either way.
Joshua Kennon
March 21, 2015
Replying to Fin
This is one of the primary questions I've had for a few years but unless I decide to go get an MBA or Ph.D. as an example to my future children and make it part of my educational program using a major university's resources, the months of endless work and research it would require to calculate it couldn't be justified. It would be a near full-time job for awhile. I'd really need a graduate assistant or seven to pull it off in a reasonable span.
Based on everything else I've looked at in my career, I have a sneaking suspicion that the highest absolute returns would be individual company reinvestment because a single Philip Morris turned $1,000 into something like $4,000,000 or $5,000,000 if you pull Jeremy Siegel's papers (the number should be a lot higher today as that was quite a few years ago). You could just have an obscene amount of Kodaks and Worldcoms go bust along the way and it would more than make up for it. Those returns were driven by the perpetual undervaluation and dividend reinvestment in it, which acted as a return accelerator.
White Coat Investor
March 16, 2015
I've never been a big fan of the S&P 500 index funds, using TSM myself for a domestic component. I'm curious how much of an effect you think the float adjustment has on returns there. Can it be quantified? Is it 0.5% a year? More, less?
And I totally agree with you about the "religious" investors, both indexers and gold bugs. Sometimes investors forget why index investing works.
Joshua Kennon
March 21, 2015
I don't think it's been implemented long enough to draw any meaningful, quantitative conclusions about the overall level of underperformance. You could measure it, sure (and I think I might have a paper around here where someone attempted to do so, including on foreign markets where it had been enacted - I want to say there was one on Turkish stocks? I'd have to check), but I'm not certain it does any good since it's barely been 10 or 15 years, which isn't a lot in stock market time when looking at broader equity results.
The interesting thing about float adjustment is it makes perfect sense for the index itself and the index funds. It avoids overpaying for the smaller components, creating perpetual bubbles in certain equities. It's just irrational for the individual investor who, acting on his or her own, can buy without impacting the market. It's one of those, "It works as long as not everybody else is doing it" things. It's kind of a wonderful, painful paradox.
I'm with you. If I were inclined to put everything into index funds, I don't think you can get much better than the joint combination of something like the Vanguard Total Stock Market Index Fund and the Vanguard Total International Stock Index Fund. Between the two, you're paying a handful of basis points for exposure to most of the commercial and industrial activity of planet Earth via thousands upon thousands of individual firms you own indirectly. You could go about your life, devote practically no time thinking about your investments, and dollar cost average for 25 or 50 working years. That is about as sweet of a deal as you're going to find anywhere.
Ari Wilson
December 29, 2015
This article seems sensationalistic at best. The real point you seems to be making is that large indices are subject to constraints (e.g. huge AUM preventing some types of non-market weighting of stocks) that individual investors aren't subject to and thus wealthy individual investors can (slightly) improve returns by buying their own baskets of stocks (presumably at the cost of their time and potential transaction fees).
Some misleading advice:
You say turnover lowers index fund returns over a basket of the index fund's stocks bought at any particular period of time. I doubt this is true across all periods of time (turnover in S&P 500 is directly related to stocks being added/removed/reweighted within the index which could increase returns if e.g. Google is added to the index and Enron is taken out) but even if it is true, it is deceptive compared to the larger point: market-weighting actually massively reduces turnover compared to most other market strategies, especially equal-weighting that has to be continuously rebalanced.
The index you advise (Vanguard's Total Stock Market Index) also commits at least the first two "sins" it mentions. It doesn't include international stocks and it does float weighting as well; see the methodology of the underlying index (CRSP US Total Market Index): http://www.crsp.com/files/Equity-Indexes-Methodology-Guide_0.pdf The third "sin" (allowing mortage REITs in the index as a stock) seems relatively minor as REITs themselves don't form a large percentage of S&P 500 assets (~3% according to some old analysis I looked at:https://www.bogleheads.org/forum/viewtopic.php?p=10838#10838).
A better article would make the point about AUM and wealthy investors, recommend VTSMX for people who have it available, and recommend S&P 500 index funds otherwise.
Joshua Kennon
December 29, 2015
First and foremost: Welcome to the blog.
Second: I'll try to take these one-by-one to address your concerns:
The proposed solution for wealthy investors that has been discussed multiple times around the blog - the one that Dr. Jeremy Siegel at Wharton studied and that, later, Vanguard's own John Bogle wrote about in some of his works - was to buy an equal weighted portfolio of directly held individual stocks then make no subsequent changes, including rebalancing. As a result, a natural market weight develops over time but the tax basis for all positions is unique to the owner of the personalized, private index fund that has effectively been created. I'm not sure why you are assuming annual rebalancing would occur. It wouldn't.
Siegel's work found that these portfolios always outperformed their more active index-fund counterparts and led to greater tax efficiency once you started stretching out to longer periods (e.g., if you're planning on holding for 25 years or more, your odds of adding a good amount of compounding above the index as currently constructed in the public funds go way up by taking this approach). To be perfectly blunt about it, as far as probabilities go, you can say with an extraordinarily high degree of certainty, were you a betting man, that the S&P 500 bought in equal weights today, held directly in a custody account with no subsequent changes, is going to outperform an identically invested amount put in something like VFIAX over the next quarter-century. The biggest question would be in whether you opted to 1.) hold spin-offs or 2.) sell spin-offs and reinvest the proceeds in the former parent company. Siegel has some interesting things in his methodology about the two approaches if you ever want to take a few months and dive into his body of work (it's extraordinarily high quality; easily, he's one of my favorite academics examining modern portfolio construction today and I recommend you read anything and everything he publishes).
You say you "doubt" this to be the case, but I'm not a fan of making decisions based on feeling. I want data. Publish research that disproves Siegel's and I'll look at it. Until then, I'll believe his math because I've looked at his methodology, I've read thousands of pages of his analysis, and I have faith that his process is correct based upon my own examination of it. If you think he's made some sort of error, or can produce verifiable information that contradicts his thesis, I'll give a fair, impartial reading and change my mind. I promise. I don't have a "side" here ... I simply want the most accurate reflection of reality I can get, even if it's solely for my own academic satisfaction. Even if you don't want to read the research papers, he's created several bestselling books that put the findings in easy-to-understand format (the best of these, in my opinion, is due for its 10-year update but it's absolutely worth buying.)
Bogle once worked out an interesting mathematical short-cut that captures a lot of what Siegel found. I, personally, find it elegant in its simplicity: He (Bogle) suggested buying the top 50 stocks by market capitalization, equally weighting them, and parking them, again, with absolutely no subsequent changes. At the outset, you effectively have nearly half of the market capitalization in the United States, you get wonderful tax efficiency, and your on-going costs end up lower than even the Admiral Class shares of the comparable index fund. By the end of the period, you'd have a bit more concentration as mergers and private buyouts reduced the number of components but you'd have a much better compounding rate and ending wealth. He tried to get Vanguard to offer it to high net worth clients his colleagues wouldn't bite, if I remember correctly. Personally, I think it's a real shame they didn't listen to him.
These days, with institutional commissions so low, I'd probably take Bogle's solution a bit further an expand it to 75 or 100 stocks, capturing more of the total market capitalization but it really is beautiful in its effectiveness and minimalism. He cut right to the heart of the matter. It triumphs over all of the short-comings of the public index services while giving the investor practically all of the upside. It's indicative of why, especially early in his writing career, he was one of my favorite investment thinkers.
There is a single public mutual fund - one - that is built upon this approach. These days, it's called the Voya Corporate Leaders Trust fund. Back in 1935, the manager who put it together wanted to prove a point that Bogle and Siegel subsequently hammered home. He created a passive trust fund, bought a collection of blue chip stocks, and wrote the rules so that almost nothing could be changed except in the most extraordinary circumstances. Even if a firm was racing toward bankruptcy, too bad, can't sell it. The fund charges a 0.50% annual fee. It's utterly trounced both the Dow Jones Industrial Average and the S&P 500 for the past 40 years for the precise reasons Siegel's research explains over and over again. It's a living, breathing didactic example of what Siegel and Bogle were talking about ... even a few bankruptcies along the way don't matter in the face of the methodology, which results in superior compounding outcomes. Time after time this plays out but people don't want to believe it. Yet there it is, in the data. Pick a date, build an equal-weight portfolio wit sufficient diversification, let it naturally develop with no changes even if some of them go bankrupt ... you beat the S&P 500 index.
My point in writing about the flaws with the S&P 500 index methodology is pointing out that the people sponsoring the products should be required to disclaim that historical market returns are now useless because the methodology has been changed sufficiently enough by the committee responsible for managing it that historical returns would be substantially lower if the same methodology had been used as is presently in place. That's it. It's about informing investors; being transparent, and maintaining honesty. You may still decide it's the best trade-off for your situation, just as I did for my family member who had a limited choice among the options in his/her employer sponsored 401(k) plan, but you at least go into it with your eyes wide open. In my case, I had the family member model a lower future compounding rate from the S&P 500 when projecting retirement assets/income and, as a result, convinced this person to substantially increase the withholding rate so a minimum of 15% of the paycheck was going into the plan.
1. Yes it does. As a result, the determination of which is superior must be left to other advantages and, when weighing the scales, I find Vanguard's Total Stock Market Index fund to be superior to the S&P 500 one. (The latter is still a perfectly acceptable, if not optimal, choice within certain 401(k) plans - I have a member of my own family heavily invested in an S&P 500 index fund through his/her employer because it was the best selection among the potential roster of choices so the trade-offs were worth it.)
2. I'm reminded of the Charlie Munger quip about "turds and raisins". Despite the juvenile, tongue-in-cheek nature of the comment, it does not strike me as an intelligent argument; "Oh this? It makes no sense, and it violates the underlying methodology of what we were doing, but it's only a tiny portion of the portfolio so we're going to do it anyway even though it's totally unnecessary and doesn't serve some other, greater purpose that might justify the practice." Who behaves like that? There's no excuse. They should be kicked out of the index.
Write it. I'm being completely serious. I'm not going to do it because I don't think any intelligent analysis of the data can support such an article therefore I'm not putting my name next to something I don't believe. It will not come from my pen. If you want it to exist, you're going to have to make it happen.
Now that I've addressed your concerns, here's my question for you: Generation after generation, the math has shown that long-term ownership of a diversified portfolio of equities, held in a tax-efficient, low-cost way leads to the best outcomes. There are multiple ways to achieve this. As an efficiency mechanism, the modern index fund was merely one of them. It is not the best way to capture that formula but it is often the easiest. Sometimes that has real utility and is "good enough", so to speak. It's perfectly acceptable to say, "Eh ... I'm fine with this. I recognize the shortcomings but it will do most of the heavy lifting."
What is wrong with acknowledging that simple fact? Why is it that the biggest advocates for indexing, Bogle and Siegel among them, write extensively about this in their thousands of pages of output but the layperson who doesn't actually do the deep analysis gets obsessed with worshiping the form rather than the principles that make the form work in the first place?
When I write on this - my personal blog - I do it to an audience of highly successful, highly intelligent people who are overwhelmingly mathematically literate. Have you checked the demographics of the people who make up this community? There is nothing sensationalist about this post because I'm engaged in a discussion with people who make up the top fraction of society in terms of ability, outcome, education, and resources. It might be dangerous for me to write a post like this if I were submitting it to something like People magazine but the members of this community are scientists, doctors, lawyers, economists, teachers ... They're intellectually and emotionally mature enough to understand that a thing might have flaws, sometimes significant flaws, and, yet, still be the most attractive choice in a given situation. If you have a problem with this type of post, you are most definitely not going to like the blog because for something to be interesting enough for me to write about, it almost always involves nuance, gray areas, or complexity. That's important to know because, often, I'll attack the things I like the most because I consider them strong enough to survive the assault. I want to know the bad; the shortcomings; the flaws. The good stuff will take care of itself.
Ari Wilson
December 30, 2015
Replying to Joshua Kennon
Hi Joshua,
Thank you for your thoughtful and polite reply.
I am aware that equally weighted S&P500 indices have outperformed the market weighted S&P index over longer periods of time (>20 years) over the last half century or so. My belief is that this has more to do with the medium/small company tilt it introduces to the index (increasing risk, increasing returns) than any fundamentally superior property of equal weighting and thus could be replicated by purchasing small/medium/large size market cap index funds in a similar initial weighting.
As far as never rebalancing goes, would it be possible for one to compare the returns of an equally weighted S&P500 index with rebalancing to one without? Rebalancing is often one of the free wins in portfolio construction; it only suffers here due to the high amount of turnover in the smaller companies in the equally weighted index.
Ang
December 31, 2015
Replying to Ari Wilson
Just a couple of thoughts while Joshua (possibly) writes up his own reply to you
You keep stating your "belief" when it comes to the underlying reasons and factors that influence index returns, but as Joshua said, it can be helpful to look at the actual data. You seem to subscribe heavily to Fama French's efficient market and factors theory that seems to be heavily in vogue (perhaps because it is teachable and, owing to "physics envy", mathematically elegant? whereas a complete education on investing and general cross-disciplinary knowledge is difficult), but there are many other sources of investing knowledge, such as Siegel's works, that you can explore.
As Charlie Munger says: “I never allow myself to have an opinion on anything that I don’t know the other side’s argument better than they do.” and “We all are learning, modifying, or destroying ideas all the time. Rapid destruction of your ideas when the time is right is one of the most valuable qualities you can acquire. You must force yourself to consider arguments on the other side.” - even if you think you already know everything there is about index investing, I guarantee you that Joshua has THOROUGHLY studied your side, including the source works of Bogle (who, as Joshua already pointed out, advocated for holding INDIVIDUAL stocks in the most tax efficient way, which is to buy them in equal amounts, then avoiding activity - Bogle's famous phrase of "don't do something, just stand there!" is very relevant in this instance), so the rational thing to do in continuing the debate is to take the time to study the other side as well - btw, Joshua's article on the "ghost ship" portfolio he reference in the comment above is here: https://www.joshuakennon.com/im-building-ghost-ship-portfolio-someone-sort-index-fund-steroids/ and it has some great embedded links
On rebalancing, I don't have the data to answer your initial question, but I would like to challenge your thinking that rebalancing is a "free win". This might seem like the case if you are a know nothing investor and prefer a smoother, less volatile ride up over time from your manually constructed index (an "average" or "market" result), but it would be a terrible idea in practice. You're in essence, from the constant rebalancing, cutting your winners and praying for mean reversion from your losers, in addition to the frictional costs associated with turnover such as brokerage fees and taxes. For a case study on how one component (Apple is the most recent example in practice/reality) of a manually constructed index can have an outsized effect, see Joshua's article here: https://www.joshuakennon.com/the-mathematics-of-diversification-and-wealth-building/
I hope you stick around, there's an enormous back catalog of useful and life altering (seriously) information around these parts that Joshua has graciously provided for free. I would look at the comment sections whenever you can because that's where he tends to dispense the most knowledge.
Rob
February 13, 2016
Very interesting article Joshua. I wonder what you would prefer if you were in the following situation. Let's say you're a 25 year old buy-and-hold investor with a €100k net worth. Would you prefer investing in:
1) an ishares msci world index fund with 1500 mid and large cap companies all over the world that reinvests dividends. Advantage is the simplicity and tax efficiency as we don't pay taxes on an accumulating fund that reinvests dividends here in Belgium. Disadvantage is the 0,2% expense fee.
2) 100 equal blocks of 1000€ of the 100 largest companies in the index. Maybe replace a few you don't like such as Facebook with a more defensive option like Hershey. Rebalancing is not allowed. Disadvantage is you pay 1000-1500€ upfront in transaction fees and a 25-40% tax on dividends. Dividends have to be reinvested manually. No
fees after the initial set up (except when reinvesting dividends).
Would be really interesting to hear your opinion.
Ang
February 13, 2016
Replying to Rob
Are you sure the reinvested dividends in your fund isn't taxed on an annual basis? It sounds strange to me to tax dividends that are distributed (and can be chose to be reinvested by your brokerage) but not dividends that are pooled by an investment vehicle and reinvested in that same vehicle
Rob
February 14, 2016
Replying to Ang
Yes, I'm pretty sure. We also don't have capital gains taxes unless we sell a stock within 6 months of buying it.
edpark
May 25, 2016
Joshua-- brilliant and incredibly thoughtful post. I read the Bogle and Siegel works you mention and decided to implement an executable strategy on Quantopian-- pick 100 stocks at random from the S&P 500, construct an equal-weight portfolio, and hold forever. I tested it with multiple runs over multiple time periods. I have not yet found a time period of over 3 years where it underperforms the S&P, and quite a few where it outperforms the S&P by a very wide margin. As the theory predicts, the longer the time period, the more significant the outperformance.
In any case, I just wanted to thank you for writing this. For those readers interested in backtesting the algorithm themselves, I'm sharing my Quantopian implementation here:
https://www.quantopian.com/posts/chimp-algorithm-100-randomly-selected-s-and-p-500-stocks-equal-weight-buy-and-hold-forever
Zack
June 21, 2016
All of this discussion of the flaws of the S&P500 seems well and reasonable for investing a fixed lump sum for maximal return. I'm struggling to understand how compatible a fixed allocation of equities (a la LEXCX or DJ30) is with a continuous inflow of savings. It would seem that in addition to requiring ~$20k/contribution to acquire an individual share of each underlying security, you'd face headwinds from transaction frictions on each individual purchase making such a strategy rather difficult to employ (and thus out of reach of most retail investors, even if they understood the financial side of things well enough to enact it).
bob%bronsons.com
June 9, 2017
Equal-weighting requires repeated rebalancing, which includes transaction costs. Equally-weighted indexes automatically do this rebalancing every day by using price relatives which are not discussed in this article.
The Value Line Geometric Index does this by taking the geometric root of all the component stock's price relatives multiplied together. This creates a negative bias since the geometric mean is always less than the arithmetic mean.
Arithmetically averaging the percentages changes creates a negative bias because percentages are not log normal. A round trip from $10 down to $9 and back up to $10 is not: -10.0% + 11.1% = +1.1%
Also, indexes that have a fixed number of components create a survivor bias that artificially boosts their performance. For the S&P 500 think about General Motors being dropped then added back in later (different company) so its effect is not real money realistic.
There is much more to this story.