There’s a saying about prediction models, garbage in/garbage out. All true. This begs the question, how long do you have to sift through garbage until you actually find something useful? The answer depends on what you are looking for.
I’m often asked about the “look back” periods in Geometric Balancing. Many investment models use months and months (or longer) of data to generate return and standard deviation (volatility) estimates. Many readers have written, surprised at how fast my allocations to each asset change. They wonder if Geometric Balancing uses short look back periods.
To find out, let’s look at how much data we actually need to develop confident estimates for return, standard deviation, and correlation.
The Random Model
To do this, we’re going to create random normally distributed data. We will then sample the random data and since we know the true value for the underlying statistics, compare the sample data against the real values. Adjusting the quantity of samples will show a relationship between sample size and prediction accuracy.
The random data will have the following properties:
- Average: 0.0004
- Standard Deviation: 0.01
- Correlation: 0
These properties mirror the daily returns of the S&P 500.1 Annualized, the values equate to 10.5% return with a 15.8% standard deviation. The study will combine 25 simultaneous runs to produce a result, followed by an absolute average error calculated across these 25 runs.
The sample will grow from 1 to 250 random values, to again simulate an entire year.
Lets see what happens.
How Does Return Converge?
Here we see the average return as a function of sample size. Our signal is converging, but its still pretty wide, even out to 250 samples. Taking the average of past returns doesn’t teach us much about the true underlying return of the process. There is just too much variation and volatility for the data to converge.
Now let’s look at the average error of the sample vs. the actual average of the random process. Optimally the dots in these charts will converge to zero, indicating a strong signal with little estimation error.
You can see the error converging downward toward zero which is good. But even out at the end of the chart, the error is still nearly 0.0005%.. This error is larger than the actual return of the underlying process (0.0004%).
Annually this error equates to a very wide return range of -3% to 25%.
Effectively, even after sampling 250 random data points, we’ve learned nothing about the underlying true return. With enough data, the error will eventually become minuscule, but it’s going to take a while. When data has a low return to variance ratio, extremely large samples are needed to provide information about the underlying return. The volatility simply hides the information.
What About Standard Deviation?
Standard deviation is different. After 20 samples, we’ve pretty much found the standard deviation. It slowly continues to converge further over time. However each additional data point adds less and less information to the estimate.
Looking at the average error we see a quick convergence toward zero. Once again the error plateaus a bit at 0.0005. But unlike the return stream this error is only 5% of the true underlying standard deviation (.01).
Yes, there is still error but the difference between a standard deviation of 1% and 1.05% is not terribly consequential. A 5% error in standard deviation creates a 10% error in variance. You can size and create portfolios with 10% levels of input error.2
It is possible to be confident about the underlying true standard deviation through sampling alone. Furthermore, a large sample is not necessary to be confident in the prediction. We don’t have to wait a year collecting data in order to find something useful.
This is part of the reason some people (including myself) say volatility is predictable. It doesn’t take much data to uncover an accurate estimate.
And Correlation?
Correlation works out similarly to standard deviation. It converges quickly towards 0. You can see it doesn’t take long for the average value to fall below 0.05 correlation.
Similarly, the standard error of the entire sample is less than 0.1 after 50 samples. It doesn’t take very long for the sample correlation to converge to a reasonable amount of error. Its clear after a while, each new data point provides less and less new information.
This convergence isn’t as fast as standard deviation, but it’s still significant. The difference between a correlation of 0.1 and 0 in portfolio construction is relevant, but not enough to drastically change portfolio allocation decisions (as long as the correlations stay away from 1).
It’s possible to have have some confidence in the underlying correlation with sampling. And as with standard deviation it doesn’t take hundreds and hundreds of data points to gain this confidence. This is also why some feel correlation is predictable.
Dynamic Variables
The above analyses uses static input variables. But what if the return, standard deviation, and correlations change over time? What if they are dynamic?
It’s clear sampling returns doesn’t give us any usable information on the true return. A dynamic return only makes this problem worse. We still won’t learn anything from sampling returns.
Standard deviation and correlation however do converge meaningfully. Therefore the question becomes, how do you value the trade-off between:
- A large sample which provides a more accurate estimation of the underlying standard deviation. However because of their size, large samples are sluggish adjusting to new data when the standard deviation changes.
- A small sample with the ability to identify and react to changes in the underlying standard deviation, but with the drawback of over-reacting to random data points.
The elbow of the curve, somewhere between 20 and 50 samples, seems to be a sweet spot.3 Here you receive the best trade off: getting an accurate read on the true volatility and correlation, while still begin capable of reacting to changes through time.
Conclusion
Now technically, this a study of random sample convergence, not of the stock market. Stock returns have fatter tails, slowing convergence. Also investment returns, volatility and correlation seem dynamic and not static.4 But we have to start our study somewhere.
If investment returns are mostly random, then prior returns possess enormous amounts of error. Therefore they are always suspect. You can’t confidently learn anything about the underlying return from even moderate sample sizes. This is another reason why I don’t use trend or momentum to determine future returns.
On the other hand, small samples produce reasonable estimates for standard deviation and correlation. Statistically it should be possible to understand, predict, and respond to the current market volatility and correlation. This is why Geometric Balancing responds quickly to changes in market volatility and correlation.
“Look back” windows for volatility and correlation can be short.
1- I like to think of stocks as broken down into days. But in many ways the returns are closer to continuous (think every second) than discrete.
2- Of course it’s better to reduce error rates further, but we have to start with the basics.
3- Unless you layer other methods of prediction on top of a very crude one like this.
4- Others would say that individual investments have momentum. I’m not on board with that view.
Is it a week then?
Will you ever spell out the exact details? Or at some point will you try to sell it?
This site is useless as is. Just you bragging, nothing to learn and adapt.
If I understand correctly from other posts, one needs to know the return for each asset class in order to determine the weight assigned to it .
If this is not possible in a short time-frame, how is it done? For example, it’s easy to determine both the arithmetic and geometric returns of the assets over the past n years, but using that to backtest introduces bias by knowing in advance what the returns are going to be, no?
Perhaps it’s sufficient to use the current risk-free rate / interest rate on savings and the arithmetic / geometric returns of each asset class over a very long period of time? e.g. S&P has data going back to the 1920’s, covering almost every type of market boom and bust.
In response to Klaus’s comment – even as an amateur R programmer, I’ve managed to formulate approximately what BTM has outlined in his posts and reproduce a backtest that shows a better return than the Permanent Portfolio, which is essentially what the mix of assets he’s proposing are. It is possible to understand what he’s outlined, although I admit that I’m struggling on some aspects (e.g. adjusting weights using correlation across four assets). Even ignoring leverage and correlation, I’m fairly happy with what the script is suggesting.
Plus it’s fun to have a puzzle, both in terms of coding, and in terms of understanding the model / mathematics behind it!
I’ve updated my R script so that it enables me to either used a fixed (arithmetic) annual return based on the length of the data available, or I can specify a rolling window.
Given that the specific window results in significantly different annual returns, I wonder what you use BTM?
Do you just use a fixed arithmetic return (i.e. over the longest period possible). This seems reasonable if the dataset is very large (e.g. 25 years). However, I can only get data as far back as 2011, which skews my arithmetic returns to that time frame (or less if I use a rolling window).
I do have gold going back to the 70’s, but that also skews the arithmetic return for that asset because it’s done amazingly well over that period.
If I use a rolling window of the last 7 years, I get similar weights to those at the top of the page (48% Stocks, 36% Bonds, 16% Gold, 0% Cash at the time of writing), but of course I’m using assets for my currency, so perhaps that’s misleading.
Incidentallly, I found that a 40 day rolling window for SD and correlation seemed to work best.
Any tips on how to get the arithmetic return for each asset BTM?
Ok, I realise I’m kinda talking to myself here, but whatever…
After a bit of thought and a bit of testing, I realised that using a “fixed” annual return based on the long-term data would be wrong for a couple of reasons:
1) Look-ahead bias – backtesting using “today’s” annual returns, even over a long period of time, introduces the bias of knowing what, for example, the total return over the NEXT period will be.
2) Lack of response to changes in the annual return. As an asset’s arithmetic return falls (and when its SD increases, which may or may not happen in tandem), we wan’t to reduce our exposure to it. This is probably where BTM has commented that one of the limitations of the method is not responding fast enough to changes in the market.
I’ll continue to experiment with the annual return moving window…
Your thought process is correct in my opinion.
OK, I’ve done some more testing and think I’ve come up with some decent compromises.
1) Since the dataset I have is 9 years in total, I’m using a 4.5 year window to calculate arithmetic returns. My reasoning is two-fold: a) I want as much data to backtest as possible – using a shorter window maximises the data over which I can test. b) On the other hand, using a longer window reduces the “response time” of any changes in asset allocation. I’m hoping that 4.5 years is long enough to get an idea of the arithmetic return for that asset, but short enough that the model reacts to changes and provides enough data over which to test. Wish I could get hold of a longer dataset.
2) I’ve added a “lag” option to my code. The risk in backtesting is always that the market closing price is known on the same day that a portfolio change can be made. In real life, this would be impossible as the closing price would of course not be known until the next day. In addition, dependin on what vehicles one is using for each asset, there may be additional delays (e.g. an ETF may be instand, but a Fund may be delayed by a day).
However, even if I lag the market by 2 days, the returns are still better than a rebalanced 25% equal split (i.e. Permanent Portfolio), so I’m going to give it a go.
Since I’m in the UK, I’m using Gold priced in GBP and long UK bonds (Gilts), but I prefer to use a global stock market, rather than a UK or US-centric one.
Using my attempt at recreating BTM’s model, this gives asset allocations of 25% Bonds, 51% Stocks and 24% Gold as of the time of writing (using yesterday’s closing prices).
On your lag. This is the reason I try and trade at the very end of the day. Normally, the last 10-15 Minutes won’t change the price enough to change the portfolio. I think this is better than trading on day old data.
Since I’m super-nerdy, I also tested various rolling window lengths for the SD / correlation (as short as 5 days to as long as 60 days).
Just like BTM, I found that the Sharpe / CAGR increases with an increasing length of data over which to calculate the SD / correlation.
For my data, the optimal was at 38 days, but anything over 35 days was very similar (e.g. all were around a Sharpe of 1.03 – 1.045 and all were within 0.2% CAGR of each other).
Thanks BTM.
Can you tell me/us what period you’re using for annual returns?
I backtested with my script using the same assets as you (I can only go as far back as 2002 though) and I’m *guessing* from the results that I get that your annual return period is between 2 and 5 years.
The reason I say that is because I get similar annual returns as the ones you outline in the Welcome page (and the 2019 results).
I also get a similar asset allocation using a 2-3 year return window.
I don’t use any period of prior returns to find the future returns. and the predictors are dynamic, just slowly dynamic.
Actually, scrap that…
Further testing suggests that a 2 year window is too short and whipsaws the asset allocations all over the place.
4.5 years seems much better, including testing over a longer time period (since 1983) using just cash, gold and stocks.
If you don’t use prior returns, what are you using to calculate future returns?
Does the risk free rate and the efficiency hypothesis play a major role in your calculations?
Never mind, I found the answer (or at least most of it) in your post regarding maxing 401k.
Thank You for this great blog!