*Using Pseudo Price Profiles in lieu of Additional In-Sample Data Testing*

We have emphasized, many times, the dangers of (over) optimization when considering the results of back-tests to determine the “best” values to use when choosing parameters for a system designed for forward trading/investment. In Part 2 of this study we presented the results of a single back-test (albeit averaged over ten different check date sequences to reduce the randomness introduced by this important variable) to examine the impact on performance of changing the look-back period parameters to be used in the ITA Wealth momentum based investment system.

However, in order to check for system robustness, we need to test the system using different price-time profiles – ideally, the same assets over different time periods and/or different assets over the same and/or different time periods. Unfortunately, insufficient historical data exists to enable us to select different (In-Sample and Out-Of-Sample) time periods for the same assets using ETFs as the assets of choice since these products have only become popular over the past ~10 years.

Our solution has been to take synthetically constructed “Pseudo Price Profiles” (PPPs) for each asset in our test portfolio in a modified Monte Carlo (MC) type simulation designed to retain temporal correlations between assets in the portfolio. Details of how these PPPs are constructed can be found in an Appendix file that is downloadable here. A sample of PPPs for one of the assets (VTI) is shown below:

As can be seen in the above figure, the starting and end points of each PPP are the same, but the routes from start to finish are significantly different. This is similar to what we might expect from a MC randomization of daily return data – with the significant difference that * we have retained the temporal intermediate term correlation relationships* that would be lost by adopting a standard MC randomization approach.

**Using Pseudo Price Profiles as an alternative to using multiple In-Sample Price-Time Series**

Having constructed PPPs for all assets in our portfolio list we then use these profiles to run “stress tests” on our momentum system. Our study parameters are the same as used in Part 2 of the study:

*Study Parameters*

1. Rutherford Portfolio: VTI, VEA, VWO, TIP, TLT, PCY, VNQ, RWX, GLD, DBC and SHY;

2. Ranking Parameters: *ROC1 = 20-170 trading days in steps of 5 days*; *ROC2 = 20 -170 trading days in steps of 5 days*; Volatility = 63 day Mean Variance/SD;

3. Weighting: ROC1 = 50%; ROC2 = 30%; Volatility = 20%.

4. Review Cycle: 23 -2/+5 (Trading day) Review cycle

5. Top 2 ranked assets (and ties) selected and Equal Weighted;

a) 2 Assets ranked higher than SHY – 50% Asset 1, 50% Asset 2;

i) 2 Assets tied ranked 2^{nd} – 34% Asset 1, 33% Asset 2, 33% Asset 3;

b) 1 Asset ranked higher than SHY – 50% Asset 1, 50% SHY;

c) No Assets ranked higher than SHY – 100% invested in SHY;

6. Test period: 06/30/2006 – present

7. EMA smoothing: None – raw data of daily closing prices

8. Number of Tests: 10 PPPs (shuffled data)

Each of the ten tests (item 8) represents a 14-hour computation covering the variation of parameters in items 1-7. These tests are then averaged, together with the results obtained using raw data, (reported in Part 2 of the study) to generate summary performance charts.

**Total Returns**

In heat map format the results of changing the ROC1 and ROC2 look-back period values (item 2 above) are summarized in the figure below. Recall, from Part 2 of the study, that we are looking for areas of high density blue coloring to indicate preferred parameter values.

**Volatility**

Of course, there is always a trade-off between return and volatility/risk. Average volatility over the eleven (10 PPP + 1 Raw data) tests is reflected in the heat map shown below:

Notice the relatively high (unfavorable) volatility along the top left to bottom right diagonal of the above chart. As described in Part 2 of this study this diagonal represents results obtained using a single lookback period and gives us a first hint that there may be significant benefits in using multiple momentum periods to reduce volatility since volatility tends to decrease as we move away from the diagonal.

**Risk Adjusted Returns (Sharpe Ratio – excluding risk-free-rate)**

When we combine returns with volatility and look at risk adjusted returns (Sharpe Ratio – excluding risk-free rates) we get the following picture:

Or, viewed slightly differently…..

In the above figure we plot the Sharpe Ratio as a function of ROC2 look-back period for selected look-back values of ROC1 between 95 and 115 (trading) days. We note that there is a range of ROC1 between 95 and 105 days and ROC2 between 45 and 75 days (within lavender oval area) where we calculate consistently high return/risk ratios. Taking the center of these ranges, i.e. ROC1 = 100 and ROC2 = 60, therefore gives us a little flexibility to be slightly off an optimal setting for a specific set of market conditions without significantly compromising performance. This is what we are looking for in a robust system. The position of the ROC1 = 100, ROC2 = 60 setting is highlighted in the heat map by the white-framed square.

An important thing to note is that the heat map is a summary of data for ROC1 weighted at 50% and ROC2 weighted at 30%. Therefore, choosing an ROC1 value of 100 days and an ROC2 value of 60 days tells us that the longer term momentum look-back period should be weighted heavier than the shorter term momentum period in the Ranking spreadsheet (current default is 50% x ROC1, 30% x ROC2). Note that the default settings are not bad – still generating return/risk values in the top two deciles – but reversing these weightings would appear to provide more robustness (less impact of being slightly off in optimal look-back periods).

**Other Performance Metrics**

Although perhaps not quite as important a consideration as returns and risk there are other parameters that investors may wish to consider. The first of these is maximum draw-down, and this is summarized, for the eleven price-time profiles, in the following heat map:

Again we note that the values along the top left to bottom right diagonal, representing the use of a single look-back momentum period, suggest higher resultant drawdowns compared to a dual look-back period momentum system represented by combinations away from the diagonal.

The final metric that may be of interest is the turnover or number of trades generated by the system:

As might be expected, turnover is, generally, reduced by increasing the momentum look-back periods – but, of course, this comes at the expense of returns.

**Conclusions**

In the above series of back-tests, although we have not been able to use real-time In-Sample data to test the system under different market conditions we have simulated such varying conditions by “shuffling” real-time data to generate bullish, bearish and sideways markets at different points in the temporal cycle.

Based on the results of this extensive series of tests we infer that a dual look-back period momentum system with short-term momentum measured as the 60-day Rate Of Change and long term momentum measured as the 100-day Rate Of Change should represent a robust system that can accommodate changes in market conditions. In more general terms we might suggest that using any look-back values in the 45 to 75 (trading) day range (or 65 to 109 calendar days*) for short-term momentum and in the 95 to 105 day range (or 138 to 152 calendar days*) for longer term momentum would constitute a robust system. In addition, weighting the momentum values in favor of the long-term rather than short term look-back period may result in a system (within a “fatter” high density area) with superior high performance tolerance.

* We use calendar days in the Kipling 8.0 spreadsheet.

One thing that should be mentioned here is that these results do not necessarily contradict Antonacci’s contention that a twelve month look-back, single period, momentum system may be superior to shorter term look-back, single period, momentum systems over a 40 year time frame. However, we should remember that Antonacci has not considered the impact of check date “noise” on the long term returns and used only a fixed month-end adjustment date.

In the end, an investor will need to consider the performance of a system within their personal frame of reference. For a young investor, with 40 years to retirement, volatility and drawdown may not be as important as to an older investor approaching or in retirement, where preservation of capital may be far more important.

One metric we might use to assess “risk of loss” is to look at the rolling 12 month return of the system:

The above figure shows the rolling 12-month returns of a typical 2-asset portfolio constructed from the Rutherford asset list. As shown in the above figure, 12-month returns were negative in only 8 months of the ~100 month test period and the maximum (12 month) loss was ~10% (in 2008).

If such charts are plotted for systems using longer look-back periods, maximum losses tend to be greater and last longer (maximum number of consecutive losing periods in the above graph is 4 months). Of course, the reverse may also be true – positive returns may be greater and also last longer – so we need to choose the scenario with which we are most comfortable.

In future parts of this study we will extend the analysis to look at the volatility variable and to use the new parameters with different assets and over a longer time period.

Herb Haynes, Ernie Stokely, Lowell Herr, David (HedgeHunter)

The post How “Robust” is the ITA Wealth Momentum Strategy? – Part 3 appeared first on ITA Wealth Management.