Some Concerns When Optimizing Trading System Portfolios

We have talked about the risks of optimizing trading systems (forcing trading systems to conform to historical data), but, one of the subtler types of optimization occurs with portfolio selection. This occurs when trading systems are only shown tested across a handful or a small number of markets (or sometimes just one market).

The trouble is that what is typically done is that most all the available markets get tested, and then only those that performed the best get shown in the portfolio. This is an tremendous blunder, because the markets that performed best historically are rarely the ones that continue to be the best. What traders end up with is something that just worked well historically.

To prevent this tendency, we believe that the most robust method to see a system test is across ALL the available markets. A few will claim that different markets should be traded different ways, and to that we say Rubbish. Markets are always transforming, and a market that traded like market XYZ today will trade like market ZYX tomorrow. Until a system is effective enough to trade every market, it is likely a useless curve fit of the data.

For us in the commodities markets, we test roughly 80 markets. There are over 100 futures contracts that trade, but we do limit the choice to those that are liquid enough (have enough trading volume) to trade.

Testing this way does cause one difficulty. The difficulty is that traders can be trading numerous markets in the same sector at a given moment. Investors will need to have some sector risk control mechanisms in place. We like to be sure that the risk in a given sector does not exceed about 5% of the account equity.

If an individual creates a system that can effectively trade nearly EVERY commodity market and uses the identical rules for each market and gets tested over a long period, he may be on to something. Just keep in mind, the next time someone shows the results of a backtest ask him “How many markets does this test include?” If the answer (for commodities) is less than about 70 or 80, then be on your guard that this may be curve fit results. Once again, curve fitting tends to produce systems that ONLY perform well in the past.

We test all the systems we make available across nearly every tradable futures market. We could easily improve performance (in the past) by simply cherry picking the best markets, but we know this is deceptive data.

This also means they need to know their risk when entering a trade. It can be an efficient use of capital. Trading Systems An average $150 winning trade in the Eurodollar is not going to offset an average $2000 losing trade in natural gas!