Invest With the Smart Money (Or the Dumb)

Ever wonder what the smart money is doing in the markets? What are the people with the deepest pockets and best information buying and selling?

A small handful of analysts has found one way to tell: the Commitments of Traders reports. Devoted fans say they may be the closest thing in the public domain to a Holy Grail of market forecasting.

These obscure reports are issued free every week by the U.S. Commodity Futures Trading Commission. The COT reports list trillions of dollars of futures and options holdings in 100-odd markets - everything from the S&P 500 to orange juice, cruide oil, silver, Treasuries and the U.S. dollar. They include many of the world's largest investment firms, commodity producers and hedge funds.

Market Info for the Rest of Us

The government has been publishing various incarnations of this important data since 1924, when it first started to put out information on speculation and hedging in grains futures. Until fairly recently, it was available only by paid subscription. But since 1995, the CFTC has released the futures and options positions of traders for free each week.

Why is all this information important to have in the public domain? The CFTC says why itself: The data is collected and published to shed transparency on the markets and provide some much-needed accountability.

Which is why I think it's so valuable. It's being provided for the public to use. So let's use it. I see it as a great leveler. The big investment firms have their armies of analysts and banks of proprietary data and trading systems.

What do the rest of us have? The COT reports!

So why don't ordinary people know about it more? Why aren't we using it? Read on, below, and you will learn, my friend...

Cracking the COT Code

Unfortunately, the raw COT numbers aren't obviously usable for trading or investing. In most markets, they don't neatly correlate with market ups and downs. For 40 years, some of the world's sharpest economists and statisticians have tried to crack the secret of the COT data - without much success.

Many investors and journalists know about the COT data, but few seem certain how to use it. Even many analysts who have studied the data suggest it doesn't give trading signals per se, but rather a roadmap for possible future market direction. They often say it must be combined with technical analysis to come up with actual trades.

I wondered if technical analysis testing could be applied directly to the COT data - much as it has been applied to market prices. Because the data is so obscure, there's been little backtesting done on it so far. Partly as a result, many of the popular conceptions about how to follow the data are also not borne out when you examine them using statistics.

This presents an amazing opportunity for traders and investors to mine an as-yet-little-explored area, which has the potential to shed new light on market action that hasn't yet been arbitraged out of usefulness because of too many people knowing about it.

I eventually found there is indeed a way to have generated highly profitable trading signals solely from the COT data - signals that in testing reliably beat the underlying markets by wide margins. I've been using my COT-based system to invest my family's savings since early 2007. I created this blog shortly after to share my findings.

This blog was listed in Blogs.com's Top 10 Trading and Investing Blogs. It was also featured in The Globe and Mail in this column. I've also written about the COT data in The Montreal Gazette, Technical Analysis of Stocks & Commodities, Futures & Options Trader and the Canadian Journal of Technical Analysis.

I am publishing my signals - and details of how I developed my setups - for free because I believe the COT reports are an important public resource. Here's how we can find out what the market insiders are doing with their money - for free, without paying anyone the big bucks. This is exactly why the Commodity Futures Trading Commission publishes this data - to create more transparency in the midst of the market chaos.

I hope my findings will encourage you to look into this fascinating data too. I am sharing the details in order to show the power of this valuable government data, learn from readers' input and, with hope, inspire others to do their own research.

Some other important features of my COTs Timer trading strategy:

1) All COT

My calculations use COT data - and only that data - to time trades. No price data. No earnings reports. My research so far hasn't found any technical analysis indicators that would give me more reliably profitable results based on including price action.

Many analysts (especially those at mutual fund companies) like to say you should "buy and hold" your investments, sticking with them through thick and thin. While that might be the right approach for many people, it's not for me. As technical analyst Bill Carrigan likes to say, "You're in for a good time, not a long time."

2) All Traders

My system relies on data from all three groups of traders found in the COT reports - the commercial traders, large speculators and small traders. Some analysts tend to focus on the commercial category - the so-called "smart money."

I found that timing trades based on the commercial traders was not the most profitable approach in many markets. In fact, it was often better to fade - or trade opposite to - the large specs or small traders. In a couple of exceptional cases, it was actually best to trade on the same side as the large specs or small traders. Each market is unique.

3) Validation

I use automated systems to rapidly backtest combinations of moving averages, standard deviations and trade delays to tell when a group of traders has hit an extreme position that in the past led reliably to profitable, market-beating trading signals. See a detailed description of my backtesting process on my FAQs page. As a result, I use the net percentage-of-open-interest position of each group of traders to get signals - or the trader group's total open interest (long plus short positioning).

I also subjected my setups to various tests of robustness using detrended price data - the Student's T-test, out-of-sample testing, "walk-around testing" (testing "neighbouring" setups with slightly altered parameter values), Sharpe and Robust Sharpe scores, and Monte Carlo testing of both the setup and of "neighbouring" setups. The goal is to reduce the risk of using a setup that gave good backtested results merely by chance. The most profitable setups are almost never the ones that are the most robust.

4) Unique

Each of my trading setups is based on exactly the same parameters (standard deviation, moving average and trade delay), but the specific parameter values are unique to each market. That's because the backtesting shows clearly that each market has its own personality.