by Joe Bridges, Senior Broker & CTA
Data mining is a common method used to uncover statistical correlations in many industries. It is used by retail merchandisers to determine the shopping and purchasing patterns of their customers, by manufacturers to determine production and inventory schedules, and by many stock, futures, and forex traders to uncover repetitive patterns in price action.
When most traders think of price patterns, they tend to think of 1-2-3's, pennants, flags, head & shoulders, double tops & bottoms, etc. Today, using data mining, we'll be looking at price patterns from a slightly more scientific approach. Specifically, I'm going to explain how to effectively use data mining in an effort to uncover recurring patterns in the S&P 500 market. I'll also show you how you can incorporate short-term closing patterns, combined with nested 1-day defining pattern bars, to help refine the process. In addition to pattern analysis, I'll further distill the process using an entry technique known as a volatility breakout, which will act as a filter and help to uncover high probability market tendencies that may be tradable.
Before we move into the S&P example, however, it's important to understand the fundamentals by which a study of price patterns should be performed. In short, there are 4 important areas that should be addressed prior to deeming any study of price patterns valid:
The sample of data studied needs to be large enough to reflect Bull and Bear markets, yet contemporary to current conditions. For the purposes of today's study, a little over 10-years of data is used. The sample spans back to the first trading day of 1995, which provides a little over 2,500 trading dates.
Over a long period of time, such as the 10-year period I've chosen, there are potentially 10's of thousands of patterns that could be studied with refinements down to the day of the week, month and year, thus creating highly optimized results. Yet, the more highly optimized a study is, the less likely it is to be repeated in the future. The purpose of data mining is to uncover legitimate, potentially profitable patterns that will continue to occur with reasonable frequency.
There should be at least 100 samples containing the base pattern, or argument of the study, for the study to be legitimate - I actually prefer a sample of 150 or more. There may be refinements of the base pattern to further identify price action, but the premise of the base study should produce an adequate sample size for the study to be deemed valid.
Price patterns can change for a variety of reasons, and sometimes for no apparent reason at all. The following are a few potential causes:
When done properly, data mining price patterns can offer valuable insights into past and probable future market behavior and provide a trader with the following:
NOTE: In the example shown below, no deduction was made for commissions or slippage. This is not an effort to build a mechanical trading system, but rather an attempt to identify recurring patterns or themes in a specific market, and then determine if they are tradable. Furthermore, this is just one example from a much larger body of work that began with closing patterns only and ultimately expanded to include closing ranges, next day's opens and identity bars to further refine market conditions.
In today's example, I use a 3-day closing pattern as the base pattern. This pattern consists of an up close 3 bars ago followed by 2 lower closes. It will look like this {+ - -} in the example.
It will be assumed that a position is taken on the open of the day following the closing pattern + 40% of the previous day's range for a "long" position and the open of the day following the closing pattern - 40% of the previous day's range for a "short" position. Positions are exited at the end of the day on the close and no protective stops are used.
Now, we're going to define the last bar by 1 of 6 one-day definitions.
When looking at the table above we want to compare Buys and Sells from same and similar patterns and compare the results.
A good example of what to look for can be found in the NR4 & WR4 patterns. After an NR4, "Buys" after the 3-day closing pattern produced reasonable results with a relatively high W-to-L ratio of 2-1, while "Sells" actually produced a higher winning percentage - 66% to 59% - but was actually produced negative results because of a reverse W-to-L ratio of 1-2.
Disparity can also be found after WR4 days where the "Buys" produced positive results with an attractive W-to-L ratio while "Sells" produced the most negative results of all. In fact, the only short-term pattern that produced with any consistency at all was the "Doji".
Remember, when looking at pattern analysis of this type, it's important to not only look at the Position Analysis and the % Profitable, it is also very important the look at the number of Points produced and the Average Win to Loss ratios as well to determine how the profits were produced. There have been many times in the past that patterns have been touted as high percentage winners that actually produced negative net results much like the "Sell" after the NR4 shown above. So, ensure you have all the information before attempting trades based on pattern analysis.
The S&P chart below shows what a few of the recent trades have looked like. Notice that the signals tend to work in the direction of the dominant trend.
A note of caution: Just because a pattern has done well in the past, it does not mean that it will continue to do so in the future. It is important to research a series of patterns across several different markets in order to trade them effectively.
The object of data mining and pattern analysis is simple - to determine exactly what a given market has done in the past using a reasonably specific set of criteria to test against. After running many sets of tests over a wide variety of criteria, I've found that price patterns previously purported to be bullish or bearish were, in reality, often not. In many cases, results were actually reversed.
In summary, data mining can provide legitimate insight into market behavior that would not be possible by scanning charts with the naked eye. All too often, traders hold opinions about markets or have perceptions about price patterns that are simply incorrect. With the speed and power of today's computers and software programs, there is no reason to trade in the dark with incomplete or incorrect information.
PLEASE NOTE THAT THERE IS AN INHERENT RISK OF LOSS ASSOCIATED WITH OPTION CONTRACTS. OPTIONS TRADING IS NOT SUITABLE FOR ALL INVESTORS. OPTIONS CAN AND DO EXPIRE WORTHLESS. IF YOU PURCHASE A COMMODITY OPTION, YOU MAY SUSTAIN A TOTAL LOSS OF THE PREMIMUM AND OF ALL TRANSACTION COSTS.