Algorithmic Trading: Can Machine Learning Technology Automate Back-testing?

Algorithmic TradingAlgorithmic Trading: One of the outgrowths of the recent “big data” craze has been a renewed interest in machine learning systems. This is, in fact, the future of dedicated AI research, and its implications are enormous. At present, Google is using machine learning algorithms in its search engine so as to better understand what web pages are about, topically, and can thereby return better search results in response to a keyword driven query, but that is just the beginning.


The Current State of the Algorithmic Trading Industry

This is a nascent industry, no doubt, but the growth potential is enormous, and the number of uses for machine learning systems is virtually unlimited. In addition to the above, Google is also currently running an algorithm that tracks the behavior of its employees, and can predict with a high degree of accuracy which ones will be leaving the company within a 30 day window.

Retail stores are crunching massive amounts of data to respond in real time to moment-by-moment surges, dips, or other changes in demand by the customers while they are shopping. They’re also using the results from all this data crunching to empower their employees to make on-the-fly store layout changes in response to consumer demand at that moment in time. Even better, it is working. The stores making use of these systems are reporting in-store sales increases ranging from 3-10%. Those are compelling results.

Local law enforcement agencies are getting in on the machine learning craze as well, though here, we have begun to see some public unease. Comparisons have been drawn in national US Newspapers about the eerie similarities between what law enforcement is doing with machine learning to predict crimes before they happen and the dystopian Sci-Fi movie “Minority Report,” and those comparisons are not incorrect. Nor are the additional comparisons that have been made with the hit TV show “Person of Interest.” These are valid concerns specifically because of the accuracy of the machine learning systems. They are extraordinarily predictive.


An Opportunity For Algorithmic Trading Systems?

Given the success that these other segments of the economy are already enjoying with big data and machine learning systems, the question naturally arises. Is there an opportunity here for algorithmic trading, and what form might it take?

One possible avenue we could explore would be buying access to the large data sets collected by companies to better predict their future price moves. Done consistently across the span of a whole industry, this would be a powerful, compelling advantage over competing automated trading platforms that did not have access to the data sets.

Another possible use would be to tie machine learning systems to existing back-testing structures, such that at least part of the back-testing could be automated. Let the machines do the trading, and let them also see to at least some of their own testing. It’s a compelling, and also slightly terrifying idea, but it may have some merit. A company wanting to test those waters should probably tread carefully and design a prototype system to be tested extensively internally before offering it to the general public. Is it an ideas whose time has come?