Statistically Sound Machine Learning for Algorithmic Trading of Financial Instruments: Developing Predictive-Model-Based Trading Systems Using TSSB

First, I think it's important for anyone reading this review to have some understanding of the person who's writing it. I'm a semi-retired individual trader who works full time at this. I have an EE degree, like math & science (obviously) but have very little statistics background which, as we all probably know, is more and more important all the time. With that in mind please read on. First, as others have said, if you view this book combined with the software as a single purchase then personally I think it's hard to imagine a much better deal. As someone who's primarily a Linux Open Source user I'm used to getting 'free' software. I can tell you much of it comes with no support, no documentation and can chew up loads of time trying to figure it out. While the TSSB software doesn't cost any money to run on your system (or multiple systems) if you're someone with my sort of background you'll decide very quickly that valuing your own time even minimally the book isn't really that expensive even if you never end up using the software for anything other than a learning experience. However if you invest the time to learn what's here then I suspect you will ending up using TSSB. I believe I will. Beyond that the authors offer paid-for support for things you want to tackle with their help. OK, yes, much of the book is just a software manual documenting all the command you can use in TSSB scripts. However, there's a LOT of additional writing in the book that happily is quite readable. The authors are to be commended for taking the time to make the software approachable to traders that don't necessarily have their level of statistical expertise. (I.e. - ME!) There's a lot of information in here I've run into before, but there's probably more info that I haven't gotten into but have been able to generally understand using Wikipedia and other free sources. As with all of these things YMMV. As best I can tell so far the way the book approaches trading system development is well thought out and, coming from a background of TradeStation/EasyLanguage, R & more recently MatLab, TSSB itself has a lot to offer that aren't easy or inexpensive to do in those tools. First, trading systems are developed across many markets with the means to trade only a percentage of the markets at any given time. TSSB facilitates finding indicators that work well, or at least similar, in multiple markets. It supports developing more types of trading models than I'm likely to use in the next year or two. It has built in walk-forward testing at the day, month and yearly level and there are likely ways to use other periods. Best of all so far the software seems quite stable and reasonably fast. I've just completed a 13 year monthly walk-forward on 88 markets using CPU-based processing in about an hour. TSSB supports GPU-processing but I haven't tested it yet. Lest I be accused of gushing let me also be clear about what are likely to be issues for some. First, TSSB is not a trading platform and actually there isn't a direct path to get TSSB models into EasyLanguage or R without considerable work on the user's part. (Read the TSSB website for more on that) Getting TradeStation data into TSSB wasn't hard but did require that I write my own EL indicator to get it formatted correctly. I want to thank the authors for their support getting past that hurdle. Depending on where you get your data this may be an issue for you. Next, as you'll learn if you dig into TSSB, the indicators in TSSB have considerable intellectual property in them which isn't made available to the user. (At least not as code that you could use anywhere else.) What this means to me is that for now my models in TSSB must remain in TSSB which doesn't easily support automated trading. However TSSB does support reading in and studying indicators I have written in EasyLanguage and, if they work well, would begin to reveal a path toward automated trading down the road. Not direct, and not easy, but that wasn't a purchase requirement for me at all. Again, YMMV. As there is no officially supported 'user's group' of any kind I've created a Google Group where other people interested in TSSB might join me for conversations. This is completely unofficial and no one should look for support from TSSB's authors in this forum. Search it out on Google Groups if you're interested. If you're considering purchasing this book I'd recommend going through the free Kindle preview in depth. It as some scripting code that will give you a good feel for whether these tools are for you. Run the code - it works - and study the results. If you purchase the book I'd recommend and your an organized individual then you may want to jump to the end of the book to the section entitled 'Developing A Stand-Alone System' and working through what is presented there. This will lead you to read about commands you need them but will also start to reveal more of what TSSB can do. For now TSSB is, for me, a way to study what indicators to use, what markets to use them on and when to potentially trade or not trade. I'm happy with that but you may or may not be. That's up to you.

Let me start this review by stating that at this point I have only limited experience using the TSSB software. So I'm not yet in a position to offer a credible review of the program, that will have to wait until later. But until then, let me offer some comments on the book, which is really what you are buying here, since the software is available for free downloading. Originally I was a little skeptical about paying an amount in the $120 range for a book. But what got me over that were two things: first, I had bought David Aronson's earlier book, Evidence Based Technical Analysis, and I loved the scientific rigor and creative thinking that he evidenced there. And secondly, I figured that if I looked at this book and the TSSB software as a package (which they are obviously intended to be), $120 is dirt cheap for almost any kind of trading software, let alone for a program that claims to be able to do what this one does. So I bit the bullet and ordered the book, it has turned out to be well worth the money, without even considering the software. The reason is the wealth of insights that Aronson and Masters supply throughout the book. If you are (or want to be) a successful systemic trader that uses sophisticated models to generate your trading signals, then I believe that this book is no less than required reading. Some of the topics that you will find addressed in this text include the following: - What are the reasons to use walk-forward testing over cross-validation in testing your model's performance? - Why are low (or even negative) values of R-squared perfectly fine for trading models? - Where is the best predictive information found in a predictive variable? - How can you use synthetic (permuted) market data to validate your models? - Is it better to have a strong model that uses weak predictive variables, or the reverse, and why? - How do developers of modern trading systems select the most effective predictors from among a set of candidates? - If I'm building models to trade at tomorrow's open and exit at tomorrow's close, how far back should my predictive data extend? 5 days? 20 days? 100 days? - Why is visual examination of a time-series plot of every predictor mandatory? - How important are committees in modern trading system design? - Are you better off having a single model for long and short trades, or should you have specialized models for each? - In addition to normal development of a model by partitioning the test data into out-of-sample and in-sample sets, what are five reasons why you sometimes might want to also try developing the model using all the data as in-sample, without saving any data for testing? After reading this book, you will have the knowledge to determine if your trading system meets the following two conditions: a) the chances that it has good performance just due to being lucky are small, and b) you know what the expected level of future performance is, and it is good enough to make it worth trading. And honestly, if you have that knowledge, you are probably ahead of 99% of the traders in the market today. (that's just my opinion, not the authors') As loaded with powerful information as this book is, I do have a few quibbles. The quality of the writing is very good, but I do think it could have used more editing, specially in getting the material organized a little better. And there are a couple of places where it is mentioned that there is the need for additional material, which will be included in a forthcoming edition of the book, which I found a little disconcerting. But overall, if you are a serious system trader or developer (or both), I believe that you owe it to youself to thoroughly read and study this book. I think you will find it contains many important and useful insights that you won't find elsewhere. These are the kinds of things that can make a difference between a middling trading system and an outstanding one. Even more importantly, they can give you a high degree of comfort that you can rely on your trading system, which in my experience, is worth its weight in gold.

I like the book but it is not what I expected. First to be clear this books main focus is TSSB which is a great concept but the software leaves much to be desired. Many of the indicators and systems mentioned in the book can't be used live because the author doesn't give you the source code for the indicators, just high level descriptions. TSSB does not support live trading so it serves as a tool which is useful for examples but nothing more. The ideas in this book however are very valuable and quite useful if you are able to build your own trading platform. You will lack some of the results mentioned in the book due to the lack of indicators (FTI for example) but this provides a nice foundation.

I tend to agree with some of the other reviews, this book is basically an instruction manual of the authors free software, which is very powerful. In hind sight I probably wouldn't buy this again at the price advertised. If it were much cheaper then definitely

Wow, what an amazing book! I have now read this book cover to cover 3 times over the last month while getting my hands dirty with TSSB. This book gives implementation level detail of how to create and test predictive models for any class of market (equity, forex etc...). The implementation is done using the free software called TSSB. The use of TSSB requires learning its associated command line language, which is very very simple to learn yet extremely powerful. As a point of comparison, to program the same models in the R programming language would require perhaps 10 or 100 times more lines of code (I know this because I programmed similar models in R before trying TSSB). Upon first use, it took me all of 1 hour to create my first model in TSSB and it was off to the races from there – having since computed over 900 predictive indicators and over 100 models. A great feature is the ability to combine models in ‘committees’ or ‘oracles’, which is common practice to achieve optimal results. One of the best features is the ability to perform a multitude of tests (boostrap, monte carlo, and permutation) that will greatly aid in ones confidence that the proposed model will in fact generalize to unseen data. In particular, the permutation test separates and measures a models ‘skill’ in comparison to its measured bias as well as luck due to trend. It is my hypothesis that the majority of hedge funds do not do such tests and is the reason why so many fail so quickly. Another great thing about the book being implementation detail is that much of the theory and best practices can be abstracted out of it. For example, it is one thing to read about ‘stationarity’ (which I have read extensively about through complex greek lettered equations and proofs in other books) yet it was still not completely clear until I actual saw its impact with my own eyes through performance of models I created. Sometimes (often, in my case), getting to the implementation level detail (developing and testing and reviewing results) is the best way to understand a concept. Additionally, I learned exactly how and why serial correlation could dupe someone into thinking a lucky model had any skill. There also subtle hints given throughout the book (e.g. committees often perform better than individual models, regime specialization per model often performs better, models tend to perform better when specializing in long-only or short-only contexts – then combine to either a portfolio or committee, more esoteric indicators (e.g. non-trend type) are better used in more complex models (non-linear), use of more than 3 indicators often overfits a model (a big time saver too!). Other great things about TSSB (and the book): Most of the indicators are scaled and normalized to an extent, which helps take care of those little things that must be done before getting to the actual modelling. There are also specific functions which help create one model for multiple markets (market regression, cross-market normalization, pooling variables, use of indexes etc...) – often such implementation complexities are overlooked in theoretical books. Indicator selection is very clearly explained with the use of tests such as Chi-Sq, Non-Redundant Indicator scanning, visual examination of indicator-target relationships, find groups, exclusion groups, and model stepwise selection. It is worth mentioning that currently TSSB is only a research a tool; the models created cannot be simply exported to a trading program (e.g. TradeStation) – but apparently this is planned for a future release. In the mean time, Perhaps this is my biggest gripe and fear – I will create an amazing system but be unable to implement it in real life. On the other hand, with some time and help, it should be possible to re-create a lot of functionality in R (and use a trading program that integrates with R) or directly re-create it in a trading program that supports predictive functions. I initially found out about the book over at the CssAnalytics blog. Shortly after, I bought it. Here is why, the book fills a HUGE gap: Over the last 6 months I have taken an interest as a hobby to learn predictive modelling and apply it to the Forex market. During that those months, I learned the basics of the data mining process, programming and how to implement predictive models in R, and how to apply it to Forex (outside of by day job) . It's been a lot of fun and have gotten somewhat successful results so far. I learned from the following materials in order: - "Data Science for Business" by Foster Provost & Tom Fawcett from my MBA alma matter NYU-Stern. This is a fantastic and clear book written which describes data mining process and various models from a conceptual and logical perspective. - Online course at Stanford from Professor's Hastie and Tibshirani, pioneers in the field which blends theory with practice to show how to implement various classification and regression models using R. - "Applied Predictive Modelling" by Max Kuhn - an amazing book on implementing predictive models in R - mostly using the R caret package, which is wrapper to over 100 predictive models in R that essentially automates re-sampling (bootstrapping, cross validation, data splitting) as well as model evaluation and comparison. - White papers that outline some predictive models in forex by generating a bunch of technical indicators and then running predictive models on them (SVM, Random Forest etc..) As you can see from the above, I have learned from some really smart people how to 1) Create and evaluate predictive models 2) Program in R 3) Apply to markets such as Forex Of the above, I have gotten a fairly good grasp on #1 and #2. This book fills the gap on #3! The following points in the blog interview that caught my initial interest in the book: "It is possible for a model to have poor error reduction across the entire range of its forecasts while being profitable for trading because when its forecasts are extreme they carry useful information. It is more appropriate to use financial measures such as the profit factor which are all included as objective functions within TSSB." - I literally just had that realization of the effective of EXTREME forecasts, just this last week - by plotting the residuals vs. prediction made this very clear. IF I read the book, I probably wouldn't have needed 6 months to discover that point! Though, the process of discovering it myself was fun too. "Even the best conventional technical indicators have only small amount predictive information. The vast majority is noise. Thus the task is to model that tiny amount of useful information in each indicator" Wow, is that true! I'm super interested in learning how to model just the useful information in each indicator. Cool concept. "In my opinion, the way to differentiate or uncover real opportunities currently lie in the clever engineering of new features- such as better indicators." I've been using R's TTR package as my sole source of indicators. While there a LOT of indicators in the TTR package, I'm very interested in the 100 or so you mentioned in the software -------------------- From an excited modeler. Cheers!

I haven't been able to put this book down. I haven't had the chance to download TSSB yet, but it is clear that this book and the TSSB software are key tools I have been seeking for developing and validating my own trading system (and have been in the early stages of developing myself). However, this book and the software are much more than I would have ever accomplished on my own. The statistical validation tools alone are worth the cost of the book. So much of what is taught in trading classes and discussed in trading rooms is based on "beliefs" about market behavior. I have learned in trading that most of those beliefs are not validated by anything other than the fact that a large number of people repeat the beliefs as revealed truth. This book cuts through beliefs and myths by providing mathematical and programming tools for individuals who want to develop and test trading algorithms against market data. There's no bulls*** about the performance over the last two years, no claims about profits made by trading a specific system, and nothing more to buy (unless you want specific support on the software). The trick in trading is to see and act on signals in a timely fashion without getting distracted by the noise. Tools for smoothing the signal or for filtering out the high frequency noise (e.g., indicators and oscillators provided by all trading platforms) provide a nicer apparent signal at the cost of time lags that generally allow the trader to see the entry only after it has passed. For the most part, these indicators are not independent of each other, and often they don't provide consistent indications of entries and exits. To a large degree, trading is a combination of driving forward while looking in the rear view mirror coupled with input from the bumps you hit while driving. Aronson and Masters have helped the trader tremendously with very sophisticated, robust, and what appear to be easy to use tools for evaluating trading models against market data. They point out pitfalls of traditional back testing methods and teach more robust (I think I mean "statistically sound") methods for ensuring that a trading algorithm is more than a burst of good luck. I don't see how this book and the software on which it is based can fail to help a sophisticated trader.

I am an engineer by education and learned the concepts of pattern recognition and predictive modeling in graduate school and then applied them in the software industry. But that was almost 18 years ago. Subsequently, my job has been managerial. During the last 5 years, my personal interest has gravitated towards financial trading but I can't claim to have any background in the financial industry. I'm a discretionary trader but ready to move into automated trading which necessitates that I dust off my basic modeling skills. I was looking for a book that would bridge the theoretical aspects of predictive modeling/ machine learning and ground it in the realities of financial time series data. For my needs this is a very timely book. Amazon prime delivered this book 8pm and I could not put it down until midnight! In one reading, the authors managed to tie multiple conceptual loose ends in my head and laid out a practical roadmap for how I could approach building models for my own use. The authors describe this is a guided tutorial to predictive modeling and they do this very well. Their language is clear and precise. They don't let the theory get in way of practice. They sprinkle tasty tidbits of their experiences for what works or doesn't when considering differing approaches. This book is based on their TSSB software. I was able to run through the first tutorial as described. [Note, at the time of my writing the review, the authors were working through copyright issues in order to place the sample data on their website. I used my own data, but they were extremely responsive to my support request.] If you scan the books table of content, you'll see that the TSSB software is incredibly rich in core models (linear and non linear regressions, trees, forests, committees, ensembles etc.) and indicators. However, other aspects of the software are work-in-progress and this is fully disclosed at the TSSB website where you can download the software and user manuals for free as open source. It is clear to me that the authors David Aronson and Timothy Masters are truly masters of their domain and also have a gift for teaching in a manner that make the reader feel smarter by the page. This book is a bargain given the high quality of editing, attention to detail and a rich functional software. I wish the authors continued success!

It's a users manual for software they produced.

The first time I read this book, it was mind boggling! I thought wtf. The second time I read the book more carefully and it was again mind boggling: whoa. I had to realize how many things swirling around in mind was brought to a point, how much I learned in one weekend and even more important: to stand on the shoulders of giants and see how much I can still learn. Definitely I will read the book once more! It has been worth even cent and every minute invested in it. Brian

Included free TSSB software does not download. Book is useless without software. This book is an instruction for the software

There is very little maths in this book. It is a tutorial on how to use the software TSSB, but you don't need to buy the book alone for that reason. It has useful ideas on how to engineer features you may not have considered before. That you won't find in other quantitative trading books. You buy it because it gives you a basic framework for how to use statistical learning prediction models to make trading strategy decisions. Filtering/boosting on existing trades (Ch 3) is also an interesting topic to consider in your tool kit for trading modeling building. Lots of interesting aspects on feature engineering using price data alone, and the discussion on ensemble methods for prediction is useful. Interesting approaches suggested for feature normalisation.

Anyone who builds automated trading systems needs to read this book. Depth and breadth in abundance, all the usual bases are covered. And more importantly, a lot of the unsual bases too. these being the steps and stuff which we either skip or were shamefully and blissfully unaware of ! Too many books focus on models and their endless flavors. Here, the technical focus is on your data, your indicators. It's a very good read, but it's not a flick-through or a book to be skimmed. The software alone is worth the price with some zeros appended in time-saved alone. Take your time over this one, benefit from 2nd and 3rd order insights.

Das Buch ist sehr gut! Ich habe es bereits zum zweiten Mal durchgearbeitet und konnte mir nochmals einige wichtige Erkenntnisse erarbeiten. Allerdings ist dieses Buch nicht für einen Einsteiger geeignet, da hier teilweise Konzepte nicht erklärt, sondern vorausgesetzt werden. Wenn man jedoch eine solide Grundlage in Bereichen wie Statistik, Ökonometrie oder auch im algorithmischen Handel hat, dann ist dieses Buch jeden Cent wert!