Research Links: Boltzmann Machines

Neural networks have long fascinated us from a distance, but quite frankly, we’ve always had lower hanging fruit to pick: there’s always been a simpler model that didn’t require us learning a whole new thing. Let’s face it, as traders we are always looking for the highest level of abstraction to get a job done, subject always to the time/labor constraints imposed by the process of acquiring new knowledge.

Then we started working with graphs of correlation networks, as you may have noticed. The human mind is incredibly plastic at times, and working with this new abstraction changed our brains in a way that made it slightly easier to grasp the concepts presented in neural networks, thus making the knowledge attainable within our time constrained world.

As such, we wanted to document some of the things we have been reading, because maybe your minds have gone through a similar transformation looking at stock market visualizations. In particular, we have been fascinated lately by Restricted Boltzmann Machines, as defined here:

A Restricted Boltzmann Machine [RBM] is an undirected graphical model that defines a probability distribution over a vector of observed, or visible, variables v and a vector of latent, or hidden, variables h. [Oftentimes] we consider the case where v and h are binary vectors. An RBM defines a joint probability over v and h.

This is a type of neural network which can be represented as a bipartite graph, where visible nodes and hidden nodes forming two mutually exclusive groups. This model is part of a family of neural networks that ultimately arise from the deterministic Hopfield Network, with various refinements along the way, of course. So without further ado:

US stock market interaction network as learned by the Boltzmann Machine [arXiv]

We study historical dynamics of joint equilibrium distribution of stock returns in the US stock market using the Boltzmann distribution model being parametrized by external fields and pairwise couplings. Within Boltzmann learning framework for statistical inference, we analyze historical behavior of the parameters inferred using exact and approximate learning algorithms. Since the model and inference methods require use of binary variables, effect of this mapping of continuous returns to the discrete domain is studied. Properties of distributions of external fields and couplings as well as industry sector clustering structure are studied…

An Efficient Learning Procedure for Deep Boltzmann Machines [MIT]

We present a new learning algorithm for Boltzmann Machines that contain many layers of hidden variables. Data-dependent statistics are estimated using a variational approximation that tends to focus on a single mode, and data-independent statistics are estimated using persistent Markov chains. The use of two quite different techniques for estimating the two types of statistic that enter into the gradient of the log likelihood makes it practical to learn Boltzmann Machines with multiple hidden layers and millions of parameters. The learning can be made more efficient…

Applying Deep Learning to Enhance Momentum Trading Strategies in Stocks [Stanford]

We use an autoencoder composed of stacked restricted Boltzmann machines to extract features from the history of individual stock prices. Our model is able to discover an enhanced version of the momentum effect in stocks without extensive hand-engineering of input features and deliver an annualized return of 45.93% over the 1990-2009 test period versus 10.53% for basic momentum.

Deep Modeling Complex Couplings within Financial Markets [University of Technology Sydney]

The global financial crisis occurred in 2008 and its contagion to other regions, as well as the long-lasting impact on different markets, show that it is increasingly important to understand the complicated coupling relationships across financial markets. This is indeed very difficult as complex hidden coupling relationships exist between different financial markets in various countries, which are very hard to model. The couplings involve interactions between homogeneous markets from various countries (we call intra-market coupling), interactions between heterogeneous markets (inter-market coupling) and interactions between current and past market behaviors (temporal coupling)…

We were helped in our journey towards understanding by two great articles by quant blogger Dekalog showing the application of both a Restricted Boltzman Machine and a Conditional Restricted Boltzman Machine to the financial markets. Very cool.

Check out SliceMatrix: a unique tool for visualizing the stock market, including views of filtered correlation networks and minimum spanning trees


Want to learn how to mine social data sources like Google Trends, StockTwits, Twitter, and Estimize? Make sure to download our book Intro to Social Data for Traders

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 277 other followers


Lead image licensed under CC BY-SA 2.0 from Martin Röll

1 reply »

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s