Machine Learning Tutorials — SVMs, RVMs & Kalman Filters

New Directions in Multiple Kernel Learning, NIPS

Currency Forecasting using Multiple Kernel Learning with Financially‑Motivated Features

Multiple Kernel Learning (MKL) is used to replicate the signal combination process that trading rules embody when they aggregate multiple sources of financial information when predicting an asset’s price movements. A set of financially motivated kernels is constructed for the EURUSD currency pair and is used to predict the direction of price movement for the currency over multiple time horizons. MKL is shown to outperform each of the kernels individually in terms of predictive accuracy. Furthermore, the kernel weightings selected by MKL highlight which of the financial features represented by the kernels are the most informative for predictive tasks.

Cumulative returns from MKL-based currency predictions (black) vs a benchmark strategy (grey) on EURUSD from 1998 to 2008, showing significant outperformance

Cumulative returns: MKL predictions (black) vs trend-following benchmark (grey), EURUSD 1998–2008.

Workshop on Applications of Pattern Analysis

Multiple Kernel Learning on the Limit Order Book

Simple features constructed from order book data for the EURUSD currency pair are used to construct a set of kernels. These kernels are used both individually and simultaneously through the Multiple Kernel Learning (MKL) methods of SimpleMKL and the more novel LPBoostMKL to train multiclass Support Vector Machines to predict the direction of future price movements. The kernel methods outperform a trend following benchmark both in their predictive ability and when used in a simple trading rule. Furthermore, the kernel weightings selected by the MKL techniques highlight which features of the EURUSD order book are the most informative for predictive tasks.

Limit order book predictions overlaid on bid/ask prices (top) and cumulative profit comparison across MKL methods (bottom)

Order book predictions and profit by method.

Tutorial Paper

Support Vector Machines Explained

This document has been written in an attempt to make the Support Vector Machines (SVM) as simple to understand as possible for those with minimal experience of Machine Learning. It assumes basic mathematical knowledge in areas such as calculus, vector geometry and Lagrange multipliers. The document has been split into Theory and Application sections so that it is obvious, after the maths has been dealt with, how to actually apply the SVM for the different forms of problem that each section is centred on.

SVM margin geometry showing the optimal separating hyperplane, margin boundaries, support vectors, and the weight vector

Tutorial Paper

Relevance Vector Machines Explained

This document has been written in an attempt to make Tipping’s Relevance Vector Machines (RVM) as simple to understand as possible for those with minimal experience of Machine Learning. It assumes knowledge of probability in the areas of Bayes’ theorem and Gaussian distributions including marginal and conditional Gaussian distributions. It also assumes familiarity with matrix differentiation, the vector representation of regression and kernel (basis) functions.

RVM regression: clean sinc function fit with relevance vectors marked (left) and robust prediction on noisy data (right)

Tutorial Paper

The Kalman Filter Explained

The aim of this document is to derive the filtering equations for the simplest Linear Dynamical System case, the Kalman Filter, outline the filter’s implementation, do a similar thing for the smoothing equations and conclude with parameter learning in an LDS (calibrating the Kalman Filter).

The Kalman Filter predict-update cycle: prediction step advances the state estimate, update step corrects it with measurements

Tutorial Paper

Machine Learning and Order Book Dynamics

A practical walkthrough of how high-frequency market data can be turned into predictive signals. The material bridges traditional market microstructure models with modern machine learning, showing how order book dynamics, volume features and probabilistic models can be combined into a unified predictive framework.

Limit order book diagram showing bid and ask volume at each price level with spread highlighted

Tutorial Paper

Fisher Kernel for High Frequency Market Microstructure Prediction

This note shows how generative models of order book behaviour can be transformed into features for discriminative models using the Fisher kernel. It works through Poisson-based volume dynamics and extends to Gaussian formulations, providing a concrete route from probabilistic modelling to practical classification features.

Fisher kernel feature extraction pipeline: observations through Poisson model to Fisher score derivatives

Tutorial Paper

Switching Autoregressive Hidden Markov Model

This paper explores regime-switching time series models where both the dynamics and parameters evolve over time. It outlines how autoregressive processes can be embedded within a hidden Markov framework, and details the forward–backward style inference required to recover latent states and regime structure from observed data.

SAR-HMM state transition diagram with two hidden states, transition probabilities and autoregressive observation model

Tutorial Paper

Bayesian Changepoint Model With Gamma Prior

A step-by-step derivation of an online Bayesian changepoint detection model, where structural breaks are inferred as data arrives. The note shows how uncertainty over variance can be handled via conjugate priors, leading to a tractable and interpretable framework for detecting regime shifts in time series.