Steps on Multi-Target Prediction and Adaptability to Dynamic Input

Abstract

How should we react to dynamically changing inputs in various areas of computer science? This is one of the main questions we discuss in this thesis. The problem is present both in machine learning environment coping with massive amount of data available today and on the low level programming of computers.

One of the hot topics currently in machine learning are so called ensemble methods. An ensemble model is a collection of multiple divergent, often simple, base models. The variance of base models has been shown to give clear benefit to the predictive power over using a single model. Not surprisingly, ensemble methods also give new possibilities for coping with dynamic online inputs; we can simply reweight the base models to adapt.

However, in this thesis we are especially interested in ensemble methods in a specific framework. In many practical problems, we have multiple related attributes that need to be predicted. For example, predicting the growth of ora or biological composition of water are tasks that can be presented with multiple attributes that clearly relate to each other. Recently there has been some progress on methods that gain both smaller and more accurate overall models by making use of relations between the predicted attributes. In this thesis, we show that we can achieve both small and accurate models with a rule based ensemble method Fire. The method is extensively evaluated experimentally.

We also pull some strings together by showing how similar problems have been solved in separate areas of computer science. In machine learning, the problem of dynamically changing input has been studied under a term of concept drift. Similarly in algorithm and data structure analysis a notion of locality of reference has been present for long. We introduce a general framework that covers both of the problems and brie y go through the work done on both of the areas. We hope that by giving pointers to a bridge over the gap between the fields, researchers in both areas could be able to pick up some fruits on the other side.