Learning with complex representations

Many real world problems do not naturally fit within the context of attribute-value learning, the most prominent representation paradigm used in machine learning. Within the context of a number of Swiss funded projects we have developed a relational learning system that allows us to model learning problems where training instances can be arbitrary complex structures. The system relies on the availability of a limited number of basic data types, which become the building blocks that allow for the representation of arbitrary complex objects, e.g. trees, graphs. These basic data types are vectors, sets and lists. Data mining operators, namely distance measures and kernel functions, are subsequently defined on these basic data types. Any complex object is then represented as a combination of these data types, and the final data mining operators on the complex objects are combinations of the data mining operators defined on the simpler data types from which the complex objects are composed. The system utilizes metric learning and is able to learn appropriate combinations of representations for a given problem, if more than one exist, and/or combinations of distance measures, in view of optimizing predictive performance.

In the same direction we explored methods for kernel learning using well-known error bounds for SVMs based on the radius and the margin and developed a new variant of multiple kernel learning that achieves state of the art performance. Finally we also combined large margin metric learning and multiple kernel learning, and developed a general framework that allows that combination of different metric learning methods with multiple kernel learning.

Relevant publications:
Alexandros Kalousis, Julien Prados, Melanie Hilario: Stability of feature selection algorithms: a study on high-dimensional spaces. Knowl. Inf. Syst. 12(1): 95-116 (2007) (pdf).
Jun Wang, Huyen Do, Adam Woznica and Alexandros Kalousis. Metric learning with multiple kernels. NIPS 2011 (pdf).
Adam Woznica, Alexandros Kalousis: Adaptive Distances on Sets of Vectors. ICDM 2010, (pdf).
Adam Woznica, Alexandros Kalousis, Melanie Hilario: Adaptive Matching Based Kernels for Labelled Graphs. PAKDD 2010, (pdf).
Huyen Do, Alexandros Kalousis, Adam Woznica, and Melanie Hilario: Margin and Radius Based Multiple Kernel Learning, ECML 2009, (pdf).
Adam Woznica, Alexandros Kalousis, Melanie Hilario: Learning to combine distances for complex representations. ICML 2007: 1031-1038 (pdf)
Adam Woznica, Alexandros Kalousis, Melanie Hilario: Distances and (Indefinite) Kernels for Sets of Objects. ICDM 2006: 1151-1156 (pdf)
Adam Woznica, Alexandros Kalousis, Melanie Hilario: Kernels over Relational Algebra Structures. PAKDD 2005: 588-598 (pdf)
Alexandros Kalousis, Adam Woznica, Melanie Hilario: A unifying framework for relational distance-based learning founded on relational algebra, 2005, Technical Report (pdf)