An Adaptive Framework for Learning over Composite Representations
National Framework
Funding by the Swiss National Science Foundation.
Description
This project will build up on the work performed within a previous
four years project funded by the Swiss National Science Foundation. Within that project we explored the
area of relational learning over complex objects, such as graphs and trees, where the
learning paradigm that we followed was that of distances and kernels.
Having developed a complete relational learning system that allows for adaptive modeling
of learning problems with a variety of representations and learning operators we identified
two main challenges. The first challenge
is the selection of the appropriate representation of the learning instances;
a correct choice can render the learning problem much easier, if not trivial. The
second challenge, tightly coupled to the selection of the appropriate representation,
is the application of the appropriate machine learning operators over the
selected representation. Ideally, both of these components should be directly
determined by domain knowledge and application requirements. However, even
in the presence of domain knowledge, such choices can be far from obvious. It
is clear that there is a need for techniques for automatic selection of representations
and operators, as this would enable more effective and robust learning
from richly structured data.
In this project we take the view that the selection of the appropriate representation
and data mining operators should be addressed within the learning
process. We focus on distance- and kernel-based learning paradigms. We will
pursue two research directions. The first concerns the exploration and development
of new techniques for learning and combining different relational representations
and operators. The second research direction concerns the definition of
adaptive distances and kernels over sets, this issue is at the core of many state
of the art relational learning systems. More specifically:
- Learning over Representations and Operators: We will explore three directions.
In the first we assume that a number of different relational
representations and data mining operators over these representations is
given and the goal is to learn the best combination. Within that context
we will explore issues such as the definition of appropriate cost and regularization
functions. In the second direction we will try to automatically
extract discriminative relational representations, that will become a part
of the combined learned representation, viewing the problem as a search
problem. Finally, in the last direction we will examine the applicability of
our system to problems with partial labeling information.
- Adaptive Operators on Sets: Most of the existing operators on sets, whether
distance or kernel based, assume a given family of mappings between the
elements of the sets, and use some optimization procedure to establish
an optimal mapping within this family. This assumption corresponds to
a form of representational bias which might be more or less appropriate
for a given application. We will try to relax this representational bias by
introducing more flexible set operators. We will examine two approaches,
in the first and less ambitious we will still be restricted within a given
class of family but we will include as a part of the optimization problem
the establishment of the appropriate representation of the elements of the
sets. In the second, we will examine ways to lift the constraint of specifying
the family of mappings and try to learn the mapping directly from
the data.
Relevant Publications
WOZ07 Adam Woznica, Alexandros Kalousis, and Melanie Hilario. Learning to Combine Distances For Complex Representations.
In Proceedings of the 24th International Conference on Machine Learning, ICML, 2007.
pdf.
WOZ06 Adam Woznica, Alexandros Kalousis, and Melanie Hilario. Kernels for Sets of Objects.
In Proceedings of the IEEE Conference on Data Mining, ICDM, 2006.
pdf.
WOZ05a - Adam Woznica, Alexandros Kalousis and Melanie Hilario. Kernels over relational algebra structures.
In Proceedings of the ninth Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD-2005). Springer.
pdf.
WOZ05b - Adam Woznica, Alexandros Kalousis and Melanie Hilario. Distance-based learning over extended relational algebra structures
In Proceedings of the 15th International Conference on Inductive Logic Programming (ILP-2005) (Late Breaking Papers).
pdf.