Meta-learning and meta-mining

My early days in research were on the area, of what I would call empirical meta-learning, where we were trying to chart the areas of expertise of learning algorithms with respect to dataset characteristics. Most of this work took place within the Metal project, which ended in the beginning of 2002, roughly the same time I finished my PHD on the same subject and swore to myself that I will not work on meta-learning again. I was proven wrong in 2009.

In the context of a European project (e-LICO), that our group was coordinating, we developed a knowledge- and experience-driven assistant for the knowledge discovery process. The project drew heavily, among others, on web 2.0 concepts such as web services, ontologies, semantic annotation of services, and social networks. The knowledge discovery assistant relied on a data mining ontology, domain ontologies, and a knowledge base to plan, compose, and rank knowledge discovery workflows, from data mining services which are available either locally or as web services. The assistant also contains a self improvement mechanism that uses meta-learning to exploit the collective experience of the community of its users in order to improve the quality of the suggested workflows. The latter is achieved through the collection of meta-data on the performance of different knowledge discovery workflows on different application problems. These meta-data are used by a meta-learner, which combines probabilistic reasoning with metric-based learning from complex structures to incrementally improve the assistant's workflow recommendations.

Relevant publications:
Phong Nguyen, Jun Wang, Alexandros Kalousis, Factorizing LambdaMART for cold start recommendations, Machine Learning Journal, 2016, (CoRR)
Maria Keet, Agnieszka Lawrynowicz, Claudia d'Amato, Alexandros Kalousis, Phong Nguyen, Raśl Palma, Robert Stevens, Melanie Hilario. The Data Mining OPtimization Ontology. Journal of Web Semantics (WS) 32:43-53 (2015) (pdf).
Phong Nguyen, Melanie Hilario, Alexandros Kalousis. Using Meta-mining to Support Data Mining Workflow Planning and Optimization. Journal of Artificial Intelligence Research (JAIR) 51:605-644 (2014) (pdf).
Melanie Hilario, Phong Nguyen, Huyen Do, Adam Woznica and Alexandros Kalousis. Ontology based meta-mining of knowledge discovery workflows. Book chapter in Meta-Learning in Computational Intelligence. Springer 2011, (pdf).
Melanie Hilario, Alexandros Kalousis, Phong Nguyen, Adam Woznica, A Data Mining Ontology for Algorithm Selection and Meta-Mining, submitted to ECML workshop on Service Oriented Data Mining, 2009.
Alexandros Kalousis, Abraham Bernstein and Melanie Hilario, Meta-learning with kernels and similarity functions for planning of data mining workflows. Workshop on Planning to Learn, ICML, 2008, (pdf).
Alexandros Kalousis, Joao Gama and Melanie Hilario, On Data and Algorithms: Understanding Inductive Performance. Machine Learning Journal, Special Issue on Meta-Learning, March 2004, 54(3), 275-312, (pdf).
Alexandros Kalousis and Melanie Hilario, Representational Issues in Meta-Learning. In Proceedings of the 20th International Conference on Machine Learning (ICML-2003), (pdf).
Hilan Bensusan and Alexandros Kalousis, Estimating the Predictive Accuracy of a Classifier. ECML 2001: 25-36.
Alexandros Kalousis and Melanie Hilario, Model Selection via Meta-learning: a Comparative Study. International Journal of Artificial Intelligence Tools, December 2001, Vol. 10(4), (ps).
Alexandros Kalousis and Theoharis Theoharis, Noemon: Design, implementation and performance results for an intelligent assistant for classifier selection. Intelligent Data Analysis Journal, 3(5). November 1999, (ps).