Title

Particle Swarm Model Selection

Author

HUGO JAIR ESCALANTE BALDERAS

MANUEL MONTES Y GOMEZ

LUIS ENRIQUE SUCAR SUCCAR

Access level

Open Access

Summary or description

This paper proposes the application of particle swarm optimization (PSO) to the problem of full model selection, FMS, for classification tasks. FMS is defined as follows: given a pool of preprocessing methods, feature selection and learning algorithms, to select the combination of these that obtains the lowest classification error for a given data set; the task also includes the selection of hyperparameters for the considered methods. This problem generates a vast search space to be explored, well suited for stochastic optimization techniques. FMS can be applied to any classification domain as it does not require domain knowledge. Different model types and a variety of algorithms can be considered under this formulation. Furthermore, competitive yet simple models can be obtained with FMS. We adopt PSO for the search because of its proven performance in different problems and because of its simplicity, since neither expensive computations nor complicated operations are needed. Interestingly, the way the search is guided allows PSO to avoid overfitting to some extend. Experimental results on benchmark data sets give evidence that the proposed approach is very effective, despite its simplicity. Furthermore, results obtained in the framework of a model selection challenge show the competitiveness of the models selected with PSO, compared to models selected with other techniques that focus on a single algorithm and that use domain knowledge.

Publisher

Journal of Machine Learning Research

Publish date

2009

Publication type

Article

Publication version

Accepted Version

Format

application/pdf

Language

English

Audience

Students

Researchers

General public

Citation suggestion

Escalante-Balderas, H.J., et al., (2009). Particle Swarm Model Selection, Journal of Machine Learning Research (10): 405-440

Source repository

Repositorio Institucional del INAOE

Downloads

90

Comments



You need to sign in or sign up to comment.