We have been working for several years now in the field of Multi-objective Learning of Neural Networks. We understand that the general problem of learning from data is intrinsically Multi-Objective, since it requires the optimization of two or more objective functions. In our approach what we basically do is to generate a Pareto set from two objective functions, usually error and norm of the weight vector, and than to pick-up one of the Pareto set solutions. Our main publications in the area are listed below.
TEIXEIRA, R. A. ; BRAGA, A. P. ; TAKAHASHI, Ricardo Hiroshi Caldeira ; SALDANHA, R. R. Improving generalization of MLPs with multi-objetive optimization. Neurocomputing (Amsterdam), v. 35, p. 189-194, 2000.
KOKSHENEV, Illya; BRAGA, Antonio Padua. An efficient multi-objective learning algorithm for RBF neural network. Neurocomputing, v. 73, n. 16, p. 2799-2808, 2010.
BRAGA, A. P. ; TAKAHASHI, Ricardo Hiroshi Caldeira ; TEIXEIRA, Roselito de Albuquerque ; COSTA, Marcelo Azevedo. Multi-objective algorithms for neural network learning. In: Y. Jin. (Org.). Multi-objective Machine Learning (Series: Studies in Computational Intelligence). 1 ed. Heidelberg: Springer Verlag, 2006, v. , p. 151-171.
COSTA, Marcelo Azevedo ; BRAGA, A. P. ; de Menezes, Benjamin Rodrigues. Convergence analysis of sliding mode trajectories in multi-objective neural networks learning. Neural Networks, v. 33, p. 21-31, 2012.
TEIXEIRA, R. A.; BRAGA, A. P. ; TAKAHASHI, Ricardo Hiroshi Caldeira; SALDANHA, R. R. Recent advances in the MOBJ algorithm for training artificial neural networks. International Journal of Neural Systems, v. 11, n. 3, p. 265-269, 2001.
COSTA, M. A. ; BRAGA, A. P. ; MENEZES, B. R. ; TEIXEIRA, Roselito de Albuquerque ; PARMA, G. G.. Training Neural Networks with a Multi-Objective Sliding Mode Control Algorithm. Neurocomputing (Amsterdam), Holanda, v. 51, p. 467-473, 2003.
PARMA, G. G. ; MENEZES, B. R. ; BRAGA, A. P.. Neural networks learning with sliding mode control: the sliding mode backpropagation algorithm. International Journal of Neural Systems, Cingapura, v. 9, n. 3, p. 187-193, 1999.
KOKSHENEV, I.; BRAGA, A. P.. Complexity Bounds of Radial Basis Functions and Multi-Objective Learning. In: European Symposium on Neural Networks, 2007, Brugges. Proceedings of the European Symposium on Neural Networks. Bruxelas: D-Side Publications, 2007. p. 73-78.
KOKSHENEV, I.; BRAGA, A. P.. A multi-objective learning algorithm for RBF neural network. In: X Simpósio Brasileiro de Redes Neurais (SBRN2008), 2008, Salvador. Proceedings of the 10th Brazilian Symposium on Neural Networks, 2008.
KOKSHENEV, I. ; BRAGA, A. P.. A multi-objective approach to RBF network learning. Neurocomputing (Amsterdam), v. 71, p. 1203-1209, 2008.
COSTA, M. A. ; BRAGA, A. P. ; MENEZES, B. R.. Improving generalization of MLPs with Sliding Mode Control and the Levenberg-Marquadt algorithm. Neurocomputing (Amsterdam), v. 70, p. 1342-1347, 2007.
ROCHA, H. P.; COSTA, M. A.; BRAGA, A. P.. Training Multi-Layer Perceptron with Multi-Objective Optimization and Spherical Weights Representation. In: European Symposium on Neural Networks, 2015, Brugges - Belgium. Proceedings of the European Symposium on Artificial Neural Networks. Louvain-la-Neuve: Ciaco - i6doc.com, 2015. p. 131-136.
ROCHA, H. P.; COSTA, M. A.; BRAGA, A. P.. Treinamento Multiobjetivo de Redes Neurais Artificiais Baseado em Coordenadas Esféricas. In: Congresso Brasileiro de Inteligência Computacional - CBIC, 2013, Porto de Galinhas - Recife. Anais do XI Congresso Brasileiro de Inteligência Computacional, 2013.
ROCHA, H. P.; CASTRO, C. L.; BRAGA, A. P.. Seleção de Modelos Neurais Utilizando Evolução Diferencial Através dos Controle de Erro e Norma do Vetor de Pesos. In: Congresso Brasileiro de Inteligência Computacional - CBIC, 2011, Fortaleza - Ceará. Anais do X Congresso Brasileiro de Inteligência Computacional, 2011.