Author: EDUARDO FRANCISCO MORALES MANZANARES

Relational reinforcement learning with continuous actions by combining behavioural cloning and locally weighted regression

EDUARDO FRANCISCO MORALES MANZANARES (2010)

Reinforcement Learning is a commonly used technique for learning tasks in robotics, however, traditional algorithms are unable to handle large amounts of data coming from the robot’s sensors, require long training times, and use dis-crete actions. This work introduces TS-RRLCA, a two stage method to tackle these problems. In the first stage, low-level data coming from the robot’s sensors is transformed into a more natural, relational representation based on rooms, walls, corners, doors and obstacles, significantly reducing the state space. We use this representation along with Be-havioural Cloning, i.e., traces provided by the user; to learn, in few iterations, a relational control policy with discrete actions which can be re-used in different environments. In the second stage, we use Locally Weighted Regression to transform the initial policy into a continuous actions policy. We tested our approach in simulation and with a real ser-vice robot on different environments for different navigation and following tasks. Results show how the policies can be used on different domains and perform smoother, faster and shorter paths than the original discrete actions policies.

Article

Relational Reinforcement Learning Behavioural Cloning Continuous Actions Robotics CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA MATEMÁTICAS CIENCIA DE LOS ORDENADORES CIENCIA DE LOS ORDENADORES

Dynamic behavior of contaminants in the water distribution network of Cuernavaca, Mexico: a real application of multiobjective distributed reinforcement learning

Comportamiento dinámico de contaminantes en la red de distribución de agua de la ciudad de Cuernavaca, México: una aplicación real de aprendizaje por refuerzo distribuido multi-objetivo.

CARLOS EDUARDO MARIANO ROMERO EDUARDO FRANCISCO MORALES MANZANARES (2008)

El uso eficiente del agua se puede establecer a través del reúso o recirculación de efluentes. El diseño o configuración de sistemas que contemplen el uso eficiente de agua con base en el reúso se torna complejo e involucra diversos criterios a optimizar. El uso de técnicas basadas en Water Pinch permite definir los efluentes más apropiados a ser reutilizados, posicionándose como una buena alternativa para los diseñadores. En trabajos previos se presentaron resultados relacionados con la minimización del agua de primer uso suministrada y el costo de inversión sobre datos reales de la ciudad de Cuernavaca. Sin embargo, uno de los retos identificados a partir de la observación del comportamiento de los efluentes en la ciudad de Cuernavaca es la necesidad de representar el comportamiento dinámico de los sistemas de distribución. En esté trabajo se presenta la respuesta del modelo de optimización a los cambios medidos en los efluentes de las operaciones unitarias de las masas de contaminantes. El escenario de validación es el sistema de distribución de la ciudad de Cuernavaca en México.

Article

Redes de distribución de agua Contaminación del agua Uso eficiente del agua Cuernavaca INGENIERÍA Y TECNOLOGÍA

Dynamic behavior of contaminants in the water distribution network of Cuernavaca Mexico, a real application of multiobjective distributed reinforcement learning

Comportamiento dinámico de contaminantes en la red de distribución de agua de la ciudad de Cuernavaca México, una aplicación real de aprendizaje por refuerzo distribuido multi-objetivo

CARLOS EDUARDO MARIANO ROMERO EDUARDO FRANCISCO MORALES MANZANARES (2008)

Water systems often allow efficient water uses via water reuse and/or recirculation. The design of the network layout connecting water-using processes is a complex problem which involves several criteria to optimize. The use of the water pinch approach to define which of the effluents from unitary operations are most convenient to reuse is a good alternative used by some practitioners. Previously papers have presented an approach to minimize the freshwater consumption and infrastructure cost, which had been tested with real data from the Cuernavaca city water distribution network with good results (Mariano2005, Mariano2007). One of the challenges identified from previous work, was the necessity to incorporate the dynamic behavior of distribution systems. In this paper the response of the optimization model to changes in the mass charges of contaminants effluents from unitary operations is presented. The test scenario is the distribution system of the city of Cuernavaca in México.

El uso eficiente del agua se puede establecer a través del reuso o recirculación de efluentes. El diseño o configuración de sistemas que contemplen el uso eficiente de agua con base en el reuso se torna complejo e involucra diversos criterios a optimizar. El uso de técnicas basadas en Water Pinch permite definir los efluentes más apropiados a ser reutilizados, posicionándose como una buena alternativa para los diseñadores. En trabajos previos se presentaron resultados relacionados con la minimización del agua de primer uso suministrada y el costo de inversión sobre datos reales de la ciudad de Cuernavaca (Mariano2005, Mariano2007). Sin embargo, uno de los retos identificados a partir del la observación del comportamiento de los efluentes en la ciudad de Cuernavaca es la necesidad de representar el comportamiento dinámico de los sistemas de distribución. En esté trabajo se presenta la respuesta del modelo de optimización a los cambios medidos en los efluentes de las operaciones unitarias de las masas de contaminantes. El escenario de validación es el sistema de distribución de la ciudad de Cuernavaca en México.

Article

Multiobjective optimization Water pinch Water reuse Optimización Multiobjetivo Reuso de agua CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA MATEMÁTICAS CIENCIA DE LOS ORDENADORES

Automatic generation of explanations: AGE

SILVIA BEATRIZ GONZALEZ BRAMBILA EDUARDO FRANCISCO MORALES MANZANARES (2007)

Explaining how engineering devices work is important to students, engineers, and operators. In general, machine

generated explanations have been produced from a particular perspective. This paper introduces a system called automatic

generation of explanations (AGE) capable of generating causal, behavioral, and functional explanations of physical

devices in natural language. AGE explanations can involve different user selected state variables at different abstraction levels.

AGE uses a library of engineering components as building blocks. Each component is associated with a qualitative model,

information about the meaning of state variables and their possible values, information about substances, and information

about the different functions each component can perform. AGE uses: (i) a compositional modeling approach to construct large

qualitative models, (ii) causal analysis to build a causal dependency graph, (iii) a novel qualitative simulation approach to

efficiently obtain the system’s behavior on large systems, and (iv) decomposition analysis to automatically divide large devices into

smaller subsystems. AGE effectiveness is demonstrated with different devices that range from a simple water tank to an industrial

chemical plant.

Article

Generation of explanations Qualitative reasoning CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA MATEMÁTICAS

Inductive transfer for learning Bayesian networks

LUIS ENRIQUE SUCAR SUCCAR EDUARDO FRANCISCO MORALES MANZANARES (2009)

In several domains it is common to have data from different, but closely related problems. For instance, in manufacturing, many products follow the same industrial process but with different conditions; or in industrial diagnosis, where there is equipment with similar specifications. In these cases it is common to have plenty of data for some scenarios but very little for others. In order to learn accurate models for rare cases, it is desirable to use data and knowledge from similar cases; a technique known as transfer learning. In this paper we propose an inductive transfer learning method for Bayesian networks, that considers both structure and parameter learning. For structure learning we use conditional independence tests, by combining measures from the target task with those obtained from one or more auxiliary tasks, using a novel weighted sum of the conditional independence measures. For parameter learning, we propose two variants of the linear pool for probability aggregation, combining the probability estimates from the target task with those from the auxiliary tasks. To validate our approach, we used three Bayesian networks models that are commonly used for evaluating learning techniques, and generated variants of each model by changing the structure as well as the parameters. We then learned one of the variants with a small dataset and combined it with information from the other variants. The experimental results show a significant improvement in terms of structure and parameters when we transfer knowledge from similar tasks. We also evaluated the method with real-world data from a manufacturing process considering several products, obtaining an improvement in terms of log-likelihood between the data and the model when we do transfer learning from related products.

Article

Inductive transfer Bayesian networks Structure learning Parameter learning CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA MATEMÁTICAS CIENCIA DE LOS ORDENADORES

Multi-objective Optimization of Water-using Systems

CARLOS EDUARDO MARIANO ROMERO VICTOR HUGO ALCOCER YAMANAKA EDUARDO FRANCISCO MORALES MANZANARES (2007)

Industrial water systems often allow ecient water uses via water

reuse and/or recirculation. The design of the network layout connect-

ing water-using processes is a complex problem which involves several

criteria to optimize. Most of the time, this design is achieved using

Water Pinch technology, optimizing the freshwater

ow rate enter-

ing the system. This paper describes an approach that considers two

criteria: (i) the minimization of freshwater consumption and (ii) the

minimization of the infrastructure cost required to build the network.

The optimization model considers water reuse between operations and

wastewater treatment as the main mechanisms to reduce freshwater

consumption. The model is solved using MDQL (Multi-objective Dis-

tributed Q-Learning), a heuristic approach based on the exploitation

of knowledge acquired during the search process. MDQL has been

previously tested on several multi-objective optimization benchmark

problems with promising results [16]. In order to compare the quality

of the results obtained with MDQL, the reduced gradient method was

applied to solve a weighted combination of the two objective functions

used in the model. The proposed approach was tested on three cases:

(i) a single contaminant four unitary operations problem where fresh-

water consumption is reduced via water reuse, (ii) a four contaminants

real-world case with ten unitary operations, also with water reuse, and

(iii) the water distribution network operation of Cuernavaca, Mexico,

considering reduction of water leaks, operation of existing treatment

plants at their design capacity, and design and construction of new

treatment infrastructure to treat 100% of the wastewater produced.

It is shown that the proposed approach can solved highly constrained

real-world multi-objective optimization problems.

Article

CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA CIENCIAS DE LA TIERRA Y DEL ESPACIO HIDROLOGÍA

AsistO: A qualitative MDP-based recommender system for power plant operation

AsistO: Un sistema de recomendaciones basado en MDPs cualitativos para la operación de plantas generadoras

ALBERTO REYES BALLESTEROS LUIS ENRIQUE SUCAR SUCCAR EDUARDO FRANCISCO MORALES MANZANARES (2009)

This paper proposes a novel and practical model-based learning approach with iterative refinement for solving continuous (and hybrid) Markov decision processes. Initially, an approximate model is learned using conventional sampling methods and solved to obtain a policy. Iteratively, the approximate model is refined using variance in the utility values as partition criterion. In the learning phase, initial reward and transition functions are obtained by sampling the state–action space. The samples are used to induce a decision tree predicting reward values from which an initial partition of the state space is built. The samples are also used to induce a factored MDP. The state abstraction is then refined by splitting states only where the split is locally important. The main contributions of this paper are the use of sampling to construct an abstraction, and a local refinement process of the state abstraction based on utility variance. The proposed technique was tested in AsistO, an intelligent recommender system for power plant operation, where we solved two versions of a complex hybrid continuous-discrete problem. We show how our technique approximates a solution even in cases where standard methods explode computationally.

Este artículo propone una técnica novedosa y práctica de aprendizaje basada en modelos con refinamiento iterativo para resolver procesos de decisión de Markov (MDPs) continuos. Inicialmente, se aprende un modelo aproximado usando métodos de muestreo convencionales, el cual se resuelve para obtener una política. Iterativamente, el modelo aproximado se refina con base en la varianza de los valores de la utilidad esperada. En la fase de aprendizaje, se obtienen las funciones de recompensa inmediata y de transición mediante muestras del tipo estado-acción. Éstas primero se usan para inducir un árbol de decisión que predice los valores de recompensa y a partir del cual se construye una partición inicial del espacio de estados. Posteriormente, las muestras también se usan para inducir un MDP factorizado. Finalmente, la abstracción de espacio de estados resultante se refina dividiendo aquellos estados donde pueda haber cambios en la política. Las contribuciones principales de este trabajo son el uso de datos para construir una abstracción inicial, y el proceso de refinamiento local basado en la varianza de la utilidad. La técnica propuesta fue probada en AsistO, un sistema inteligente de recomendaciones para la operación de plantas generadoras de electricidad, donde resolvimos dos versiones de un problema complejo con variables híbridas continuas y discretas. Aquí mostramos como nuestra técnica aproxima una solución aun en casos donde los métodos estándar explotan computacionalmente.

Article

Recommender systems Power plants Markov decision processes Abstractions Sistemas de recomendaciones Plantas generadoras Procesos de decisión de Markov Abstracciones CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA MATEMÁTICAS CIENCIA DE LOS ORDENADORES

Inductive transfer for learning Bayesian networks

ROGER LUIS VELAZQUEZ LUIS ENRIQUE SUCAR SUCCAR EDUARDO FRANCISCO MORALES MANZANARES (2010)

In several domains it is common to have data from different, but closely related problems. For instance, in manufacturing, many products follow the same industrial process but with different conditions; or in industrial diagnosis, where there is equipment with similar specifications. In these cases it is common to have plenty of data for some scenarios but very little for others. In order to learn accurate models for rare cases, it is desirable to use data and knowledge from similar cases; a technique known as transfer learning. In this paper we propose an inductive transfer learning method for Bayesian networks, that considers both structure and parameter learning. For structure learning we use conditional independence tests, by combining measures from the target task with those obtained from one or more auxiliary tasks, using a novel weighted sum of the conditional independence measures. For parameter learning, we propose two variants of the linear pool for probability aggregation, combining the probability estimates from the target task with those from the auxiliary tasks. To validate our approach, we used three Bayesian networks models that are commonly used for evaluating learning techniques, and generated variants of each model by changing the structure as well as the parameters. We then learned one of the variants with a small dataset and combined it with information from the other variants. The experimental results show a significant improvement in terms of structure and parameters when we transfer knowledge from similar tasks. We also evaluated the method with real-world data from a manufacturing process considering several products, obtaining an improvement in terms of log-likelihood between the data and the model when we do transfer learning from related products.

Article

Inductive transfer Bayesian networks Structure learning Parameter learning CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA MATEMÁTICAS CIENCIA DE LOS ORDENADORES CIENCIA DE LOS ORDENADORES

A dynamic Bayesian network for estimating the risk of falls from real gait data

GERMAN CUAYA SIMBRO ANGELICA MUÑOZ MELENDEZ LIDIA NUÑEZ CARRERA Eduardo Francisco Morales Manzanares IVETT QUIÑONES URIOSTEGUI ALDO ALESSI MONTERO (2012)

Pathological and age-related changes may affect an individual’s gait, in turn raising the risk of falls. In elderly, falls are common and may eventuate in severe injuries, long-term disabilities, and even death. Thus, there is interest in estimating the risk of falls from gait analysis. Estimation of the risk of falls requires consideration of the longitudinal evolution of different variables derived from human gait. Bayesian networks are probabilistic models which graphically express dependencies among variables. Dynamic Bayesian networks (DBNs) are a type of BN adequate for modeling the dynamics of the statistical dependencies in a set of variables. In this work, a DBN model incorporates gait derived variables to predict the risk of falls in elderly within 6 months subsequent to gait assessment. Two DBNs were developed; the first (DBN1; expert-guided) was built using gait variables identified by domain experts, whereas the second (DBN2; strictly computational) was constructed utilizing gait variables picked out by a feature selection algorithm. The effectiveness of the second model to predict falls in the 6 months following assessment is 72.22 %. These results are encouraging and supply evidence regarding the usefulness of dynamic probabilistic models in the prediction of falls from pathological gait.

Article

Probabilistic models Dynamic Bayesian networks Elderly Gait analysis Risk of falls CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA MATEMÁTICAS CIENCIA DE LOS ORDENADORES CIENCIA DE LOS ORDENADORES

The segmented and annotated IAPR TC-12 benchmark

HUGO JAIR ESCALANTE BALDERAS CARLOS ARTURO HERNANDEZ GRACIDAS JESUS ANTONIO GONZALEZ BERNAL AURELIO LOPEZ LOPEZ MANUEL MONTES Y GOMEZ EDUARDO FRANCISCO MORALES MANZANARES LUIS ENRIQUE SUCAR SUCCAR LUIS VILLASEÑOR PINEDA (2009)

Automatic image annotation (AIA), a highly popular topic in the field of information retrieval research, has experienced significant progress within the last decade. Yet, the lack of a standardized evaluation platform tailored to the needs of AIA, has hindered effective evaluation of its methods, especially for region-based AIA. Therefore in this paper, we introduce the segmented and annotated IAPR TC-12 benchmark; an extended resource for the evaluation of AIA methods as well as the analysis of their impact on multimedia information retrieval. We describe the methodology adopted for the manual segmentation and annotation of images, and present statistics for the extended collection. The extended collection is publicly available and can be used to evaluate a variety of tasks in addition to image annotation. We also propose a soft measure for the evaluation of annotation performance and identify future research areas in which this extended test collection is likely to make a contribution.

Article

Data set creation Ground truth collection Evaluation metrics Automatic image annotation Image retrieval CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA MATEMÁTICAS CIENCIA DE LOS ORDENADORES