Control de sistemas usando aprendizaje de máquina

Systems control using machine learning

Jesús Martín Miguel Martínez (2023, [Tesis de maestría])

El aprendizaje por refuerzo es un paradigma del aprendizaje de máquina con un amplio desarrollo y una creciente demanda en aplicaciones que involucran toma de decisiones y control. Es un paradigma que permite el diseño de controladores que no dependen directamente del modelo que describe la dinámica del sistema. Esto es importante ya que en aplicaciones reales es frecuente que no se disponga de dichos modelos de manera precisa. Esta tesis tiene como objetivo implementar un controlador óptimo en tiempo discreto libre de modelo. La metodología elegida se basa en algoritmos de aprendizaje por refuerzo, enfocados en sistemas con espacios de estado y acción continuos a través de modelos discretos. Se utiliza el concepto de función de valor (Q-función y función V ) y la ecuación de Bellman para resolver el problema del regulador cuadrático lineal para un sistema mecánico masa-resorte-amortiguador, en casos donde se tiene conocimiento parcial y desconocimiento total del modelo. Para ambos casos las funciones de valor son definidas explícitamente por la estructura de un aproximador paramétrico, donde el vector de pesos del aproximador es sintonizado a través de un proceso iterativo de estimación de parámetros. Cuando se tiene conocimiento parcial de la dinámica se usa el método de aprendizaje por diferencias temporales en un entrenamiento episódico, que utiliza el esquema de mínimos cuadrados con mínimos cuadrados recursivos en la sintonización del crítico y descenso del gradiente en la sintonización del actor, el mejor resultado para este esquema es usando el algoritmo de iteración de valor para la solución de la ecuación de Bellman, con un resultado significativo en términos de precisión en comparación a los valores óptimos (función DLQR). Cuando se tiene desconocimiento de la dinámica se usa el algoritmo Q-learning en entrenamiento continuo, con el esquema de mínimos cuadrados con mínimos cuadrados recursivos y el esquema de mínimos cuadrados con descenso del gradiente. Ambos esquemas usan el algoritmo de iteración de política para la solución de la ecuación de Bellman, y se obtienen resultados de aproximadamente 0.001 en la medición del error cuadrático medio. Se realiza una prueba de adaptabilidad considerando variaciones que puedan suceder en los parámetros de la planta, siendo el esquema de mínimos cuadrados con mínimos cuadrados recursivos el que tiene los mejores resultados, reduciendo significativamente ...

Reinforcement learning is a machine learning paradigm with extensive development and growing demand in decision-making and control applications. This technique allows the design of controllers that do not directly depend on the model describing the system dynamics. It is useful in real-world applications, where accurate models are often unavailable. The objective of this work is to implement a modelfree discrete-time optimal controller. Through discrete models, we implemented reinforcement learning algorithms focused on systems with continuous state and action spaces. The concepts of value-function, Q-function, V -function, and the Bellman equation are employed to solve the linear quadratic regulator problem for a mass-spring-damper system in a partially known and utterly unknown model. For both cases, the value functions are explicitly defined by a parametric approximator’s structure, where the weight vector is tuned through an iterative parameter estimation process. When partial knowledge of the dynamics is available, the temporal difference learning method is used under episodic training, utilizing the least squares with a recursive least squares scheme for tuning the critic and gradient descent for the actor´s tuning. The best result for this scheme is achieved using the value iteration algorithm for solving the Bellman equation, yielding significant improvements in approximating the optimal values (DLQR function). When the dynamics are entirely unknown, the Q-learning algorithm is employed in continuous training, employing the least squares with recursive least squares and the gradient descent schemes. Both schemes use the policy iteration algorithm to solve the Bellman equation, and the system’s response using the obtained values was compared to the one using the theoretical optimal values, yielding approximately zero mean squared error between them. An adaptability test is conducted considering variations that may occur in plant parameters, with the least squares with recursive least squares scheme yielding the best results, significantly reducing the number of iterations required for convergence to optimal values.

aprendizaje por refuerzo, control óptimo, control adaptativo, sistemas mecánicos, libre de modelo, dinámica totalmente desconocida, aproximación paramétrica, Q-learning, iteración de política reinforcement learning, optimal control, adaptive control, mechanical systems, modelfree, utterly unknown dynamics, parametric approximation, Q-learning, policy iteration INGENIERÍA Y TECNOLOGÍA CIENCIAS TECNOLÓGICAS TECNOLOGÍA DE LOS ORDENADORES INTELIGENCIA ARTIFICIAL INTELIGENCIA ARTIFICIAL

Situated learning through local stories: positioning socio-ecological concerns, knowledge and practices in school

Rosa Guadalupe Mendoza Zuany Juan Carlos A. Sandoval Rivera Paula Martínez Bautista (2023, [Artículo, Artículo])

The article analyzes the importance of the local stories, which contain concerns, knowledge and practices to look after the socio-ecological environment, to trigger situated and pertinent teaching and learning processes in primary education in rural and indigenous contexts in Veracruz, Mexico. In particular, it focuses on the stories about care in the framework of a deep socio-ecological crisis in two Nahua communities of the Huasteca region. Interview-conversations were carried out with community actors in the two communities in which 32 stories emerged. The analysis allowed the identification of socio-ecological concerns of the community, characteristics of the stories that show their potential in the learning processes, as well as types of knowledge and practices that are rarely considered in the classroom, which are capable of being linked to curricular content, in order to contribute to reflection and action on the socio-ecological crisis of the communities.

aprendizaje narración de historias medio ambiente ambiente socio-cultural escuela rural CIENCIAS SOCIALES; HUMANIDADES Y CIENCIAS DE LA CONDUCTA CIENCIAS SOCIALES HUMANIDADES Y CIENCIAS DE LA CONDUCTA Learning story telling environment socio-cultural environment rural school

COVID-19 and family participation in school activities.: Teaching experiences around the Learn at Home program

Evangelina Cervantes Holguín Pavel Roel Gutiérrez Sandoval (2022, [Artículo, Artículo])

The article analyzes, from the qualitative method, the participation of families and teaching staff of the first cycle of primary education in the state of Chihuahua (Mexico) to carry out the various activities of the Learn at Home program implemented in March 2020 as a response to the resulting health confinement by COVID-19. It is concluded that the participation of families in school emergency situations implies improving the relationship between teachers, families, and the community in the implementation processes of educational programs with greater support to organize study times, take advantage of the different cultural capitals and promote family co-responsibility.

Aprendizaje Educación a distancia Enseñanza primaria Epidemia Familia COVID-19 Chihuahua HUMANIDADES Y CIENCIAS DE LA CONDUCTA HUMANIDADES Y CIENCIAS DE LA CONDUCTA Family Elementary education Epidemics Distance education Learning

Multi-modal deep learning improves grain yield prediction in wheat breeding by fusing genomics and phenomics

Xu Wang Sandesh Kumar Shrestha Philomin Juliana Suchismita Mondal Francisco Pinto Govindan Velu Leonardo Abdiel Crespo Herrera JULIO HUERTA_ESPINO Ravi Singh Jesse Poland (2023, [Artículo])

New Crop Varieties Plant Breeding Programs Yield Prediction CIENCIAS AGROPECUARIAS Y BIOTECNOLOGÍA LEARNING GRAIN YIELDS WHEAT BREEDING FOOD SECURITY

Editorial: Genome wide association studies and genomic selection for crop improvement in the era of Big Data

Alison Bentley Charles Chen Nunzio D'Agostino (2022, [Artículo])

Allele Mining High-Throughput Phenotyping Genomic Estimated Breeding Value CIENCIAS AGROPECUARIAS Y BIOTECNOLOGÍA CROP IMPROVEMENT DNA CHROMOSOME MAPPING GENETIC LINKAGE GENOMES GENOTYPING GERMINATION HEAT STRESS QUALITY CONTROL SINGLE NUCLEOTIDE POLYMORPHISM TRITICUM AESTIVUM GENETIC DIVERSITY (AS RESOURCE) HIGH-THROUGHPUT SEQUENCING

The spatial control of migrants on the Chihuahua border

EDGAR ABEL CASTRO (2023, [Artículo, Artículo])

This article tries to link the immigration policies of the United States and Mexico with the narrative developed by Michel Foucault. It shows how racism is the axis on which the State of biopower exercises its claims and its effects of power on bodies and on life. Thus, the current political rationality goes through the management of the living body of people, their health, and their spatiality. This principle extends to the homicidal function of the State. Two events that occurred on the Chihuahua border demonstrate this.

migrant control Foucault border racism migrante frontera racismo HUMANIDADES Y CIENCIAS DE LA CONDUCTA HUMANIDADES Y CIENCIAS DE LA CONDUCTA

Distance learning for farmers: Experience during the pandemic

Andrea Gardeazabal (2023, [Documento de trabajo])

In response to the COVID-19 pandemic's disruption of farmer training—a crucial component for enhancing the resilience and livelihoods of smallholder farmers—CIMMYT innovated educational solutions to sustain capacity building in agri-food systems. Addressing the challenges of limited mobile device access, poor internet connectivity, and digital illiteracy, CIMMYT implemented two pilot projects in Mexico. These projects facilitated distance learning for adult farmers in rural areas, employing both internet-based and non-internet methods. The non-internet approach utilized traditional media like print, while the internet-based approach leveraged WhatsApp for educational content delivery. Building on these experiences, CIMMYT expanded its offerings by creating micro -courses delivered through WhatsApp, hosted on the Co-LAB's new Learning Network platform, specifically targeting farmers. This paper delves into the various strategies, methods, and techniques adopted, documenting the learning outcomes, results, and key conclusions drawn from these innovative training initiatives.

Distance Learning Digital Inclusion Innovative Training CIENCIAS AGROPECUARIAS Y BIOTECNOLOGÍA DISTANCE EDUCATION CAPACITY DEVELOPMENT METHODS COMMUNICATION TECHNOLOGY

A line follower robot implementation using Lego's Mindstorms Kit and Q-Learning

VICTOR RICARDO CRUZ ALVAREZ ENRIQUE HIDALGO PEÑA HECTOR GABRIEL ACOSTA MESA (2012, [Artículo])

Un problema común al trabajar con robots móviles es que la fase de programación puede ser un proceso largo, costoso y difícil para los programadores. Los Algoritmos de Aprendizaje por Refuerzo ofrecen uno de los marcos de trabajo más generales en el ámbito de aprendizaje de máquina. Este trabajo presenta un enfoque usando el algoritmo de Q-Learning en un robot Lego para que aprenda "por sí mismo" a seguir una línea negra dibujada en una superficie blanca. El entorno de programación utilizado en este trabajo es Matlab.

INGENIERÍA Y TECNOLOGÍA Algoritmos de aprendizaje reforzado Q-learning (Algoritmo de aprendizaje reforzado) Lego Mindstorms (Robótica) Matlab Reinforcement learning algorithms Q-Learning (Reinforcement learning algorithm) Lego Mindstorms (Robotics) Matlab

Multi-environment genomic prediction of plant traits using deep learners with dense architecture

Osval Antonio Montesinos-Lopez Jose Crossa (2018, [Artículo])

Shared Data Resources Deep Learning Genomic Prediction CIENCIAS AGROPECUARIAS Y BIOTECNOLOGÍA ACCURACY GENOMICS NEURAL NETWORKS FORECASTING DATA MARKER-ASSISTED SELECTION

High-throughput plot-level quantitative phenotyping using convolutional neural networks on very high-resolution satellite images

Francisco Pinto Matthew Paul Reynolds Robert Furbank (2024, [Artículo])

Deep Learning Object-Based Image Analysis Optical Imagery CIENCIAS AGROPECUARIAS Y BIOTECNOLOGÍA AGRICULTURE IMAGE ANALYSIS PLANT BREEDING REMOTE SENSING MACHINE LEARNING