Control de sistemas usando aprendizaje de máquina

Systems control using machine learning

Jesús Martín Miguel Martínez (2023, [Tesis de maestría])

El aprendizaje por refuerzo es un paradigma del aprendizaje de máquina con un amplio desarrollo y una creciente demanda en aplicaciones que involucran toma de decisiones y control. Es un paradigma que permite el diseño de controladores que no dependen directamente del modelo que describe la dinámica del sistema. Esto es importante ya que en aplicaciones reales es frecuente que no se disponga de dichos modelos de manera precisa. Esta tesis tiene como objetivo implementar un controlador óptimo en tiempo discreto libre de modelo. La metodología elegida se basa en algoritmos de aprendizaje por refuerzo, enfocados en sistemas con espacios de estado y acción continuos a través de modelos discretos. Se utiliza el concepto de función de valor (Q-función y función V ) y la ecuación de Bellman para resolver el problema del regulador cuadrático lineal para un sistema mecánico masa-resorte-amortiguador, en casos donde se tiene conocimiento parcial y desconocimiento total del modelo. Para ambos casos las funciones de valor son definidas explícitamente por la estructura de un aproximador paramétrico, donde el vector de pesos del aproximador es sintonizado a través de un proceso iterativo de estimación de parámetros. Cuando se tiene conocimiento parcial de la dinámica se usa el método de aprendizaje por diferencias temporales en un entrenamiento episódico, que utiliza el esquema de mínimos cuadrados con mínimos cuadrados recursivos en la sintonización del crítico y descenso del gradiente en la sintonización del actor, el mejor resultado para este esquema es usando el algoritmo de iteración de valor para la solución de la ecuación de Bellman, con un resultado significativo en términos de precisión en comparación a los valores óptimos (función DLQR). Cuando se tiene desconocimiento de la dinámica se usa el algoritmo Q-learning en entrenamiento continuo, con el esquema de mínimos cuadrados con mínimos cuadrados recursivos y el esquema de mínimos cuadrados con descenso del gradiente. Ambos esquemas usan el algoritmo de iteración de política para la solución de la ecuación de Bellman, y se obtienen resultados de aproximadamente 0.001 en la medición del error cuadrático medio. Se realiza una prueba de adaptabilidad considerando variaciones que puedan suceder en los parámetros de la planta, siendo el esquema de mínimos cuadrados con mínimos cuadrados recursivos el que tiene los mejores resultados, reduciendo significativamente ...

Reinforcement learning is a machine learning paradigm with extensive development and growing demand in decision-making and control applications. This technique allows the design of controllers that do not directly depend on the model describing the system dynamics. It is useful in real-world applications, where accurate models are often unavailable. The objective of this work is to implement a modelfree discrete-time optimal controller. Through discrete models, we implemented reinforcement learning algorithms focused on systems with continuous state and action spaces. The concepts of value-function, Q-function, V -function, and the Bellman equation are employed to solve the linear quadratic regulator problem for a mass-spring-damper system in a partially known and utterly unknown model. For both cases, the value functions are explicitly defined by a parametric approximator’s structure, where the weight vector is tuned through an iterative parameter estimation process. When partial knowledge of the dynamics is available, the temporal difference learning method is used under episodic training, utilizing the least squares with a recursive least squares scheme for tuning the critic and gradient descent for the actor´s tuning. The best result for this scheme is achieved using the value iteration algorithm for solving the Bellman equation, yielding significant improvements in approximating the optimal values (DLQR function). When the dynamics are entirely unknown, the Q-learning algorithm is employed in continuous training, employing the least squares with recursive least squares and the gradient descent schemes. Both schemes use the policy iteration algorithm to solve the Bellman equation, and the system’s response using the obtained values was compared to the one using the theoretical optimal values, yielding approximately zero mean squared error between them. An adaptability test is conducted considering variations that may occur in plant parameters, with the least squares with recursive least squares scheme yielding the best results, significantly reducing the number of iterations required for convergence to optimal values.

aprendizaje por refuerzo, control óptimo, control adaptativo, sistemas mecánicos, libre de modelo, dinámica totalmente desconocida, aproximación paramétrica, Q-learning, iteración de política reinforcement learning, optimal control, adaptive control, mechanical systems, modelfree, utterly unknown dynamics, parametric approximation, Q-learning, policy iteration INGENIERÍA Y TECNOLOGÍA CIENCIAS TECNOLÓGICAS TECNOLOGÍA DE LOS ORDENADORES INTELIGENCIA ARTIFICIAL INTELIGENCIA ARTIFICIAL

Expansión poblacional reciente en la historia evolutiva de la anchoveta de California Engraulis mordax

Recent population expansion in the evolutionary history of the Californian anchovy Engraulis mordax

NOE DIAZ VILORIA LAURA SANCHEZ VELASCO RICARDO PEREZ ENRIQUEZ (2012, [Artículo])

"La anchoveta de California Engraulis mordax, es una especie templada que pudo haber pasado por un proceso de disyunción poblacional, debido al proceso postglacial de calentamiento del agua alrededor de la punta de la península de Baja California, hace unos 10,000 años. Se realizó un análisis genético para probar la hipótesis nula de homogeneidad genética entre el Golfo de California, México y el sur de California, EUA y si este era el caso, estimar el tiempo de surgimiento de haplotipos en términos de coalescencia. Se analizaron en total 80 secuencias de la región control hipervariable (ADNmt) de E. mordax, capturadas en la región central del Golfo de California (n = 40) y el sur de California (n = 40). A pesar del gran número de haplotipos únicos, no se observó diferenciación genética significativa entre localidades (FST = –0.0025, p = 0.686). Una distribución unimodal en la frecuencia del número de diferencias entre haplotipos indica un modelo de expansión rápida en el tamaño poblacional, que basado en una tasa mutacional de 3.6% por millón de años para la región control, indicó un tiempo de diferenciación nucleotídica relativamente reciente de aproximadamente 61,000 años. Este periodo de tiempo corresponde al Pleistoceno tardío, después de la formación de la península de Baja California, sugiriendo expansiones poblacionales en cada una de las localidades, seguidas del último episodio de glaciación, el cual quizás contribuyó a la migración de esta especie de afinidad templada entre las dos localidades y a su homogenización genética. Sin embargo este único evento reciente de flujo genético en la historia evolutiva de la especie, no explica por sí solo los patrones de distribución encontrados en las frecuencias de diferencias nucleotídicas."

"The Californian anchovy Engraulis mordax, a temperate species, may have undergone a process of population disjunction from experiencing post-glacial water heating processes around the tip of the Baja California Peninsula, Mexico about 10,000 b.p. A genetic analysis was performed to test the null hypothesis of genetic homogeneity between the Gulf of California and Southern California, U. S. A., and if this is the case, to estimate the time of haplotype emergence in terms of coalescence. A total of 80 sequences of the mtDNA hypervariable control region of E. mordax captured in the central Gulf of California (n = 40) and Southern California (n = 40) were analyzed. In spite of the large number of private haplotypes, no significant genetic differentiation among sites (FST = –0.0025, p = 0.686) was observed. An unimodal distribution of mismatch frequency between haplotypes indicated a model of rapid expansion in population size that, based on a mutation rate of 3.6% per million years in the control region, indicates a relatively recent nucleotide differentiation

time of approximately 61,000 years. This time period corresponds to the late Pleistocene, suggesting population expansions at each locality, followed by the last episode of glaciation, which may have contributed to migration of this temperate-affinity species between two locations and the genetic homogenization. However this unique recent event of gene flow in the evolutionary history of species does not explain by itself the mismatch distribution patterns found."

ADN mitocondrial, expansión poblacional reciente, flujo genético, región control, reloj molecular. Control region, gene flow, mitochondrial DNA, molecular clock, recent population expansion. BIOLOGÍA Y QUÍMICA CIENCIAS DE LA VIDA BIOLOGÍA ANIMAL (ZOOLOGÍA) ZOOLOGÍA MARINA ZOOLOGÍA MARINA

Modernidad, seguridad y corrupción : control de drogas en México durante la posguerra (1946-1949)

Andrea Olvera (2021, [Tesis de doctorado])

"Esta tesis analiza mediante diversas fuentes de archivo y hemerográficas cuáles fueron las transformaciones y permanencias que se dieron en el control de las drogas en México entre 1946 a 1948, que se caracterizó por su prohibición, criminalización y judicialización. Lo cual, estuvo enmarcado en el contexto de la posguerra y la consolidación de un régimen autoritario en el país".

Narcóticos; Control; México; Aspectos sociales; Historia; Drogas; Abuso; Aspectos sanitarios; Corrupción (En política); Política y gobierno; 1946-1952 CIENCIAS SOCIALES HISTORIA HISTORIA DE PAÍSES HISTORIA DE PAÍSES

Contrasting spatial patterns in active-fire and fire-suppressed mediterranean climate old-growth mixed conifer forests

Danny L. Fry  (2014, [Artículo])

In Mediterranean environments in western North America, historic fire regimes in frequent-fire conifer forests are highly variable both temporally and spatially. This complexity influenced forest structure and spatial patterns, but some of this diversity has been lost due to anthropogenic disruption of ecosystem processes, including fire. Information from reference forest sites can help management efforts to restore forests conditions that may be more resilient to future changes in disturbance regimes and climate. In this study, we characterize tree spatial patterns using four-ha stem maps from four old-growth, Jeffrey pine-mixed conifer forests, two with active-fire regimes in northwestern Mexico and two that experienced fire exclusion in the southern Sierra Nevada. Most of the trees were in patches, averaging six to 11 trees per patch at 0.007 to 0.014 ha-1, and occupied 27-46% of the study areas. Average canopy gap sizes (0.04 ha) covering 11-20% of the area were not significantly different among sites. The putative main effects of fire exclusion were higher densities of single trees in smaller size classes, larger proportion of trees (≥56%) in large patches (≥10 trees), and decreases in spatial complexity. While a homogenization of forest structure has been a typical result from fire exclusion, some similarities in patch, single tree, and gap attributes were maintained at these sites. These within-stand descriptions provide spatially relevant benchmarks from which to manage for structural heterogeneity in frequent-fire forest types.

article, climate, controlled study, ecosystem fire history, forest structure, geographic distribution, geographic mapping, land use, mathematical computing, mathematical model, Mexico, spatial analysis, taiga, United States, comparative study, conife CIENCIAS AGROPECUARIAS Y BIOTECNOLOGÍA CIENCIAS AGROPECUARIAS Y BIOTECNOLOGÍA

A simple extension to the CMASA method for the prediction of catalytic residues in the presence of single point mutations

David Israel Flores Granados (2014, [Artículo])

The automatic identification of catalytic residues still remains an important challenge in structural bioinformatics. Sequence-based methods are good alternatives when the query shares a high percentage of identity with a well-annotated enzyme. However, when the homology is not apparent, which occurs with many structures from the structural genome initiative, structural information should be exploited. A local structural comparison is preferred to a global structural comparison when predicting functional residues. CMASA is a recently proposed method for predicting catalytic residues based on a local structure comparison. The method achieves high accuracy and a high value for the Matthews correlation coefficient. However, point substitutions or a lack of relevant data strongly affect the performance of the method. In the present study, we propose a simple extension to the CMASA method to overcome this difficulty. Extensive computational experiments are shown as proof of concept instances, as well as for a few real cases. The results show that the extension performs well when the catalytic site contains mutated residues or when some residues are missing. The proposed modification could correctly predict the catalytic residues of a mutant thymidylate synthase, 1EVF. It also successfully predicted the catalytic residues for 3HRC despite the lack of information for a relevant side chain atom in the PDB file. © 2014 Flores et al.

1UU9 protein, 3HRC protein, protein, thymidylate synthase, unclassified drug, protein kinase, thymidylate synthase, accuracy, algorithm, Article, CMASA, CMASA Substitution Matrix, Contact Matrix Average Deviation, controlled study, correlation coeffi CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA MATEMÁTICAS ANÁLISIS NUMÉRICO ANÁLISIS NUMÉRICO

Maintenance of Coastal Surface Blooms by Surface Temperature Stratification and Wind Drift

MARY CARMEN RUIZ DE LA TORRE (2013, [Artículo])

Algae blooms are an increasingly recurrent phenomenon of potentially socio-economic impact in coastal waters globally and in the coastal upwelling region off northern Baja California, Mexico. In coastal upwelling areas the diurnal wind pattern is directed towards the coast during the day. We regularly found positive Near Surface Temperature Stratification (NSTS), the resulting density stratification is expected to reduce the frictional coupling of the surface layer from deeper waters and allow for its more efficient wind transport. We propose that the net transport of the top layer of approximately 2.7 kilometers per day towards the coast helps maintain surface blooms of slow growing dinoflagellate such as Lingulodinium polyedrum. We measured: near surface stratification with a free-rising CTD profiler, trajectories of drifter buoys with attached thermographs, wind speed and direction, velocity profiles via an Acoustic Doppler Current Profiler, Chlorophyll and cell concentration from water samples and vertical migration using sediment traps. The ADCP and drifter data agree and show noticeable current shear within the first meters of the surface where temperature stratification and high cell densities of L. polyedrum were found during the day. Drifters with 1m depth drogue moved towards the shore, whereas drifters at 3 and 5 m depth showed trajectories parallel or away from shore. A small part of the surface population migrated down to the sea floor during night thus reducing horizontal dispersion. The persistent transport of the surface bloom population towards shore should help maintain the bloom in favorable environmental conditions with high nutrients, but also increasing the potential socioeconomic impact of the blooms. The coast wise transport is not limited to blooms but includes all dissolved and particulate constituents in surface waters. © 2013 Ruiz-de la Torre et al.

chlorophyll, algal bloom, article, cell count, cell density, coastal waters, controlled study, dinoflagellate, Lingulodinium polyedrum, meteorological phenomena, Mexico, near surface temperature stratification, nonhuman, nutrient concentration, popul CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA CIENCIAS DE LA TIERRA Y DEL ESPACIO OCEANOGRAFÍA OCEANOGRAFÍA

Genetic analysis of Vibrio parahaemolyticus O3:K6 strains that have been isolated in Mexico since 1998

CARLOS ABRAHAM GUERRERO RUIZ (2017, [Artículo])

Vibrio parahaemolyticus is an important human pathogen that has been isolated worldwide from clinical cases, most of which have been associated with seafood consumption. Environmental and clinical toxigenic strains of V. parahaemolyticus that were isolated in Mexico from 1998 to 2012, including those from the only outbreak that has been reported in this country, were characterized genetically to assess the presence of the O3:K6 pandemic clone, and their genetic relationship to strains that are related to the pandemic clonal complex (CC3). Pathogenic tdh+ and tdh+/trh+ strains were analyzed by pulsed-field gel electrophoresis (PFGE) and multilocus sequence typing (MLST). Also, the entire genome of a Mexican O3:K6 strain was sequenced. Most of the strains were tdh/ORF8-positive and corresponded to the O3:K6 serotype. By PFGE and MLST, there was very close genetic relationship between ORF8/O3:K6 strains, and very high genetic diversities from non-pandemic strains. The genetic relationship is very close among O3:K6 strains that were isolated in Mexico and sequences that were available for strains in the CC3, based on the PubMLST database. The whole-genome sequence of CICESE-170 strain had high similarity with that of the reference RIMD 2210633 strain, and harbored 7 pathogenicity islands, including the 4 that denote O3:K6 pandemic strains. These results indicate that pandemic strains that have been isolated in Mexico show very close genetic relationship among them and with those isolated worldwide. © 2017 Guerrero et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Article, bacterial strain, biofouling, controlled study, Crassostrea, food intake, gene sequence, genetic analysis, genetic variability, Japan, Mexican, Mexico, molecular phylogeny, nonhuman, pandemic, pathogenicity island, sea food, serotyping, toxi BIOLOGÍA Y QUÍMICA CIENCIAS DE LA VIDA GENÉTICA GENÉTICA

Assessing the Response of Nematode Communities to Climate Change-Driven Warming: A Microcosm Experiment

RUTH GINGOLD WERMUTH (2013, [Artículo])

Biodiversity has diminished over the past decades with climate change being among the main responsible factors. One consequence of climate change is the increase in sea surface temperature, which, together with long exposure periods in intertidal areas, may exceed the tolerance level of benthic organisms. Benthic communities may suffer structural changes due to the loss of species or functional groups, putting ecological services at risk. In sandy beaches, free-living marine nematodes usually are the most abundant and diverse group of intertidal meiofauna, playing an important role in the benthic food web. While apparently many functionally similar nematode species co-exist temporally and spatially, experimental results on selected bacterivore species suggest no functional overlap, but rather an idiosyncratic contribution to ecosystem functioning. However, we hypothesize that functional redundancy is more likely to observe when taking into account the entire diversity of natural assemblages. We conducted a microcosm experiment with two natural communities to assess their stress response to elevated temperature. The two communities differed in diversity (high [HD] vs. low [LD]) and environmental origin (harsh vs. moderate conditions). We assessed their stress resistance to the experimental treatment in terms of species and diversity changes, and their function in terms of abundance, biomass, and trophic diversity. According to the Insurance Hypothesis, we hypothesized that the HD community would cope better with the stressful treatment due to species functional overlap, whereas the LD community functioning would benefit from species better adapted to harsh conditions. Our results indicate no evidence of functional redundancy in the studied nematofaunal communities. The species loss was more prominent and size specific in the HD; large predators and omnivores were lost, which may have important consequences for the benthic food web. Yet, we found evidence for alternative diversity-ecosystem functioning relationships, such as the Rivets and the Idiosyncrasy Model. © 2013 Gingold et al.

aquaculture, article, bacterivore, benthos, biodiversity, biomass, climate, community dynamics, controlled study, ecosystem, environmental temperature, microcosm, nematode, nonhuman, population abundance, species diversity, species richness, taxonomy CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA CIENCIAS DE LA TIERRA Y DEL ESPACIO OCEANOGRAFÍA OCEANOGRAFÍA

Whole-genome comparison between reference sequences and oyster Vibrio vulnificus C-genotype strains

CARLOS ABRAHAM GUERRERO RUIZ (2019, [Artículo])

Whole-genome sequences of Vibrio vulnificus clinical genotype (C-genotype) from the CICESE Culture Collection, isolated from oysters, were compared with reference sequences of CMCP6 and YJ016 V. vulnificus C-genotype strains of clinical origin. The RAST web server estimated the whole genome to be ~4.8 Mb in CICESE strain 316 and ~4.7 Mb in CICESE strain 325. No plasmids were detected in the CICESE strains. Based on a phylogenetic tree that was constructed with the whole-genome results, we observed high similarity between the reference sequences and oyster C-genotype isolates and a sharp contrast with environmental genotype (E-genotype) reference sequences, indicating that the differences between the C- and E-genotypes do not necessarily correspond to their isolation origin. The CICESE strains share 3488 genes (63.2%) with the YJ016 strain and 3500 genes (63.9%) with the CMCP6 strain. A total of 237 pathogenicity associated genes were selected from reference clinical strains, where—92 genes were from CMCP6, 126 genes from YJ016, and 19 from MO6-24/ O; the presence or absence of these genes was recorded for the CICESE strains. Of the 92 genes that were selected for CMCP6, 67 were present in both CICESE strains, as were as 86 of the 126 YJ016 genes and 13 of the 19 MO6-24/O genes. The detection of elements that are related to virulence in CICESE strains—such as the RTX gene cluster, vvhA and vvpE, the type IV pili cluster, the XII genomic island, and the viuB genes, suggests that environmental isolates with the C-genotype, have significant potential for infection. © 2019 Guerrero et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Article, bacterial gene, bacterial strain, bacterial virulence, comparative study, controlled study, gene cluster, gene identification, genomic island, genotype, nonhuman, phylogenetic tree, sequence analysis, strain identification, Vibrio vulnificus BIOLOGÍA Y QUÍMICA CIENCIAS DE LA VIDA GENÉTICA GENÉTICA

High quality diet improves lipid metabolic profile and breeding performance in the blue-footed booby, a long-lived seabird

ERICK GONZALEZ MEDINA (2018, [Artículo])

Understanding the role of diet in the physiological condition of adults during reproduction and hence its effect on reproductive performance is fundamental to understand reproductive strategies in long-lived animals. In birds, little is known about the influence of the quality of food consumed at the beginning of the reproductive period and its short-term effects on reproductive performance. To assess the role of diet in the physiological condition of female blue-footed booby, Sula nebouxii (BFBO), during reproduction we evaluated whether individual differences in diet (assessed by using δ13C and δ15N values of whole blood from female birds and muscle tissue of the principal prey species) prior to egg laying and during incubation influenced their lipid metabolic profile (measured as triglyceride levels and C:N ratio) and their reproductive performance (defined by laying date, clutch size and hatching success). Females with higher δ15N values in their blood during the courtship and incubation periods had a higher lipid metabolic profile, earlier laying date, greater clutch size (2–3 eggs) and higher hatching success. Females that laid earlier and more eggs (2–3 eggs) consumed more Pacific anchoveta (Cetengraulis mysticetus) and Pacific thread herring (Opisthonema libertate) than did other females. These two prey species also had high amounts of lipids (C:N ratio) and caloric content (Kcal/g fresh weight). The quality of food consumed by females at the beginning of reproduction affected their physiological condition, as well as their short-term reproductive performance. Our work emphasizes the importance of determining the influence of food quality during reproduction to understand the reproductive decisions and consequences in long-lived animals. © 2018 González-Medina et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

nitrogen 15, triacylglycerol, lipid, animal experiment, Article, breeding, carbon nitrogen ratio, clutch size, controlled study, courtship, diet, egg laying, female, food intake, hatching, lipid metabolism, muscle tissue, nonhuman, prey, reproduction CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA CIENCIAS DE LA TIERRA Y DEL ESPACIO OCEANOGRAFÍA OCEANOGRAFÍA