Título
Multimodal indexing based on semantic cohesion for image retrieval
Autor
Hugo Jair Escalante Balderas
Manuel Montes y Gómez
Luis Enrique Sucar Succar
Nivel de Acceso
Acceso Abierto
Materias
Multimedia image retrieval - (MULTIMEDIA IMAGE RETRIEVAL) Image annotation - (IMAGE ANNOTATION) Distributional term representations - (DISTRIBUTIONAL TERM REPRESENTATIONS) Semantic cohesion modeling - (SEMANTIC COHESION MODELING) CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA - (CTI) MATEMÁTICAS - (CTI) CIENCIA DE LOS ORDENADORES - (CTI) CIENCIA DE LOS ORDENADORES - (CTI)
Resumen o descripción
This paper introduces two novel strategies for representing multimodal images with application to multimedia image retrieval. We consider images that are composed of both text and labels: while text describes the image content at a very high semantic level (e.g., making reference to places, dates or events), labels provide a mid-level description of the image (i.e., in terms of the objects that can be seen in the image). Accordingly, the main assumption of this work is that by combining information from text and labels we can develop very effective retrieval methods. We study standard information fusion techniques for combining both sources of information. However, whereas the performance of such techniques is highly competitive, they cannot capture effectively the content of images. Therefore, we propose two novel representations for multimodal images that attempt to exploit the semantic cohesion among terms from different modalities. Such representations are based on distributional term representations widely used in computational linguistics. Under the considered representations the content of an image is modeled by a distribution of co-occurrences over terms or of occurrences over other images, in such a way that the representation can be considered an expansion of the multimodal terms in the image. We report experimental results using the SAIAPR TC12 benchmark on two sets of topics used in ImageCLEF competitions with manually and automatically generated labels. Experimental results show that the proposed representations outperform significantly both, standard multimodal techniques and unimodal methods. Results on manually assigned labels provide an upper bound in the retrieval performance that can be obtained, whereas results with automatically generated labels are encouraging. The novel representations are able to capture more effectively the content of multimodal images. We emphasize that although we have applied our representations to multimedia image retrieval the same formulation can be adopted for modeling other multimodal documents (e.g., videos).
Editor
Springer Science + Business Media
Fecha de publicación
2012
Tipo de publicación
Artículo
Versión de la publicación
Versión aceptada
Recurso de información
Formato
application/pdf
Idioma
Inglés
Audiencia
Estudiantes
Investigadores
Público en general
Sugerencia de citación
Escalante-Balderas, H.J., et al., (2012). Multimodal indexing based on semantic cohesion for image retrieval, Information Retrieval, Vol. 15 (1): 1–32
Repositorio Orígen
Repositorio Institucional del INAOE
Descargas
332