Loading...
Projects / Programmes source: ARIS

Detection of inconsistencies in complex visual data using deep learning

Research activity

Code Science Field Subfield
2.07.00  Engineering sciences and technologies  Computer science and informatics   

Code Science Field
P176  Natural sciences and mathematics  Artificial intelligence 

Code Science Field
1.02  Natural Sciences  Computer and information sciences 
Keywords
Deep learning, computer vision, supervised learning, semi-supervised learning, unsupervised learning, visual learning, segmentation, anomaly detection
Evaluation (rules)
source: COBISS
Researchers (14)
no. Code Name and surname Research area Role Period No. of publicationsNo. of publications
1.  50367  Borja Bovcon  Computer science and informatics  Researcher  2018 - 2019  19 
2.  25402  PhD Jože Guna  Communications technology  Researcher  2019  236 
3.  30155  PhD Matej Kristan  Computer science and informatics  Researcher  2018 - 2022  323 
4.  31985  PhD Janez Križaj  Systems and cybernetics  Researcher  2018 - 2022  38 
5.  05896  PhD Aleš Leonardis  Computer science and informatics  Researcher  2018 - 2022  455 
6.  50843  Jon Natanael Muhovič  Computer science and informatics  Researcher  2018 - 2022  23 
7.  21310  PhD Janez Perš  Systems and cybernetics  Researcher  2018 - 2022  238 
8.  39056  Anže Pratnemer    Technical associate  2019 
9.  18198  PhD Danijel Skočaj  Computer science and informatics  Head  2018 - 2022  306 
10.  28458  PhD Vitomir Štruc  Systems and cybernetics  Researcher  2018 - 2022  351 
11.  34398  PhD Domen Tabernik  Computer science and informatics  Researcher  2018 - 2022  49 
12.  18185  PhD Andrej Trost  Electronic components and technologies  Researcher  2020  325 
13.  53924  Vitjan Zavrtanik  Computer science and informatics  Researcher  2019 - 2020  14 
14.  19237  MSc Rok Žurbi  Telecommunications  Researcher  2019  20 
Organisations (2)
no. Code Research organisation City Registration number No. of publicationsNo. of publications
1.  1538  University of Ljubljana, Faculty of Electrical Engineering  Ljubljana  1626965  27,603 
2.  1539  University of Ljubljana, Faculty of Computer and Information Science  Ljubljana  1627023  16,011 
Abstract
Huge volumes of data are constantly being collected and a lot of this data is available at no additional cost. However, making use of this data poses a huge challenge. Due to enormous quantities of data it is clear that manual processing is prohibitive and calls for automated procedures. Computer vision, driven with machine learning methods, is a scientific discipline that addresses automatic extraction of information from raw visual data. In the last couple of years we have witnessed a great increase in the performance of computer vision methods, mainly due to the success of deep learning approaches, that are well suited for extracting information from large quantities of data. However, most of the deep learning approaches rely on labelled data; they therefore still require a significant human effort for labelling the required amounts of data, which is very costly, tedious and sometimes error-prone or even impossible. In this project we will address this issue for a particular computer vision task of anomaly detection in images. We define an anomaly as a part or parts of an image that significantly deviate from a regular appearance observed over a large number of images. As main problem domains we selected three different tasks: detection of surface anomalies in industrial visual inspection, detection of changes in satellite images, and detection of unusual scenes in surveillance images. Most of the data in each of these three problem domains is quite consistent. The objective of the proposed project is to develop novel deep learning methods for modelling complex consistency and detecting inconsistencies in visual data using training images annotated with different levels of accuracy. We will therefore go beyond the traditional supervised learning, where all anomalies on all training images have to be adequately labelled. We will relax this requirement in several ways. First, we will allow coarser labelling of anomalous regions and develop methods for weakly-supervised discriminative learning to solve this problem. We will then go further and allow even weaker annotations, on the level of images, without any region annotations, and consider the case where only a subset of training images is labelled. We will develop corresponding methods for semi-supervised learning based on Multiple instance learning and self-supervised surrogate objective methods. The final goal will be to train the model only on anomaly-free training samples using deep generative modelling. We will base our work on methods such as variational autoencoders and generative adversarial networks by replacing classic deep networks with the architectures that have explicit compositional structure. Since such fully-unsupervised learning process may lead to detection of data inconsistencies that are meaningless from the predefined task point of view, we will also consider the cognitive relevance of the learned models and will adapt the learning process accordingly. The goal of this project is to develop general methods for anomaly detection in images that require minimal amount of labelled data. We will validate the developed methods in three related but different problem domains; visual inspection, remote sensing, and visual surveillance. The corresponding image datasets will be acquired and used to assess the performance and the limits of the developed methods. We will develop several concepts related to the fundamental understanding of deep learning, which we will apply to the problem of anomaly detection, where the primary impact is to be expected. However, these concepts could be applied in deep learning approaches in general, thus applicable on various computer vision problems. On the other hand, we will also apply the developed approaches on three specific problem domains, therefore a significant impact can be expected also in more narrower research subfields, such as machine vision and remote sensing and related disciplines.
Significance for science
The project addresses research issues important to several research areas. Our primary goal is to go beyond the supervised deep learning for the specific case of automated detection of anomalies in visual data. We therefore expect to go beyond the state-of-the-art in this specific research area. However, to achieve this goal, we will develop several concepts related to the fundamental understanding of deep learning, which could be applicable in deep learning approaches in general, thus applicable on various computer vision problems.   More specifically, our work will primarily focus on the development of basic methods for semi- and unsupervised learning. We will address adversarial learning of generative deep networks and show novel methods that incorporate compositional properties in deep models. Using the properties of compositional hierarchies, we will improve the understanding of deep networks. Although the computer vision and deep learning community has started investigating approaches that do not require a huge number of labelled training data, the research field still predominantly relies on supervised learning, so we expect that the results of the proposed project will have a significant impact to the future development in this research area.   We will validate the proposed methods on three problem domains, namely visual inspection, remote sensing, and visual surveillance. The proposed approaches will therefore have a direct impact in each of these three subfields of computer vision. Moreover, through interdisciplinary research, the results could also achieve a significant impact in other research areas. Our methods for anomaly detection in satellite images will be directly applicable to the problems related to the environment monitoring. In particular, monitoring of agricultural fields, forests and sea surfaces through satellite images can have significant impact on advancements in the related research fields. Similarly, with novel methods for detection of anomalies on surfaces of industrial products, we expect to enter the rather conservative field of machine vision, introducing the deep learning methods into machine vision systems for automated visual inspection even where not a lot of training samples are available. There will be therefore numerous possibilities to transfer our findings into practice.   Besides the research endeavours we also plan to contribute our share in community building. During the project we will build image datasets for evaluation and benchmarking developed methods for visual anomaly detection. In particular, as today, there is almost no realistic dataset for detection of anomalies on object surfaces, which makes the comparison of developed approaches difficult. We expect that public release of our datasets will facilitate further research in this field on part of other, international, research groups. It will make the comparison of developed approaches possible, which will set a solid basis for further research and community building.
Significance for the country
The project addresses research issues important to several research areas. Our primary goal is to go beyond the supervised deep learning for the specific case of automated detection of anomalies in visual data. We therefore expect to go beyond the state-of-the-art in this specific research area. However, to achieve this goal, we will develop several concepts related to the fundamental understanding of deep learning, which could be applicable in deep learning approaches in general, thus applicable on various computer vision problems.   More specifically, our work will primarily focus on the development of basic methods for semi- and unsupervised learning. We will address adversarial learning of generative deep networks and show novel methods that incorporate compositional properties in deep models. Using the properties of compositional hierarchies, we will improve the understanding of deep networks. Although the computer vision and deep learning community has started investigating approaches that do not require a huge number of labelled training data, the research field still predominantly relies on supervised learning, so we expect that the results of the proposed project will have a significant impact to the future development in this research area.   We will validate the proposed methods on three problem domains, namely visual inspection, remote sensing, and visual surveillance. The proposed approaches will therefore have a direct impact in each of these three subfields of computer vision. Moreover, through interdisciplinary research, the results could also achieve a significant impact in other research areas. Our methods for anomaly detection in satellite images will be directly applicable to the problems related to the environment monitoring. In particular, monitoring of agricultural fields, forests and sea surfaces through satellite images can have significant impact on advancements in the related research fields. Similarly, with novel methods for detection of anomalies on surfaces of industrial products, we expect to enter the rather conservative field of machine vision, introducing the deep learning methods into machine vision systems for automated visual inspection even where not a lot of training samples are available. There will be therefore numerous possibilities to transfer our findings into practice.   Besides the research endeavours we also plan to contribute our share in community building. During the project we will build image datasets for evaluation and benchmarking developed methods for visual anomaly detection. In particular, as today, there is almost no realistic dataset for detection of anomalies on object surfaces, which makes the comparison of developed approaches difficult. We expect that public release of our datasets will facilitate further research in this field on part of other, international, research groups. It will make the comparison of developed approaches possible, which will set a solid basis for further research and community building.
Most important scientific results Interim report
Most important socioeconomically and culturally relevant results
Views history
Favourite