Projects / Programmes source: ARIS

Robust computer vision methods for autonomous water surface vehicles

Research activity

Code Science Field Subfield
2.07.07  Engineering sciences and technologies  Computer science and informatics  Intelligent systems - software 

Code Science Field
P176  Natural sciences and mathematics  Artificial intelligence 

Code Science Field
1.02  Natural Sciences  Computer and information sciences 
Autonomous vessels, computer vision, obstacle detection, robust visual tracking
Evaluation (rules)
source: COBISS
Researchers (13)
no. Code Name and surname Research area Role Period No. of publicationsNo. of publications
1.  50367  Borja Bovcon  Computer science and informatics  Researcher  2017 - 2020  19 
2.  29381  PhD Luka Čehovin Zajc  Computer science and informatics  Researcher  2017 - 2019  124 
3.  35425  PhD Marko Janković  Computer science and informatics  Researcher  2019  24 
4.  06856  PhD Stanislav Kovačič  Systems and cybernetics  Retired researcher  2017 - 2020  390 
5.  30155  PhD Matej Kristan  Computer science and informatics  Head  2017 - 2020  323 
6.  33172  PhD Rok Mandeljc  Systems and cybernetics  Researcher  2017  56 
7.  25049  MSc Dean Mozetič  Computer science and informatics  Researcher  2017 - 2020  15 
8.  50843  Jon Natanael Muhovič  Computer science and informatics  Researcher  2017 - 2020  23 
9.  21310  PhD Janez Perš  Systems and cybernetics  Researcher  2017 - 2020  238 
10.  18198  PhD Danijel Skočaj  Computer science and informatics  Researcher  2017 - 2020  309 
11.  34398  PhD Domen Tabernik  Computer science and informatics  Researcher  2020  50 
12.  18185  PhD Andrej Trost  Electronic components and technologies  Researcher  2018  328 
13.  21901  Duško Vranac    Technical associate  2017 - 2020 
Organisations (3)
no. Code Research organisation City Registration number No. of publicationsNo. of publications
1.  1538  University of Ljubljana, Faculty of Electrical Engineering  Ljubljana  1626965  27,758 
2.  1539  University of Ljubljana, Faculty of Computer and Information Science  Ljubljana  1627023  16,239 
3.  2514  ROBOTINA, podjetje za inženiring, marketing, trgovino in proizvodnjo d.o.o. (Slovene)  Kozina  5361907  31 
Autonomous robotics is a fast growing research discipline posing significant scientific as well as technical challenges. The most apparent evidence this field’s potential is the onslaught of self-driving cars. Even though significant scientific and practical hurdles still need to be overcome, the self-driving cars are predicted to launch in near future. While long-term ground vehicle and space exploration are open but investigated problems, the state of marine robotics lags far behind, even though the potential market size for marine robotics, combined with economic value of oceans, is estimated to approximately $40.4bn between 2015 and 2019 according to the EU Robotics 2020 MAR. Recent advances in marine robotics have resulted in development of small-sized unmanned robotic vehicles (USVs). The advancements have been made predominantly in hardware, low-level guidance, control, self-organization and communication systems, but the level of autonomy in small-sized USVs is still relatively low. The reason is that research in advanced environment perception capabilities required for a long-term autonomous performance in uncontrolled environments lags behind the control and hardware research. Cameras as light-weight, low-power, information-rich sensors are becoming a viable alternative or addition to other sensorial modalities. The recent research in computer vision has significantly advanced the image interpretation techniques. But these techniques usually do not take into account the constraints of autonomous robots, which is why they perform poorly in uncontrolled conditions in which the robots operate. This project will focus on developing functionalities for robust autonomous navigation of USVs in uncontrolled environments, primarily relying on the captured visual information. Our objectives will be to develop efficient and robust computer vision approaches for obstacle detection, long-term tracking and fusion with other sensors and camera modalities. A critical requirement of the approaches will be real-time performance, environment adaptation and long-term robustness to temporary failures of sensory information and visual uncertainties. We will propose a framework that will combine such approaches into a model of robot environment, thus enabling robust long-term fully autonomous operation. The developed framework will be verified and validated on an existing integrated system, a USV, performing in real environment. The project will be composed of six work packages. Development of robust obstacle detection approaches able to detect and extract 3D position of large and small obstacles (WP1). Long-term tracking capabilities will be developed (WP2). Agile approach for USV environment construction by fusing detection and tracking outputs will be developed (WP3). We will construct several challenging datasets for objective offline development of the approaches, which will be tested on real USV as well (WP4). Two work packages (WP5 and WP6) will contain support activities such as results dissemination and project management. Three partners will be involved in the project, namely the Visual Cognitive Systems Laboratory at the Faculty of Computer and Information Science, University of Ljubljana (ViCoS), the Machine Vision Laboratory at the Faculty of Electrical engineering, University of Ljubljana (MVL) and Harpha Sea d.o.o. The ViCoS members will focus on semantic segmentation and tracking, while the MVL members will focus on stereo processing and USV environment modelling. Harpha Sea d.o.o. has been developing unmanned surface vehicles for over a decade. Their research group will be responsible for integration of the system on their USV, and performing data capture and validation of the system. The combination of the expertise in the field of computer vision and machine learning and the marine-environment-specific expertise in development of robotic boats will guarantee the project success.
Significance for science
The project addresses research issues important to several research areas. Long-term and short-term tracking, object identification and obstacle detection are central challenges in computer vision. Our emphasis on real-time and robust performance will contribute to the field of computer vision. Addressing the requirements specific to autonomous robots will make contributions to this field as well. Obstacle detection via semantic segmentation is a new paradigm with a great potential in the wider field of robotics. Given our extensive expertise in visual tracking, we realistically expect advances made in short-term and long-term tracking for robotics. Sensor fusion and interpretation is partially underdeveloped in USV research due to the lack of publicly available realistic datasets. This will be addressed by acquisition of our datasets and development of a state player that will enable evaluation of various USV-related performance aspects. We expect that public release of our datasets will facilitate further research in the field of USV on part of other, international, research groups.
Significance for the country
Fully autonomous and robust navigation of robotic boats opens up a range of applications in automatic measurement acquisition and surveillance with many beneficial effects. The autonomy will vastly reduce the costs of large-scale 3D depth scanning and search for sunken objects, broadening the spread of these services and opening new economy niches. The full autonomy is expected to affect the renewable energy harvesting by contributing to safer operation of hydroelectric power plants, which will use the small USVs for automatic structural inspection of their reservoirs. We have been informed in informal discussion with representatives of hydroelectric plant Hidroelektrarne na Spodnji Savi, d.o.o., that they plan to introduce USVs on their accumulation lakes. Thus we arguably expect positive effect on several industry sectors. Results of the project will encourage entrepreneurship in niche high-tech services related to autonomous boats. The development of USVs is expected to have beneficial effects on society. The autonomous vessels can be used by coastal communities for regularly performing water quality control, early detection of environmentally hazardous substances and natural disasters like fires. Recently the Slovenian marine biology surveillance station expressed their interest in our development of the USVs for evaluation of populations of marine life. Finally, we envision USV as a common platform for development of the new and improved methods and sensor modalities with potential use in enhancing safety of commercial shipping at reduced costs, thus going far beyond mobile robotics.
Most important scientific results Interim report, final report
Most important socioeconomically and culturally relevant results Interim report, final report
Views history