Projects / Programmes source: ARIS

Learning and autonomous adaptation of dual arm assembly and service tasks

Research activity

Code Science Field Subfield
2.10.04  Engineering sciences and technologies  Manufacturing technologies and systems  Robotics 

Code Science Field
T125  Technological sciences  Automation, robotics, control engineering 

Code Science Field
2.03  Engineering and Technology  Mechanical engineering 
Dual-arm manipulation, dynamic movement primitives, robot learning, automated assembly, absolute and relative motion
Evaluation (rules)
source: COBISS
Researchers (17)
no. Code Name and surname Research area Role Period No. of publicationsNo. of publications
1.  25638  PhD Andrej Gams  Manufacturing technologies and systems  Researcher  2016 - 2018  235 
2.  37512  PhD Blaž Jakopin  Systems and cybernetics  Researcher  2018  10 
3.  50489  PhD David Kraljić  Computer intensive methods and applications  Researcher  2017  25 
4.  36333  PhD Aljaž Kramberger  Manufacturing technologies and systems  Junior researcher  2016 - 2017  22 
5.  32769  PhD Nejc Likar  Manufacturing technologies and systems  Researcher  2016 - 2018  25 
6.  07134  PhD Marko Munih  Systems and cybernetics  Researcher  2016 - 2018  724 
7.  00118  PhD Bojan Nemec  Systems and cybernetics  Researcher  2016 - 2018  289 
8.  39154  PhD Rok Pahič  Manufacturing technologies and systems  Junior researcher  2018  36 
9.  34505  PhD Janez Pavčič  Forestry, wood and paper technology  Researcher  2017  10 
10.  30885  PhD Tadej Petrič  Manufacturing technologies and systems  Researcher  2016 - 2018  196 
11.  51579  Jožica Piškur  Manufacturing technologies and systems  Researcher  2018 
12.  39071  Blaž Potočnik  Manufacturing technologies and systems  Researcher  2016 - 2017 
13.  39258  Simon Reberšek    Technical associate  2016 - 2018 
14.  34534  PhD Sebastjan Šlajpah  Systems and cybernetics  Researcher  2016 - 2018  71 
15.  11772  PhD Aleš Ude  Manufacturing technologies and systems  Head  2016 - 2018  472 
16.  34457  PhD Rok Vuga  Systems and cybernetics  Researcher  2016  19 
17.  03332  PhD Leon Žlajpah  Systems and cybernetics  Researcher  2016 - 2018  266 
Organisations (2)
no. Code Research organisation City Registration number No. of publicationsNo. of publications
1.  0106  Jožef Stefan Institute  Ljubljana  5051606000  90,664 
2.  1538  University of Ljubljana, Faculty of Electrical Engineering  Ljubljana  1626965  27,756 
Most of the tasks performed by humans require coordinated motion of both arms. Dual-arm robotic systems are therefore needed if the robots are to effectively cooperate with human workers or even replace them in production processes. The advantage of dual-arm anthropomorphic robots is that they can work with humans without major redesigns of the task or workplace. Bimanual manipulation is also the key feature to more advanced humanoid and service robots, which are expected act and some problems in ways similar to humans. Lately several major manufacturers of industrial robots have started to develop dual arm systems (Yaskawa, ABB, Rethink Robotics, etc.) However, to effectively exploit dual-arm robotic systems in industry and services, it is necessary to develop efficient control, learning and adaptation methods, which are currently still not available in industrial and service robotics. The proposed project deals with these opened questions by addressing the following issues: Develop new efficient methods for learning and demonstration of bimanual tasks. Two modes will be considered: kinesthetic guiding and demonstration using haptic devices.  Within this approach, besides position and orientation trajectories, also force and torque profiles will need to be captured. Develop new algorithms for efficient and autonomous adaptation of bimanual coordinated tasks to deviations, which are induced by inaccurate grasping, deviations in workpiece geometry and inaccurately demonstrated trajectories. It is well known that already small errors in trajectories can result in excessive undesired interaction forces. Therefore, our approach with adapt the trajectories of a bimanual skill in such a way, which will minimize the error between the desired and actual measured forces. For adaptation, we will consider iterative learning controllers (ILC), adaptive iterative learning controllers and reinforcement learning algorithms (RL). In order to increase the adaptation speed and exploit the benefits of model-free approaches, we will develop novel algorithms based on fusion of RL and ILC algorithms. Verify the developed methods and algorithms on some typical industrial and domestic tasks that involve automated bimanual assembly. The following three tasks will be analyzed in detail:  bimanual peg in hole, bimanual assembly of an automotive light and bimanual glass wiping. The developed methods for control, learning and adaptation of dual-arm manipulation tasks will be refined based on the results obtained from the above practical experiments. Our main aims are: 1. efficient and intuitive learning of bimanual skills, 2. safe and precise execution of the learned skill and 3. fast and reliable task execution. We expect that the outcome of this project will contribute to the faster introduction of dual-arm systems in industrial environments, including in SMEs, where the need for fast setup times and effective learning and adaptation of dual-arm tasks is especially important. Moreover, efficient bimanual task learning and adaptation algorithms will contribute to the new applications of robot assistants in our homes and replacing humans in hazardous operations. With such tasks it is crucial that the robot can effectively learn in new tasks in a user-friendly manner. With the successful execution of the proposed project we are going to contribute to one of the major problems of modern robotics with a significant market potential: the introduction of dual-arm robotic systems into industrial processes and service robots.
Significance for science
Dual-arm manipulation is becoming increasingly interesting for international robot manufacturers because it provides possibilities for opening new application areas to robotics, both in industry and in service domain. As mentioned in the overview of state of the art, there are still no effective methods for learning and adaptation of dual-arm manipulation tasks. With the proposed approach we will enable for the first time effective and user-friendly teaching of force-based dual-arm behaviors based on the programming by demonstration paradigm. A comprehensive framework for learning and adaptation of dual arm behaviors will be developed. Progress beyond state of the art includes The development of new programming by demonstration methods to acquire force-based dual arm behaviors. New methods for optimal adaptation of dual arm behaviors (encoded in term of absolute and relative coordinates) based on ILC and RL. Unlike in our previous works, where only the motion of one arm has been adapted to achieve the desired outcome, in this project we are going to exploit the redundancy of the dual-arm mechanism to achieve optimal behavior of both arms in terms of their trajectories and interaction forces and torques. Further novelties include new methods for a) self-tuning of ILC parameters to speed up the adaptation of dual arm behaviors, b) generalization of dual arm behaviors to new task contexts, and c) optimization of execution time of dual arm behaviors. It is expected that the project results will contribute to the more efficient implementation of robotic technology also in low batch production, where faster, intuitive and efficient learning and autonomous adaptation of dual-arm manipulation tasks is of particular importance. Moreover, the developed algorithms will contribute to the development of the next generation of robot assistants in our everyday life and more efficient service robot application with the general aim to replace human workers in hazardous environments. Scientific results will be disseminated through presentation at scientific conferences (IEEE International Conference on Robotics and Automation (ICRA), IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE-RAS International Conference on Humanoid Robots) and papers in leading scientific journals with impact factor (target journals are IEEE Transactions on Robotics, The International Journal of Robotics Research, Robotics and Autonomous Systems, Autonomous Robots, Robotica).
Significance for the country
The interest in dual arm systems in industrial setups has increased significantly in recent years. Already some major robot manufacturers have started to introduce dual arm systems into their portfolio, e.g. Yaskawa Motoman SDA10D, which is a dual-arm, 15-axis robot, ABB 14-axis dual-arm concept robot for small parts assembly applications, and the dual arm system Baxter from Rethink Robotics. The aim of these companies is to widen the application of robotics in industry and other. domains. On the other hand, dual-arm systems are in most cases still programmed as two detached robots with the ability to time-synchronize robot movements, but no ability for physical interaction. The results of the proposed project will contribute to more natural and efficient demonstration of dual-arm tasks and will enable easier (both autonomous and human influenced) adaptation. Thus the project will make a direct impact to the automation of production technologies.  There are many enterprises in Slovenia that produce robot workcells and automated production lines, both for domestic industry and for export. In addition, numerous companies, most of them export oriented, have been working on the introduction of robot technologies into their production lines. The proposed project can contribute to the increased automation of production in Slovenia, which is important because industrial production in Slovenia is becoming less profitable due to the relatively high labor costs. The existing companies in Slovenia active in the area of robotics are the assurance that the Slovenian companies will be able to use the results of the proposed projects. We would like to emphasize that JSI currently collaborates with one of the biggest robot manufacturers in the world, Yaskawa, in the area of programming by demonstration for industrial tasks. Yaskawa also has a branch in Ribnica, Slovenia, with whom both JSI and FEE already collaborate. Two knowledge transfer projects have already been signed between Yaskawa Japan and JSI, with dr. Ude as the leader of these projects. Yaskawa is greatly interested in the results of this project and intend to support the transfer of our results to industrial applications through joint industrial projects after the end of this project. As previously mentioned, Yaskawa has already developed a few models of dual arm robotic systems, but it does not yet have a solution for effective programming of such mechanisms. Thus this project is a great opportunity for Slovenian robotics research to be at the forefront of developments in industrial robotics.
Most important scientific results Interim report, final report
Most important socioeconomically and culturally relevant results Interim report, final report
Views history