mdh.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Towards Creation of Robot Programs Through User Interaction
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. (ISS)
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
(English)Article in journal (Other academic) Submitted
Abstract [en]

This paper proposes a novel system for task-level programming of industrial robots. The user interacts with an industrial robot by giving instructions in a structured natural language and by selecting objects through an augmented reality interface. The proposed system consists of two parts. First, a multimodal framework that provides a natural language interface to the user. This framework performs modality fusion, semantic analysis and helps the user to interact with the system easier and more naturally. The proposed language architecture makes it possible to manipulate, pick or place objects in a scene through high-level commands. The second component is the POPStar planner, which is based on partial order planner (POP), that takes landmarks extracted from user instructions as input, and creates a sequence of actions to operate the robotic cell with minimal makespan. The proposed planner takes advantage of partial order capabilities of POP to plan execution of actions in parallel and employs a best-first search algorithm to seek a series of actions that lead to a minimal makespan. The proposed planner can as well handle robots with multiple grippers, and  parallel machines. Using different topologies for the landmark graphs, we show that it is possible to create schedules for changing object types, which are processed in different stages in the robot cell. Results show that the proposed system can create and adapt schedules for robot cells with changing product types in low volume production based on the user's instructions.

National Category
Robotics
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:mdh:diva-26473OAI: oai:DiVA.org:mdh-26473DiVA, id: diva2:761369
Available from: 2014-11-06 Created: 2014-11-06 Last updated: 2014-11-06Bibliographically approved
In thesis
1. Planning and Sequencing Through Multimodal Interaction for Robot Programming
Open this publication in new window or tab >>Planning and Sequencing Through Multimodal Interaction for Robot Programming
2014 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Over the past few decades the use of industrial robots has increased the efficiency as well as the competitiveness of several sectors. Despite this fact, in many cases robot automation investments are considered to be technically challenging. In addition, for most small and medium-sized enterprises (SMEs) this process is associated with high costs. Due to their continuously changing product lines, reprogramming costs are likely to exceed installation costs by a large margin. Furthermore, traditional programming methods of industrial robots are too complex for most technicians or manufacturing engineers, and thus assistance from a robot programming expert is often needed. The hypothesis is that in order to make the use of industrial robots more common within the SME sector, the robots should be reprogrammable by technicians or manufacturing engineers rather than robot programming experts. In this thesis, a novel system for task-level programming is proposed. The user interacts with an industrial robot by giving instructions in a structured natural language and by selecting objects through an augmented reality interface. The proposed system consists of two parts: (i) a multimodal framework that provides a natural language interface for the user to interact in which the framework performs modality fusion and semantic analysis, (ii) a symbolic planner, POPStar, to create a time-efficient plan based on the user's instructions. The ultimate goal of this work in this thesis is to bring robot programming to a stage where it is as easy as working together with a colleague.This thesis mainly addresses two issues. The first issue is a general framework for designing and developing multimodal interfaces. The general framework proposed in this thesis is designed to perform natural language understanding, multimodal integration and semantic analysis with an incremental pipeline. The framework also includes a novel multimodal grammar language, which is used for multimodal presentation and semantic meaning generation. Such a framework helps us to make interaction with a robot easier and more natural. The proposed language architecture makes it possible to manipulate, pick or place objects in a scene through high-level commands. Interaction with simple voice commands and gestures enables the manufacturing engineer to focus on the task itself, rather than the programming issues of the robot. The second issue addressed is due to inherent characteristics of communication with the use of natural language; instructions given by a user are often vague and may require other actions to be taken before the conditions for applying the user's instructions are met. In order to solve this problem a symbolic planner, POPStar, based on a partial order planner (POP) is proposed. The system takes landmarks extracted from user instructions as input, and creates a sequence of actions to operate the robotic cell with minimal makespan. The proposed planner takes advantage of the partial order capabilities of POP to execute actions in parallel and employs a best-first search algorithm to seek the series of actions that lead to a minimal makespan. The proposed planner can also handle robots with multiple grippers, parallel machines as well as scheduling for multiple product types.

Place, publisher, year, edition, pages
Västerås: Mälardalen University, 2014. p. 164
Series
Mälardalen University Press Dissertations, ISSN 1651-4238 ; 166
Keywords
human robot interaction, industrial robots, planning, multimodal interaction
National Category
Engineering and Technology Computer Sciences Computer Vision and Robotics (Autonomous Systems)
Research subject
Computer Science
Identifiers
urn:nbn:se:mdh:diva-26474 (URN)978-91-7485-175-5 (ISBN)
Public defence
2014-12-08, Gamma, Mälardalens högskola, Västerås, 09:15 (English)
Opponent
Supervisors
Available from: 2014-11-06 Created: 2014-11-06 Last updated: 2018-01-11Bibliographically approved

Open Access in DiVA

No full text in DiVA

Search in DiVA

By author/editor
Akan, Batu
By organisation
Embedded Systems
Robotics

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 170 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf