mdh.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Planning and Sequencing Through Multimodal Interaction for Robot Programming
Mälardalens högskola, Akademin för innovation, design och teknik, Inbyggda system. (ISS)
2014 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)
Abstract [en]

Over the past few decades the use of industrial robots has increased the efficiency as well as the competitiveness of several sectors. Despite this fact, in many cases robot automation investments are considered to be technically challenging. In addition, for most small and medium-sized enterprises (SMEs) this process is associated with high costs. Due to their continuously changing product lines, reprogramming costs are likely to exceed installation costs by a large margin. Furthermore, traditional programming methods of industrial robots are too complex for most technicians or manufacturing engineers, and thus assistance from a robot programming expert is often needed. The hypothesis is that in order to make the use of industrial robots more common within the SME sector, the robots should be reprogrammable by technicians or manufacturing engineers rather than robot programming experts. In this thesis, a novel system for task-level programming is proposed. The user interacts with an industrial robot by giving instructions in a structured natural language and by selecting objects through an augmented reality interface. The proposed system consists of two parts: (i) a multimodal framework that provides a natural language interface for the user to interact in which the framework performs modality fusion and semantic analysis, (ii) a symbolic planner, POPStar, to create a time-efficient plan based on the user's instructions. The ultimate goal of this work in this thesis is to bring robot programming to a stage where it is as easy as working together with a colleague.This thesis mainly addresses two issues. The first issue is a general framework for designing and developing multimodal interfaces. The general framework proposed in this thesis is designed to perform natural language understanding, multimodal integration and semantic analysis with an incremental pipeline. The framework also includes a novel multimodal grammar language, which is used for multimodal presentation and semantic meaning generation. Such a framework helps us to make interaction with a robot easier and more natural. The proposed language architecture makes it possible to manipulate, pick or place objects in a scene through high-level commands. Interaction with simple voice commands and gestures enables the manufacturing engineer to focus on the task itself, rather than the programming issues of the robot. The second issue addressed is due to inherent characteristics of communication with the use of natural language; instructions given by a user are often vague and may require other actions to be taken before the conditions for applying the user's instructions are met. In order to solve this problem a symbolic planner, POPStar, based on a partial order planner (POP) is proposed. The system takes landmarks extracted from user instructions as input, and creates a sequence of actions to operate the robotic cell with minimal makespan. The proposed planner takes advantage of the partial order capabilities of POP to execute actions in parallel and employs a best-first search algorithm to seek the series of actions that lead to a minimal makespan. The proposed planner can also handle robots with multiple grippers, parallel machines as well as scheduling for multiple product types.

Ort, förlag, år, upplaga, sidor
Västerås: Mälardalen University , 2014. , s. 164
Serie
Mälardalen University Press Dissertations, ISSN 1651-4238 ; 166
Nyckelord [en]
human robot interaction, industrial robots, planning, multimodal interaction
Nationell ämneskategori
Teknik och teknologier Datavetenskap (datalogi) Datorseende och robotik (autonoma system)
Forskningsämne
datavetenskap
Identifikatorer
URN: urn:nbn:se:mdh:diva-26474ISBN: 978-91-7485-175-5 (tryckt)OAI: oai:DiVA.org:mdh-26474DiVA, id: diva2:761392
Disputation
2014-12-08, Gamma, Mälardalens högskola, Västerås, 09:15 (Engelska)
Opponent
Handledare
Tillgänglig från: 2014-11-06 Skapad: 2014-11-06 Senast uppdaterad: 2018-01-11Bibliografiskt granskad
Delarbeten
1. Object selection using a spatial language for flexible assembly
Öppna denna publikation i ny flik eller fönster >>Object selection using a spatial language for flexible assembly
2009 (Engelska)Konferensbidrag, Publicerat paper (Refereegranskat)
Identifikatorer
urn:nbn:se:mdh:diva-13294 (URN)
Konferens
Swedish Workshop on Autonomous Robots (SWAR)
Tillgänglig från: 2011-11-17 Skapad: 2011-11-17 Senast uppdaterad: 2014-11-06Bibliografiskt granskad
2. A General Framework for Incremental Processing of Multimodal Inputs
Öppna denna publikation i ny flik eller fönster >>A General Framework for Incremental Processing of Multimodal Inputs
2011 (Engelska)Ingår i: Proceedings of the 13th international conference on multimodal interfaces, New York: ACM Press, 2011, s. 225-228Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Humans employ different information channels (modalities) such as speech, pictures and gestures in their commu- nication. It is believed that some of these modalities are more error-prone to some specific type of data and therefore multimodality can help to reduce ambiguities in the interaction. There have been numerous efforts in implementing multimodal interfaces for computers and robots. Yet, there is no general standard framework for developing them. In this paper we propose a general framework for implementing multimodal interfaces. It is designed to perform natural language understanding, multi- modal integration and semantic analysis with an incremental pipeline and includes a multimodal grammar language, which is used for multimodal presentation and semantic meaning generation.

Ort, förlag, år, upplaga, sidor
New York: ACM Press, 2011
Nationell ämneskategori
Data- och informationsvetenskap
Identifikatorer
urn:nbn:se:mdh:diva-13586 (URN)10.1145/2070481.2070521 (DOI)2-s2.0-83455176699 (Scopus ID)978-1-4503-0641-6 (ISBN)
Konferens
International Conference on Multimodal Interaction - ICMI 2011
Tillgänglig från: 2011-12-15 Skapad: 2011-12-15 Senast uppdaterad: 2018-01-12Bibliografiskt granskad
3. Intuitive Industrial Robot Programming Through Incremental Multimodal Language and Augmented Reality
Öppna denna publikation i ny flik eller fönster >>Intuitive Industrial Robot Programming Through Incremental Multimodal Language and Augmented Reality
2011 (Engelska)Ingår i: 2011 IEEE International Conferance on Robotics and Automation (ICRA 2011), IEEE , 2011, s. 3934-3939Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Developing easy to use, intuitive interfaces is crucial to introduce robotic automation to many small medium sized enterprises (SMEs). Due to their continuously changing product lines, reprogramming costs exceed installation costs by a large margin. In addition, traditional programming methods for industrial robots is too complex for an inexperienced robot programmer, thus external assistance is often needed. In this paper a new incremental multimodal language, which uses augmented reality (AR) environment, is presented. The proposed language architecture makes it possible to manipulate, pick or place the objects in the scene. This approach shifts the focus of industrial robot programming from coordinate based programming paradigm, to object based programming scheme. This makes it possible for non-experts to program the robot in an intuitive way, without going through rigorous training in robot programming.

Ort, förlag, år, upplaga, sidor
IEEE, 2011
Nationell ämneskategori
Elektroteknik och elektronik
Identifikatorer
urn:nbn:se:mdh:diva-13588 (URN)10.1109/ICRA.2011.5979887 (DOI)000324383403027 ()2-s2.0-84871705923 (Scopus ID)978-1-61284-386-5 (ISBN)
Konferens
2011 IEEE International Conferance on Robotics and Automation (ICRA 2011)
Tillgänglig från: 2011-12-15 Skapad: 2011-12-15 Senast uppdaterad: 2018-08-10Bibliografiskt granskad
4. Scheduling for Multiple Type Objects Using POPStar Planner
Öppna denna publikation i ny flik eller fönster >>Scheduling for Multiple Type Objects Using POPStar Planner
2014 (Engelska)Ingår i: Proceedings of the 19th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA'14), Barcelona, Spain, September, 2014, 2014, s. Article number 7005148-Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

In this paper, scheduling of robot cells that produce multiple object types in low volumes are considered. The challenge is to maximize the number of objects produced in a given time window as well as to adopt the  schedule for changing object types. Proposed algorithm, POPStar, is based on a partial order planner which is guided by best-first search algorithm and landmarks. The best-first search, uses heuristics to help the planner to create complete plans while minimizing the makespan. The algorithm takes landmarks, which are extracted from user's instructions given in structured English as input. Using different topologies for the landmark graphs, we show that it is possible to create schedules for changing object types, which will be processed in different stages in the robot cell. Results show that the POPStar algorithm can create and adapt schedules for robot cells with changing product types in low volume production.

Nationell ämneskategori
Robotteknik och automation
Forskningsämne
datavetenskap
Identifikatorer
urn:nbn:se:mdh:diva-26465 (URN)10.1109/ETFA.2014.7005148 (DOI)000360999100099 ()2-s2.0-84946692437 (Scopus ID)978-147994846-8 (ISBN)
Konferens
19th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA'14), Barcelona, Spain, 16-19 September, 2014
Tillgänglig från: 2014-11-05 Skapad: 2014-11-05 Senast uppdaterad: 2016-01-18Bibliografiskt granskad
5. Towards Creation of Robot Programs Through User Interaction
Öppna denna publikation i ny flik eller fönster >>Towards Creation of Robot Programs Through User Interaction
(Engelska)Manuskript (preprint) (Övrigt vetenskapligt)
Abstract [en]

This paper proposes a novel system for task-level programming of industrial robots. The user interacts with an industrial robot by giving instructions in a structured natural language and by selecting objects through an augmented reality interface. The proposed system consists of two parts. First, a multimodal framework that provides a natural language interface to the user. This framework performs modality fusion, semantic analysis and helps the user to interact with the system easier and more naturally. The proposed language architecture makes it possible to manipulate, pick or place objects in a scene through high-level commands. The second component is the POPStar planner, which is based on partial order planner (POP), that takes landmarks extracted from user instructions as input, and creates a sequence of actions to operate the robotic cell with minimal makespan. The proposed planner takes advantage of partial order capabilities of POP to plan execution of actions in parallel and employs a best-first search algorithm to seek a series of actions that lead to a minimal makespan. The proposed planner can as well handle robots with multiple grippers, and  parallel machines. Using different topologies for the landmark graphs, we show that it is possible to create schedules for changing object types, which are processed in different stages in the robot cell. Results show that the proposed system can create and adapt schedules for robot cells with changing product types in low volume production based on the user's instructions.

Nationell ämneskategori
Robotteknik och automation
Forskningsämne
datavetenskap
Identifikatorer
urn:nbn:se:mdh:diva-26473 (URN)
Tillgänglig från: 2014-11-06 Skapad: 2014-11-06 Senast uppdaterad: 2018-12-14Bibliografiskt granskad

Open Access i DiVA

fulltext(1020 kB)852 nedladdningar
Filinformation
Filnamn FULLTEXT02.pdfFilstorlek 1020 kBChecksumma SHA-512
c18d1debed56262bfdcb389d16d2d157a79d0d1f60d5ed984872973513ee41b4e4c51e9f61fa570ffb40bea2c0ba3f76a80498c9770d3d09e7d204feb7d22fcd
Typ fulltextMimetyp application/pdf

Sök vidare i DiVA

Av författaren/redaktören
Akan, Batu
Av organisationen
Inbyggda system
Teknik och teknologierDatavetenskap (datalogi)Datorseende och robotik (autonoma system)

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 852 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

isbn
urn-nbn

Altmetricpoäng

isbn
urn-nbn
Totalt: 885 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf