https://www.mdu.se/

mdu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Using Augmented Reality to Improve Productivity and Safety for Heavy Machinery Operators: State of the Art
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. CrossControl AB, Västerås, Sweden.ORCID iD: 0000-0002-1930-4181
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. CrossControl AB, Västerås, Sweden.ORCID iD: 0000-0001-7134-9574
2019 (English)In: Proceedings of the 17th International Conference on Virtual-Reality Continuum and its Applications in Industry, 2019, p. 8:1-8:9, article id 8Conference paper, Published paper (Refereed)
Abstract [en]

The machinery used in industrial applications, such as in agriculture, construction, and forestry, are increasingly equipped with digital tools that aim to aid the operator in task completion, improved productivity, and enhanced safety. In addition, as machines are increasingly connected, there are even more opportunities to integrate external information sources. This situation provides a challenge in mediating the information to the operator. One approach that could be used to address this challenge is the use of augmented reality. This enables the system-generated information to be combined with the user’s perception of the environment. It has the potential to enhance the operators’ awareness of the machine, the surroundings, and the operation that needs to be performed. In this paper, we review the current literature to present the state of the art, discuss the possible benefits, and the use of augmented reality in heavy machinery.

Place, publisher, year, edition, pages
2019. p. 8:1-8:9, article id 8
Keywords [en]
augmented reality, mixed reality, see-through interfaces, heavy machinery, heavy vehicle
National Category
Engineering and Technology Computer Systems
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:mdh:diva-46209DOI: 10.1145/3359997.3365689Scopus ID: 2-s2.0-85077121298OAI: oai:DiVA.org:mdh-46209DiVA, id: diva2:1374445
Conference
17th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry
Projects
ImmerSafe - Immersive Visual Technologies for Safety-critical ApplicationsITS-EASY Post Graduate School for Embedded Software and Systems
Funder
EU, Horizon 2020, 764951Available from: 2019-12-01 Created: 2019-12-01 Last updated: 2021-11-17Bibliographically approved
In thesis
1. Information Visualization Using Transparent Displays in Mobile Cranes and Excavators
Open this publication in new window or tab >>Information Visualization Using Transparent Displays in Mobile Cranes and Excavators
2022 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Operating heavy machinery, such as mobile cranes and excavators, is a complex task. While driving the machine, operators are also performing industrial tasks, e.g. lifting or digging, monitoring the machine’s status, and observing the surroundings. Modern heavy machinery is increasingly equipped with information systems that present supportive information to operators, so that they could perform their work safely and productively. Supportive information in heavy machinery is generally presented visually using head-down displays, which are placed in lower positions inside the cabin in order to avoid obstructing operators’ view. However, this placement makes visual information presented using head-down displays tend to be overlooked by operators, as the information is presented outside their field of view.

This dissertation investigates the possible use of transparent mediums for presenting visual information on the windshield of mobile cranes and excavators. By presenting information on the windshield, operators are expected to acquire visual information without diverting their attention away from the operational area. The design process includes (1) observing heavy machinery operators in natural settings through available videos on the Internet, (2) conducting an empirical study on the impact of different information placements, (3) reviewing the state of the art of display technologies that could be used to visualize information around the windshield of heavy machinery, (4) reviewing relevant safety guidelines to determine what kinds of critical information that operators should know, (5) conducting design workshops to generate visualization designs that represent critical information in operations of mobile cranes and excavators, (6) involving professional operators to evaluate and improve the proposed visualization designs, and (7) developing a functioning transparent display prototype that visualizes one kind of critical information that professional operators considered as the most important one.

The main finding from the observation using online videos suggested that heavy machinery operators spent considerable amount of time looking through the front windshield, and thus the front windshield could be used as a potential space for presenting visual information. The main finding of the empirical study also indicated that presenting information closer to the line of sight produced higher information acquisition and lower workload, compared to when information was presented farther from the line of sight. Based on the evaluation with professional operators, there seemed to be a good match between the proposed visualization designs and the operators' way of thinking, since the operators were able to understand and use the proposed visualization designs with little explanations. On the basis of the three most important findings above, there is a strong indication that placing the developed transparent display on the front windscreen of heavy machinery would make it easier for operators to perceive and process the presented information.

Abstract [sv]

Att köra stora mobila maskiner, exempelvis mobilkranar och grävmaskiner, är en komplex uppgift. Förutom att köra maskinen utför operatörerna också andra arbetsuppgifter, t.ex. lyfta eller gräva, övervaka maskinens status och observera omgivningen. Moderna stora maskiner blir i ökande gradutrustade med informationssystem som presenterar stödinformation för operatörerna så att de ska kunna utföra sitt arbete säkert och produktivt. Informationsstödet i stora maskiner presenteras visuellt med hjälp av displayer som placeras i lägre positioner inuti kabinen, s.k. \textit{head-down}, för att inte hindra operatörens sikt genom vindrutan. Den låga placeringen gör att operatörer tenderar att missa information som presenteras på head-down-skärmar eftersom informationen presenteras utanförderas synfält.

Den här avhandlingen undersöker hur transparenta skärmar kan användas för att presentera visuell information på vindrutan på mobilkranar och grävmaskiner. Genom att presentera information på vindrutan förväntas operatörerna inhämta visuell information utan att avleda deras uppmärksamhet från operationsområdet. Designprocessen omfattar att (1) observera operatörer av stora maskiner i naturliga miljöer genom tillgängliga videor på Internet, (2) genomföra en empirisk studie om effekterna av olika informationsplaceringar, (3) granska den senaste tekniken inom displayteknik som kan användas för att visualisera information på vindrutan hos stora maskiner, (4) granska relevanta säkerhetsriktlinjer för att avgöra vilken typ av kritisk information som operatörerna bör känna till, (5) genomföra designverkstäder för att generera visuell design som representerar kritisk information vid drift av mobilkranar och grävmaskiner, (6) involvera professionella operatörer för att utvärdera och förbättra de designförslagen och (7) utveckla en fungerande transparent displayprototyp som visualiserar den sorts kritisk information som professionella operatörer ansåg vara viktigast.

Det viktigaste fyndet från observationen med hjälp av online-videor visade att operatörer av stora maskiner främst tittar genom den främre vindrutan, därmed utgör potentiellt vindrutan en yta för presentation av visuell information. Det viktigaste resultatet från den empiriska studien indikerade också att information som presenterades närmare siktlinjen gjorde att användare inhämtade mer information med lägre arbetsbelastning jämfört med när information presenterades längre från siktlinjen. Baserat på utvärderingen med professionella operatörer tycktes det finnas en bra överenstämmelse mellan de föreslagna informationsvisualiseringarna och operatörernas sätt att tänka, eftersom operatörerna kunde förstå och använda de föreslagna informationsvisualiseringarna med korta små förklaringar. Med utgångspunkt i de tre viktigaste fynden ovan finns det en stark indikation till att en transparent display, placerad på den främre vindrutan på stora maskiner, underlättar för operatörer att uppfatta och bearbeta den presenterade informationen.

Place, publisher, year, edition, pages
Västerås: Mälardalen University, 2022
Series
Mälardalen University Press Dissertations, ISSN 1651-4238 ; 351
National Category
Interaction Technologies Design Human Computer Interaction Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:mdh:diva-56512 (URN)978-91-7485-539-5 (ISBN)
Public defence
2022-01-12, Lambda och on-line via Zoom/Teams, Mälardalens högskola, Västerås, 13:15 (English)
Opponent
Supervisors
Projects
Immersive Visual Technologies for Safety-critical Applications (ImmerSAFE)
Funder
EU, Horizon 2020, 764951
Available from: 2021-11-17 Created: 2021-11-12 Last updated: 2021-12-22Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Sitompul, Taufik AkbarWallmyr, Markus

Search in DiVA

By author/editor
Sitompul, Taufik AkbarWallmyr, Markus
By organisation
Embedded Systems
Engineering and TechnologyComputer Systems

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 326 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf