https://www.mdu.se/

mdu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Information Visualization Using Transparent Displays in Mobile Cranes and Excavators
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. (Ubiquitous Computing)ORCID iD: 0000-0002-1930-4181
2022 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Operating heavy machinery, such as mobile cranes and excavators, is a complex task. While driving the machine, operators are also performing industrial tasks, e.g. lifting or digging, monitoring the machine’s status, and observing the surroundings. Modern heavy machinery is increasingly equipped with information systems that present supportive information to operators, so that they could perform their work safely and productively. Supportive information in heavy machinery is generally presented visually using head-down displays, which are placed in lower positions inside the cabin in order to avoid obstructing operators’ view. However, this placement makes visual information presented using head-down displays tend to be overlooked by operators, as the information is presented outside their field of view.

This dissertation investigates the possible use of transparent mediums for presenting visual information on the windshield of mobile cranes and excavators. By presenting information on the windshield, operators are expected to acquire visual information without diverting their attention away from the operational area. The design process includes (1) observing heavy machinery operators in natural settings through available videos on the Internet, (2) conducting an empirical study on the impact of different information placements, (3) reviewing the state of the art of display technologies that could be used to visualize information around the windshield of heavy machinery, (4) reviewing relevant safety guidelines to determine what kinds of critical information that operators should know, (5) conducting design workshops to generate visualization designs that represent critical information in operations of mobile cranes and excavators, (6) involving professional operators to evaluate and improve the proposed visualization designs, and (7) developing a functioning transparent display prototype that visualizes one kind of critical information that professional operators considered as the most important one.

The main finding from the observation using online videos suggested that heavy machinery operators spent considerable amount of time looking through the front windshield, and thus the front windshield could be used as a potential space for presenting visual information. The main finding of the empirical study also indicated that presenting information closer to the line of sight produced higher information acquisition and lower workload, compared to when information was presented farther from the line of sight. Based on the evaluation with professional operators, there seemed to be a good match between the proposed visualization designs and the operators' way of thinking, since the operators were able to understand and use the proposed visualization designs with little explanations. On the basis of the three most important findings above, there is a strong indication that placing the developed transparent display on the front windscreen of heavy machinery would make it easier for operators to perceive and process the presented information.

Abstract [sv]

Att köra stora mobila maskiner, exempelvis mobilkranar och grävmaskiner, är en komplex uppgift. Förutom att köra maskinen utför operatörerna också andra arbetsuppgifter, t.ex. lyfta eller gräva, övervaka maskinens status och observera omgivningen. Moderna stora maskiner blir i ökande gradutrustade med informationssystem som presenterar stödinformation för operatörerna så att de ska kunna utföra sitt arbete säkert och produktivt. Informationsstödet i stora maskiner presenteras visuellt med hjälp av displayer som placeras i lägre positioner inuti kabinen, s.k. \textit{head-down}, för att inte hindra operatörens sikt genom vindrutan. Den låga placeringen gör att operatörer tenderar att missa information som presenteras på head-down-skärmar eftersom informationen presenteras utanförderas synfält.

Den här avhandlingen undersöker hur transparenta skärmar kan användas för att presentera visuell information på vindrutan på mobilkranar och grävmaskiner. Genom att presentera information på vindrutan förväntas operatörerna inhämta visuell information utan att avleda deras uppmärksamhet från operationsområdet. Designprocessen omfattar att (1) observera operatörer av stora maskiner i naturliga miljöer genom tillgängliga videor på Internet, (2) genomföra en empirisk studie om effekterna av olika informationsplaceringar, (3) granska den senaste tekniken inom displayteknik som kan användas för att visualisera information på vindrutan hos stora maskiner, (4) granska relevanta säkerhetsriktlinjer för att avgöra vilken typ av kritisk information som operatörerna bör känna till, (5) genomföra designverkstäder för att generera visuell design som representerar kritisk information vid drift av mobilkranar och grävmaskiner, (6) involvera professionella operatörer för att utvärdera och förbättra de designförslagen och (7) utveckla en fungerande transparent displayprototyp som visualiserar den sorts kritisk information som professionella operatörer ansåg vara viktigast.

Det viktigaste fyndet från observationen med hjälp av online-videor visade att operatörer av stora maskiner främst tittar genom den främre vindrutan, därmed utgör potentiellt vindrutan en yta för presentation av visuell information. Det viktigaste resultatet från den empiriska studien indikerade också att information som presenterades närmare siktlinjen gjorde att användare inhämtade mer information med lägre arbetsbelastning jämfört med när information presenterades längre från siktlinjen. Baserat på utvärderingen med professionella operatörer tycktes det finnas en bra överenstämmelse mellan de föreslagna informationsvisualiseringarna och operatörernas sätt att tänka, eftersom operatörerna kunde förstå och använda de föreslagna informationsvisualiseringarna med korta små förklaringar. Med utgångspunkt i de tre viktigaste fynden ovan finns det en stark indikation till att en transparent display, placerad på den främre vindrutan på stora maskiner, underlättar för operatörer att uppfatta och bearbeta den presenterade informationen.

Place, publisher, year, edition, pages
Västerås: Mälardalen University , 2022.
Series
Mälardalen University Press Dissertations, ISSN 1651-4238 ; 351
National Category
Interaction Technologies Design Human Computer Interaction Computer Sciences
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:mdh:diva-56512ISBN: 978-91-7485-539-5 (print)OAI: oai:DiVA.org:mdh-56512DiVA, id: diva2:1610954
Public defence
2022-01-12, Lambda och on-line via Zoom/Teams, Mälardalens högskola, Västerås, 13:15 (English)
Opponent
Supervisors
Projects
Immersive Visual Technologies for Safety-critical Applications (ImmerSAFE)
Funder
EU, Horizon 2020, 764951Available from: 2021-11-17 Created: 2021-11-12 Last updated: 2021-12-22Bibliographically approved
List of papers
1. Analyzing Online Videos: A Complement to Field Studies in Remote Locations
Open this publication in new window or tab >>Analyzing Online Videos: A Complement to Field Studies in Remote Locations
2019 (English)In: 17th IFIP TC 13 International Conference, Paphos, Cyprus, September 2–6, 2019, Proceedings, Part III, Cham, 2019, Vol. 117438, p. 371-389Conference paper, Published paper (Refereed)
Abstract [en]

The paper presents a complementary method, called online video study, to conducting field studies in remote locations, by using available videos on YouTube. There are two driving factors for the online video study. Firstly, there are some occasions where conducting field studies are difficult, for example, due to the remoteness of the location where the research subject is located. Secondly, there is a growing interest among researchers to use available data on the internet as their research data source. To give a context, the study specifically investigates how operators of forest harvesters work in their natural settings. The online video study was started by collecting suitable videos on YouTube using certain criteria. We found 26 videos that meet our criteria, which also provide diverse samples of forest harvesters, operators, and working situations. We used five prior field studies, which investigated forest harvesters-related issues, to evaluate the feasibility of our approach. The results of the online video study method are promising, since we are able to find answers for research questions that we have predefined. The paper does not only contribute to the understanding of how operators of forest harvesters work in natural settings, but also the feasibility of conducting the online video study, which can be utilized when the research subject is located in remote locations.

Place, publisher, year, edition, pages
Cham: , 2019
Series
Lecture Notes in Computer Science, ISSN 0302-9743
Keywords
Field studies, Forest harvesters, Online videos, YouTube
National Category
Computer Systems
Research subject
Computer Science
Identifiers
urn:nbn:se:mdh:diva-45582 (URN)10.1007/978-3-030-29387-1_21 (DOI)000702140400021 ()2-s2.0-85072868444 (Scopus ID)
Conference
17th IFIP TC13 International Conference on Human-Computer Interaction, INTERACT 2019; Paphos; Cyprus; 2 September 2019 through 6 September 2019
Projects
Immersive Visual Technologies for Safety-critical Applications (ImmerSAFE)ITS-EASY Post Graduate School for Embedded Software and Systems
Funder
EU, Horizon 2020, 764951
Available from: 2019-10-17 Created: 2019-10-17 Last updated: 2021-11-17Bibliographically approved
2. Evaluating Mixed Reality Notifications to Support Excavator Operator Awareness
Open this publication in new window or tab >>Evaluating Mixed Reality Notifications to Support Excavator Operator Awareness
2019 (English)In: 17th IFIP TC 13 International Conference, Paphos, Cyprus, September 2–6, 2019, Proceedings, Part I, Cham: Springer, 2019, Vol. 11746, p. 743-762Conference paper, Published paper (Refereed)
Abstract [en]

Operating heavy vehicles, for instance an excavator, requires a high level of attention to the operation done using the vehicle and awareness of the surroundings. Digital transformation in heavy vehicles aims to improve productivity and user experience, but it can also increase the operators mental load because of a higher demand of attention to instrumentation and controls, subsequently leading to reduced situation awareness. One way to mitigate this, is to display information within the operators’ field of view, which enhances information detectability through quick glances, using mixed reality interfaces. This work explores two types of mixed reality visualizations and compares them to a traditional display setup in a simulated excavator environment. We have utilized eye-tracking glasses to study users’ attention to the task, surrounding awareness, and interfaces, followed by a NASA-RTLX questionnaire to evaluate the users’ reported mental workload. The results indicate benefits for the mixed reality approaches, with lower workload ratings together with an improved rate in detection of presented information.

Place, publisher, year, edition, pages
Cham: Springer, 2019
Series
Lecture Notes in Computer Science, ISSN 0302-9743 ; 11746
Keywords
Excavator, Head-up display, Heavy-vehicles, Human-vehicle interaction, Mixed reality, Situational awareness
National Category
Computer Systems
Research subject
Computer Science
Identifiers
urn:nbn:se:mdh:diva-45583 (URN)10.1007/978-3-030-29381-9_44 (DOI)000561042200044 ()2-s2.0-85072870944 (Scopus ID)
Conference
17th IFIP TC13 International Conference on Human-Computer Interaction, INTERACT 2019; Paphos; Cyprus; 2 September 2019 through 6 September 2019
Projects
Immersive Visual Technologies for Safety-critical Applications (ImmerSAFE)ITS-EASY Post Graduate School for Embedded Software and Systems
Funder
EU, Horizon 2020, 764951
Available from: 2019-10-17 Created: 2019-10-17 Last updated: 2021-11-17Bibliographically approved
3. Using Augmented Reality to Improve Productivity and Safety for Heavy Machinery Operators: State of the Art
Open this publication in new window or tab >>Using Augmented Reality to Improve Productivity and Safety for Heavy Machinery Operators: State of the Art
2019 (English)In: Proceedings of the 17th International Conference on Virtual-Reality Continuum and its Applications in Industry, 2019, p. 8:1-8:9, article id 8Conference paper, Published paper (Refereed)
Abstract [en]

The machinery used in industrial applications, such as in agriculture, construction, and forestry, are increasingly equipped with digital tools that aim to aid the operator in task completion, improved productivity, and enhanced safety. In addition, as machines are increasingly connected, there are even more opportunities to integrate external information sources. This situation provides a challenge in mediating the information to the operator. One approach that could be used to address this challenge is the use of augmented reality. This enables the system-generated information to be combined with the user’s perception of the environment. It has the potential to enhance the operators’ awareness of the machine, the surroundings, and the operation that needs to be performed. In this paper, we review the current literature to present the state of the art, discuss the possible benefits, and the use of augmented reality in heavy machinery.

Keywords
augmented reality, mixed reality, see-through interfaces, heavy machinery, heavy vehicle
National Category
Engineering and Technology Computer Systems
Research subject
Computer Science
Identifiers
urn:nbn:se:mdh:diva-46209 (URN)10.1145/3359997.3365689 (DOI)2-s2.0-85077121298 (Scopus ID)
Conference
17th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry
Projects
ImmerSafe - Immersive Visual Technologies for Safety-critical ApplicationsITS-EASY Post Graduate School for Embedded Software and Systems
Funder
EU, Horizon 2020, 764951
Available from: 2019-12-01 Created: 2019-12-01 Last updated: 2021-11-17Bibliographically approved
4. Presenting Information Closer to Mobile Crane Operators' Line of Sight: Designing and Evaluating Visualization Concepts Based on Transparent Displays
Open this publication in new window or tab >>Presenting Information Closer to Mobile Crane Operators' Line of Sight: Designing and Evaluating Visualization Concepts Based on Transparent Displays
2020 (English)In: Proceedings - Graphics Interface 2020, Canadian Information Processing Society , 2020Conference paper, Published paper (Refereed)
Abstract [en]

We have investigated the visualization of safety information for mobile crane operations utilizing transparent displays, where the information can be presented closer to operators' line of sight with minimum obstruction on their view. The intention of the design is to help operators in acquiring supportive information provided by the machine, without requiring them to divert their attention far from operational areas. We started the design process by reviewing mobile crane safety guidelines to determine which information that operators need to know in order to perform safe operations. Using the findings from the safety guidelines review, we then conducted a design workshop to generate design ideas and visualisation concepts, as well as to delineate their appearances and behaviour based on the capability of transparent displays. We transformed the results of the workshop to a low-fidelity paper prototype, and then interviewed six mobile crane operators to obtain their feedback on the proposed concepts. The results of the study indicate that, as information will be presented closer to operators' line of sight, we need to be selective on what kind of information and how much information that should be presented to operators. However, all the operators appreciated having information presented closer to their line of sight, as an approach that has the potential to improve safety in their operations.

Place, publisher, year, edition, pages
Canadian Information Processing Society, 2020
Keywords
Human-centered computing-Visualization-Visualization application domains-Information visualization, Visualization design and evaluation methods, Design, Visualization, Design workshops, Operational area, Paper prototypes, Presenting informations, Safe operation, Safety guidelines, Safety information, Transparent displays, Cranes
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Computer Science
Identifiers
urn:nbn:se:mdh:diva-50908 (URN)10.20380/GI2020.41 (DOI)2-s2.0-85090761923 (Scopus ID)
Conference
Graphics Interface 2020, GI 2020; Toronto, Virtual, Online; Canada; 28 May 2020 through 29 May 2020.
Projects
ImmerSAFE - Immersive Visual Technologies for Safety-critical Applications
Funder
EU, Horizon 2020, 764951
Available from: 2020-09-25 Created: 2020-09-25 Last updated: 2021-11-17Bibliographically approved
5. Conceptual design and evaluation of windshield displays for excavators
Open this publication in new window or tab >>Conceptual design and evaluation of windshield displays for excavators
2020 (English)In: Multimodal Technologies and Interaction, E-ISSN 2414-4088, Vol. 4, no 4, p. 1-19, article id 86Article in journal (Refereed) Published
Abstract [en]

This paper investigates the possible visualization using transparent displays, which could be placed on the excavator’s windshield. This way, the information could be presented closer to operators’ line of sight, without fully obstructing their view. Therefore, excavator operators could acquire the supportive information provided by the machine without diverting their attention from operational areas. To ensure that there is a match between the supportive information and operators’ contextual needs, we conducted four different activities as parts of our design process. Firstly, we looked at four relevant safety guidelines to determine which information is essential to perform safe operations. Secondly, we reviewed all commercially available technologies to discover their suitability in the excavator context. Thirdly, we conducted a design workshop to generate ideas on how the essential information should look like and behave based on the performed operation and the chosen available technology. Fourthly, we interviewed seven excavator operators to test their understanding and obtain their feedback on the proposed visualization concepts. The results indicated that four out of six visualization concepts that we proposed could be understood easily by the operators and we also revised them to better suit the operators’ way of thinking. All the operators also positively perceived this approach, since all of them included at least three visualization concepts to be presented on the windshield.

Place, publisher, year, edition, pages
Basel, Switzerland: MDPI AG, 2020
Keywords
Design process, Excavators, Heavy machinery, Human-machine interface, Information visualization, Paper prototype, Transparent displays
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Computer Science
Identifiers
urn:nbn:se:mdh:diva-52861 (URN)10.3390/mti4040086 (DOI)000623577200019 ()2-s2.0-85097228074 (Scopus ID)
Projects
ImmerSAFE - Immersive Visual Technologies for Safety-critical Applications
Funder
EU, Horizon 2020, 764951
Available from: 2020-12-17 Created: 2020-12-17 Last updated: 2021-11-17Bibliographically approved
6. Developing a Windshield Display for Mobile Cranes
Open this publication in new window or tab >>Developing a Windshield Display for Mobile Cranes
2020 (English)In: Proceedings of the 37th International Symposium on Automation and Robotics in Construction, Kitakyushu, Japan, 2020Conference paper, Published paper (Refereed)
Abstract [en]

Modern heavy machinery, including mobile cranes, are increasingly equipped with information systems in the form of head-down displays that present supportive information. Due to the physical presence of head-down displays, they are usually placed far away from operators' line of sight, thus the information is unintentionally overlooked. This paper investigated the use of transparent displays in mobile cranes, which enables supportive information to be presented near operators' line of sight without fully obstructing their view. We developed both virtual and physical versions of the transparent display, which show information related to the lifting capacity and its influencing factors. The virtual version was developed using the Unity game engine with the data from the official load charts for a specific mobile crane model were incorporated into the mobile crane simulation. The physical version was made using off-the-shelves items, such as glasses, LEDs, super glues, insulated copper wires, and an Arduino. We then connected the virtual transparent display with its physical counterpart using an asset called Uduino, which enabled the Unity game engine to send data to the Arduino. The result in this study showed that the virtual transparent display and its real counterpart were functioning as what we intended. The developed prototype would be used in future user evaluations, thus we could determine to what extent this approach would affect the operators' performance and experience.

Place, publisher, year, edition, pages
Kitakyushu, Japan: , 2020
Keywords
human-machine interface, head-up display; transparent display; prototype; mobile cranes; heavy machinery
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Computer Science
Identifiers
urn:nbn:se:mdh:diva-52865 (URN)10.22260/ISARC2020/0200 (DOI)978-952-94-3634-7 (ISBN)
Conference
37th International Symposium on Automation and Robotics in Construction (ISARC 2020)
Projects
ImmerSAFE - Immersive Visual Technologies for Safety-critical Applications
Funder
EU, Horizon 2020, 764951
Available from: 2020-12-17 Created: 2020-12-17 Last updated: 2021-11-17Bibliographically approved

Open Access in DiVA

fulltext(20781 kB)771 downloads
File information
File name FULLTEXT02.pdfFile size 20781 kBChecksum SHA-512
74939b79d97e0e980ae1b1f8376f55e574733b2d0994dc5e9d58d269844b9d73a0d41c7607599c41eed2ed9fc01b1f7270613ed349ee319a1e3ee5e75286406e
Type fulltextMimetype application/pdf

Authority records

Sitompul, Taufik Akbar

Search in DiVA

By author/editor
Sitompul, Taufik Akbar
By organisation
Embedded Systems
Interaction TechnologiesDesignHuman Computer InteractionComputer Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 773 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 1049 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf