https://www.mdu.se/

The Black Pearl is a custom made autonomous underwater vehicle developed at Mälardalen University, Sweden. It is built in a modular fashion, including its mechanics, electronics and software. After a successful participation at the RoboSub competition in 2012 and winning the prize for best craftsmanship, this year we made minor improvements to the hardware, while the focus of the robot's evolution shifted to the software part. In this paper we give an overview of how the Black Pearl is built, both from the hardware and software point of view.

This paper presents GIMME2, an embedded stereovision system, designed to be compact, power efficient, cost effective, and high performing in the area of image processing. GIMME2 features two 10 megapixel image sensors and a Xilinx Zynq, which combines FPGA-fabric with a dual-core ARM CPU on a single chip. This enables GIMME2 to process video-rate megapixel image streams at real-time, exploiting the benefits of heterogeneous processing.

This paper presents GIMME (General Image Multiview Manipulation Engine), a highly flexible reconfigurable stand-alone mobile two-camera vision platform with stereo-vision capability. GIMME relies on reconfigurable hardware (FPGA) to perform application-specific low to medium-level image-processing at video rate. The Qseven-extension enables additional processing power. Thanks to its compact design, low power consumption and standardized interfaces (power and communication), GIMME is an ideal vision platform for autonomous and mobile robot applications.

Developing easy to use, intuitive interfaces is crucial to introduce robotic automation to many small medium sized enterprises (SMEs). Due to their continuously changing product lines, reprogramming costs exceed installation costs by a large margin. In addition, traditional programming methods for industrial robots is too complex for an inexperienced robot programmer, thus external assistance is often needed. In this paper a new incremental multimodal language, which uses augmented reality (AR) environment, is presented. The proposed language architecture makes it possible to manipulate, pick or place the objects in the scene. This approach shifts the focus of industrial robot programming from coordinate based programming paradigm, to object based programming scheme. This makes it possible for non-experts to program the robot in an intuitive way, without going through rigorous training in robot programming.

Over the past few decades the use of industrial robots has increased a company's efficiency as well as strengthening their competitiveness in the market.

Despite this fact, in many cases, robot automation investments are considered to be technically challenging as well as costly by small and medium sized enterprises (SME). We hypothesize that in order to make industrial robots more common within the SME sector, the robots should be reprogrammable by task experts rather than robot programming experts. Within this project we propose to develop a high level language for intelligent human robot interaction that relies on multi-sensor inputs providing an abstract instructional programming environment for the user. Eventually to bring robot programming to stage where it is as easy as working together with a colleague

Typical pick and place, and machine tending applications often require an industrial robot to be embedded in a cell and to communicate with other devices in the cell. Programming the program logic is a tedious job, requiring expert programming knowledge, and it can take more time than programming the specific robot movements itself. We propose a new system, which takes in the description of the whole manufacturing process in natural language as input, fills in the implicit actions, and plans the sequence of actions to accomplish the task described in minimal makespan using a modified partial planning algorithm. Finally we demonstrate that the proposed system can come up with a sensible plan for the given instructions.

In this paper we present a new simplified natural language that makes use of spatial relations between the objects in scene to navigate an industrial robot for simple pick and place applications. Developing easy to use, intuitive interfaces is crucial to introduce robotic automation to many small medium sized enterprises (SMEs). Due to their continuously changing product lines, reprogramming costs are far higher than installation costs. In order to hide the complexities of robot programming we propose a natural language where the use can control and jog the robot based on reference objects in the scene. We used Gaussian kernels to represent spatial regions, such as left or above. Finally we present some dialogues between the user and robot to demonstrate the usefulness of the proposed system.

In this paper a system, which is driven through natural language, that allows operators to select and manipulate objects in the environment using an industrial robot is proposed. In order to hide the complexities of robot programming we propose a natural language where the user can control and jog the robot based on reference objects in the scene. We used semantic networks to relate different types of objects in the scene

Humans employ different information channels (modalities) such as speech, pictures and gestures in their commu- nication. It is believed that some of these modalities are more error-prone to some specific type of data and therefore multimodality can help to reduce ambiguities in the interaction. There have been numerous efforts in implementing multimodal interfaces for computers and robots. Yet, there is no general standard framework for developing them. In this paper we propose a general framework for implementing multimodal interfaces. It is designed to perform natural language understanding, multi- modal integration and semantic analysis with an incremental pipeline and includes a multimodal grammar language, which is used for multimodal presentation and semantic meaning generation.

The Ravenscar profile for high integrity systems using Ada 95 is well defined in all real-time aspects. The complexity of the run-time system has been reduced to allow full utilization of formal methods for applications using the Ravenscar profile. In the Mana project a tool set is being developed including a formal model of a Ravenscar compliant run-time system, a gnat compatible run-time system, and an ASIS based tool to allow for the verification of a system including both COTS and code that is reused.

The paper contributes with three methods that together will make a complete tool-set for verification of mission critical applications. The first method is the transformation of existing Ada or VHDL code into an intermediate form. This form is used for verification by numerous different model checkers. The second method is a predictable runtime kernel that has both a verifiable formal model and is implemented in hardware to achieve full predictability. Finally, a method for transforming the intermediate form of the complete system into a hardware unit, the SafetyChip that performs runtime control of the system. This SafetyChip can catch 'out-of-state' behaviors.

The requirements and specification of a protocol for low level communication between the run-time systems in a distributed Ada environment is presented. This allows an Ada system to be separated into software resources and run-time controllers. Calls to the local run-time system of a node, concerning task management, are transformed into remote calls to the controller, that schedules all tasks in the application. The calls to the run-time system together with all messages, requests and replies, that are triggered as a consequence, are described. The controller will be implemented in hardware separate from the processors. Communication between processors and controllers are by means of high speed (Gigabit) networks. In the proposed system, partitioning and distribution of Ada programs can fully utilize the inherent and strong type checking in Ada.

Distribution of a single Ada program on a local area network is accomplished by partitioning the run-time system into two parts. A central scheduling module is responsible for task management. Distributed run-time executives handle context switches and remote entry calls; however all activities are supervised by the scheduler. The scheduler can be implemented in hardware in order to achieve high efficiency. A network based on optical fibers is necessary due to the high speed required for system calls. Asynchronous Transfer Mode is suggested as the protocol for the communication. We describe an implementation of the divided run-time system on an Ethernet network, using MC68030-based micro computers as targets and an Ada program executing on a Rational host as the scheduler.

This paper presents results and conclusions from design, manufacturing, and benchmarking of a heterogeneous computing low power fault tolerant computer, realized on an industrial Qseven^® small form factor (SFF) platform. A heterogeneous computer in this context features multi-core processors (CPU), a graphical processing unit (GPU), and a field programmable gate array (FPGA). The x86 compatible CPU enables the use of vast amounts of commonly available software and operating systems, which can be used for space and harsh environments. The developed heterogeneous computer shares the same core architecture as game consoles such as Microsoft Xbox One and Sony Playstation 4 and has an aggregated computational performance in the TFLOP range. The processing power can be used for on-board intelligent data processing and higher degrees of autonomy in general. The module feature quad core 1.5 GHz 64 bit CPU (24 GFLOPs), 160 GPU shader cores (127 GFLOPs), and a 12 Mgate equivalent FPGA fabric with a safety critical ARM® Cortex-M3 MCU.

This paper proposes a limited implementation of areabasedstereo matching for minimal resource utilization. It shows that it is possible to achieve an acceptable disparity map without the use of expensive resources. The matching accuracy for the single-row SAD can even outperform that of its full-row counterpart. Additionally, it excels in terms of frame rate and resource utilization, and is highly suitable for real-time stereo-vision systems.

This paper presents a set of approaches for increasing the accuracy of basic area-based stereo matching methods. It is targeting real-time FPGA systems for dense disparity map estimation. The methods are focused on low resource usage and maximized improvement per cost unit to enable the inclusion of an autonomous system in an FPGA. The approach performs on par with other area-matching implementations, but at substantially lower resource usage. Additionally, the solution removes the requirement for external memory for reconfigurable hardware together with the limitation in image size accompanying standard methods. As a fully piped complete on-chip solution, it is highly suitable for real-time stereo-vision systems, with a frame rate over 100 fps for Megapixel images.

By utilizing the power of hardware we can make a versatile, real-time system capable of both navigation and object gripping, based on basic feature detectors. By using an FPGA together with two cameras we can remove the need for descriptors for navigation by performing what we call spurious matching and the use of 3D landmarks. The approach bypasses the problem of outliers and reduces the time consuming task of data association, which slows many matching algorithms. We also present an approach to object gripping by taking advantage of the full capability of the feature detector.

A new approach to interact with an industrial robot using hand gestures is presented. System proposed here can learn first time user's hand gestures rapidly. This improves product usability and acceptability. Artificial neural networks trained with the evolution strategy technique are found to be suited for this problem. The gesture recognition system is an integrated part of a larger project for addressing intelligent human-robot interaction using a novel multi-modal paradigm. The goal of the overall project is to address complexity issues related to robot programming by providing a multi-modal user friendly interacting system that can be used by SMEs.

FPGA-based solutions have become more common in embedded systems these days. These systems need to communicate with external world. Considering high-speed and popularity of Ethernet communication, a reliable real-time Ethernet component inside FPGA is of special value. To that end, this paper presents a new solution for 100 Mb/s FPGA-based Ethernet communications with timing analysis. The solution deals with "Reduced Media-Independent Interface" in its physical layer. UDP is the network protocol which is implemented from physical to transport layer. For getting used in real-time applications, timing analysis is done in the communication system. Component based software engineering is used in the design and development processes. In order to test the components inside FPGA, two different approaches are utilized. Signal measurement in combination with introduced windows based application contributes much in testing and validation phases.

This paper mainly illustrates the background and development of soccer robot for RoboCup Middle Sized League, and puts more emphasis on both hardware and software design for the soccer robot based on the microcontroller AT90CAN128. In the hardware design, the sensor module is implemented by high precision laser range finder and electric compass for robot self-localization; USB-CAN converter are designed in the communication module; based on the introduction of main chips in use and workflow of the system, the two most important actuator in the soccer robot are designed separately, including motor module and kicking module. According to the hardware above, the software design shows the working flow of each module, including the laser ranger finder and electric compass in the sensor module, the transition mechanism between USB protocol and CAN protocol and easy-to-use bootloader for AT90CAN128 debugging online. The test shows both hardware and software designs for the soccer robot work well. © 2011 Springer-Verlag.

Current approaches to feature detection and matching

in images strive to increase the repeatability of the detector

and minimize the degree of outliers in the matching.

In this paper we present a conflicting approach; we suggest

that a lower performance feature detector can produce

a result more than adequate for robot navigation irrespectively

of the amount of outliers. By using an FPGA

together with two cameras we can remove the need for

descriptors by performing what we call spurious matching

and the use of 3D landmarks. The approach bypasses

the problem of outliers and reduces the time consuming

task of data association, which slows many matching algorithms.

In this paper a new method is presented for evaluat- ing visual stereo matching. Features are extracted and stereo paired according to their vertical positions in the image, with constraints on the horizontal position, leaving only sane, possible pairs. From feature pairs landmarks are constructed. The landmarks are evaluated using mo- tion estimation from a complementary relative naviga- tion system like odometry, only landmarks calculated from correctly paired features should survive the motion, thus transferring the problem into three dimensional space.

The Ravenscar tasking profile for Ada 95 has been designed to allow implementation of highly safety critical systems in Ada. Ravenscar defines a tasking run-time system with deterministic behaviour and low complexity. We provide a formal model of the primitives provided by Ravenscar including exceptions. This formal model can be used to verify safety properties of applications targeting a Ravenscar-compliant run-time system. As an illustration of this, we model a sample application using all features of Ravenscar and formally verify its correctness using the real-time model checker UPPAAL

The definition of the Ravenscar Tasking Profile for Ada 95 provides a definition of a tasking runtime system with deterministic behaviour and low enough complexity to permit a formal description and verification of the model. A complete run-time system is being modeled using the real-time model checker UPPAAL, and this work describes the handling of delay until. Since scheduling is not yet modelled a simple non-preemptive scheduler has been used when verifying the delay queue.

The Ravenscar tasking profile for Ada 95 has been designed to allow implementation of highly safety critical systems. Ravenscar defines a tasking system with deterministic behavior and low complexity. We provide a formal model using UPPAAL of the primitives provided by Ravenscar including exceptions. This formal model is used to verify the correctness of the Ravenscar model and can be used to verify safety properties of applications using the Ravenscar profile. As an illustration of this, we model a sample application using all features of Ravenscar and formally verify its correctness. Furthermore, an introduction to the Ravenscar model is given.

The SafetyChip proposes a strategy where parts of the effort invested in the formal verification during the development of a system can be reused during the system's operation. The strength in a formal verification of a system is that a system can mathematically be proven to fulfil certain requirements, e.g., timing requirements. The SafetyChip uses information from verification to monitor and police a system during run-time. The monitoring is done by surveillance of the applications communication with the run-time kernel. If deviance from the predefined verified behaviour is detected, the SafetyChip can signal (police) this in different ways, e.g., by generating interrupts the system can respond to. In our experiments we use systems written in Ravenscar compliant Ada code and have automated model extraction from source code to the models used to verify the system. This paper presents the functionality and design of the SafetyChip. Properties of an implementation of the Safety-Chip are also presented. 

This paper presents an intermediate notation used in a framework for verification of real-time properties. The framework aims at overcoming the need for the framework user to have significant knowledge of the verification specific detail that formal verification at some level is bound to impose on a model. In order to accomplish this, model extraction from source code of an initial formal model, a timing skeleton, is made automatically. The model refinement needed to transform the temporal skeleton into a model that can be verified is not done immediately. This allows postponement of the abstraction and specialisation needed for the verification which further improves the readability of the skeleton. The purpose of the timing skeleton is that it easily can be validated to represent the source code it was created from. The timing skeleton is then automatically refined with verification detail, and then hidden for the user, transformed into the notation of a verification tool. This transformation is hidden from the user. In order to reduce the complexity of the application model the framework uses a formally verified run-time kernel with a clear separation from the application. The kernel supports preemption, dynamic priorities and multiple processors.

Today's software based real-time operating systems (RTOS) have the ability to be configured in order to decrease memory footprint, important when the RTOS is used in resource restricted embedded environments. When the kernel is implemented in hardware, not only memory footprint is motivation for configuration for minimization, the number of logic cells occupied in the FPGA has to be considered as well.

We have modified existing hardware support, the real-time unit (RTU) for increased configurability. The modified RTU was compared regarding configuration and footprint with a corresponding system based on the commercial software RTOS MicroC/OS-II. Four different configuration settings were compared and the results show that the RTU memory footprint was 24% to 38% of the size of the MicroC/OS-II footprint. In FPGA area, the smallest configuration of the RTU occupied only 50% of the logic cells used by the largest configuration. Configurability of an RTOS with hardware support, like the RTU, decreases project cost since it is more likely to be able to use a low cost FPGA when adjusting implemented functionality to only include what is required by the application.

Senior citizens and patients recovering from surgery or using strong medications with severe side effects tend to fall unexpectedly. The consequences of such an uncontrolled fall could be worse than the original malady, especially when there is no communication with the care-takers. We describe a fall-detector device capable of distinguishing falls from normal daily activities. Based on three-axis accelerometer and advanced data processing, the microcontroller emits an alarm requesting help in the case of a physical fall. We design and construct the fall-detector prototype for either inside or outside use. In order to determine the device performance, fifty instances of each fall event have been evaluated; all of them detected as fall event. In the case of daily activities, the only movement that produces an alarm is the transition from standing up to lying in 5% of the occurrences.

FPGA has been used in many robotics projects for real-time image processing. It provides reliable systems with low execution time and simplified timing analysis. Many of these systems take a lot of time in development and testing phases. In some cases, it is not possible to test the system in real environments very often, due to accessibility, availability or cost problems. This paper is the result of a case study on vision systems for two robotics projects in which the vision team consisted of seven students working for six months fulltime on developing and implementing different image algorithms. While FPGA has been used for real-time image processing, some steps have been taken in order to reduce the development and testing phases. The main focus of the project is to integrate different testing methods with FPGA development. It includes a component based solution that uses a two-way communication with a PC controller for system evaluation and testing. Once the data is acquired from the vision board, the system stores it and simulates the same environment that has been captured earlier by feeding back the obtained data to FPGA. This approach addresses and implements a debugging methodology for FPGA based solutions which accelerate the development phase. In order to transfer massive information of images, RMII which is an interface for Ethernet communication, has been investigated and implemented. The provided solution makes changes easier, saves time and solves the problems mentioned earlier.

A versatile General Camera Model, GCM, has been developed, and is described in detail. The model is general in the sense that it can capture both fisheye and conventional as well as catadioptric cameras in a unified framework. The camera model includes efficient handling of non-central cameras as well as compensations for decentring distortion. A novel way of analysing radial distortion functions of camera models leads to a straightforward improvement of conventional models with respect to generality, accuracy and simplicity. Different camera models are experimentally compared for two cameras with conventional and fisheye lenses, and the results show that the overall performance is favourable for the GCM. (C) 2011 Elsevier Inc. All rights reserved.

This paper presents an embedded vision system based on reconfigurable hardware (FPGA) and two CMOS cameras to perform stereo image processing and 3D mapping for autonomous navigation. We propose an EKF based visual SLAM and sparse feature detectors to achieve 6D localization of the vehicle in non flat scenarios. The system can operate regardless of the odometry information from the vehicle since visual odometry is used. As a result, the final system is compact and easy to install and configure.

This paper presents an embedded vision system based on reconfigurable hardware (FPGA) to perform stereo image processing and 3D mapping of sparse features for autonomous navigation and obstacle detection in industrial settings. We propose an EKF based visual SLAM to achieve a 6D localization of the vehicle even in non flat scenarios. The system uses vision as the only source of information. As a consequence, it operates regardless of the odometry from the vehicle since visual odometry is used. © 2013 IEEE.

The research presented addresses the emerging topic of AGVs (Automated Guided Vehicles) specifically related to industrial sites. The work presented has been carried out in the frame of the MALTA project (Multiple Autonomous forklifts for Loading and Transportation Applications), a joint research project between industry and university, funded by the European Regional Development and Robotdalen, in partnership with theSwedish Knowledge Foundation. The project objective is to create fully autonomous forklift trucks for paper reel handling. The result is expected to be of general benefit for industries that use forklift trucks in their material handling through higher operating efficiency and better flexibility with reduced risk for accidents and handling damages than if only manual forklift trucks are used. A brief overview of the state of the art in AGVs will be reported in order to better understand the new challenges and technologies. Among the emerging technologies used for vehicle automation, vision is one of the most promising in terms of versatility and efficiency, with a high potential to drastically reduce the costs.

This paper proposes a stereo vision based localization and mapping strategy for vehicular navigation within industrial environments using natural landmarks. The work proposed is strictly related to factory automation, since focus is on industrial vehicle autonomous navigation for material handling, in order to increase the operating efficiency with reduced risk for accidents. The stereovision system, proposed as the main sensor, provides the necessary feedback to navigate and simultaneously calibrate the stereocamera parameters (like the camera separation, focal length, camera placement with respect to the robot, etc.). It uses the natural landmarks already present in the environment without additional infrastructures. Some simulation and experimental results are presented in order to explain the proposed method and current status.