We present what is destined to become the de-facto standard for hardware platforms for next generation cyber-physical systems. Heterogeneous System Architecture (HSA) is an initiative to harmonize the industry around a common architecture which is easier to program and is an open standard defining the key interfaces for parallel computation. Since HSA is supported by virtually all major players in the silicon market we can conjecture that HSA, with its capabilities and quirks, will highly influence both the hardware and software for next generation cyber-physical systems. In this paper we describe HSA and discuss how its nature will influence architectures of system software and application software. Specifically, we believe that the system software needs to both leverage the hyperparallel nature of HSA while providing predictable and efficient resource allocation to different parallel activities. The application software, on the other hand, should be isolated from the complexity of the hardware architecture but yet be able to efficiently use the full potential of the hyperparallel nature of HSA.
The HyTI (Hyperspectral Thermal Imager) mission, funded by NASA’s Earth Science TechnologyOffice InVEST (In-Space Validation of Earth Science Technologies) program, is the first US science satellite to leverage heterogenous SpaceCloud hardware with CPU and GPU acceleration. The mission will demonstrate how high spectral and spatial long-wave infrared image data can be acquired from a 6U CubeSat platform and perform advanced on-orbit real-time data processing and creating L1 and L2 products. The mission will use a spatially modulated interferometric imaging technique to produce spectro-radiometrically calibrated image cubes, with 25 channels between 8-10.7 μm, at 13cm-1resolution) at a ground sample distance of ~60 m. The small form factor of HyTI is made possible via a no-moving-parts Fabry-Perot interferometer and JPL’s cryogenically cooled HOTBIRD FPA technology. The value of HyTI to Earth scientists will be demonstrated via on-board processing of the raw instrument data to generate L1 and L2 products, with a focus on rapid deliveryof data regarding volcanic degassing, land surface temperature, and precision agriculture metrics.This presentation will provide an overview of the HyTI measurement approach, the onboard data reduction approach, and the spacecraft design. We will also update HyTI integration, testing, andfuture mission concepts based on the SpaceCloud Framework containerization of mission management and data applications.
This paper presents results and conclusions from design, manufacturing, and benchmarking of a heterogeneous computing low power fault tolerant computer, realized on an industrial Qseven® small form factor (SFF) platform. A heterogeneous computer in this context features multi-core processors (CPU), a graphical processing unit (GPU), and a field programmable gate array (FPGA). The x86 compatible CPU enables the use of vast amounts of commonly available software and operating systems, which can be used for space and harsh environments. The developed heterogeneous computer shares the same core architecture as game consoles such as Microsoft Xbox One and Sony Playstation 4 and has an aggregated computational performance in the TFLOP range. The processing power can be used for on-board intelligent data processing and higher degrees of autonomy in general. The module feature quad core 1.5 GHz 64 bit CPU (24 GFLOPs), 160 GPU shader cores (127 GFLOPs), and a 12 Mgate equivalent FPGA fabric with a safety critical ARM® Cortex-M3 MCU.
The Spherical Mobile Investigator for Planetary Surface (SMIPS) concept aims at making use of the latest developments within extreme miniaturization of space systems. The introduction of Microelectromechanical Systems (MEMSs) and higher level Multifunctional MicroSystems (MMSs) design solutions gives the robot high performance per weight unit. The untraditional spherical shape makes it easily maneuverable and thus provides a platform for scientific investigations of interplanetary bodies. Preliminary investigations of the SMIPS concept show several advantages over conventional robots and rovers in maneuverability, coverage, size, and mass. A locomotion proof-of-concept has been studied together with a new distributed on-board data system configuration. This paper discusses theoretical robot analysis, an overall concept, possible science, enabling technologies, and how to perform scientific investigations. A preliminary design of an inflatable multifunctional shell is proposed.
The last decade has seen a dramatic increase in small satellite missions for commercial, public, and government intelligence applications. Given the rapid commercialization of constellation-driven services in Earth Observation, situational domain awareness, communications including machine-to-machine interface, exploration etc., small satellites represent an enabling technology for a large growth market generating truly Big Data. Examples of modern sensors that can generate very large amounts of data are optical sensing, hyperspectral, Synthetic Aperture Radar (SAR), and Infrared imaging. Traditional handling and downloading of Big Data from space requires a large onboard mass storage and high bandwidth downlink with a trend towards optical links. Many missions and applications can benefit significantly from onboard cloud computing similarly to Earth-based cloud services. Hence, enabling space systems to provide near real-time data and enable low latency distribution of critical and time sensitive information to users. In addition, the downlink capability can be more effectively utilized by applying more onboard processing to reduce the data and create high value information products. This paper discusses current implementations and roadmap for leveraging high performance computing tools and methods on small satellites with radiation tolerant hardware. This includes runtime analysis with benchmarks of convolutional neural networks and matrix multiplications using industry standard tools (e.g., TensorFlow and PlaidML). In addition, a 1/2 CubeSat volume unit (0.5U) (10 x 10 x 5 cm(3)) cloud computing solution, called SpaceCloud (TM) iX5100 based on AMD 28 nm APU technology is presented as an example of heterogeneous computer solution. An evaluation of the AMD 14 nm Ryzen APU is presented as a candidate for future advanced onboard processing for space vehicles.
Hyperspectral thermal imagers provide characteristic information that conventional spectral imagers cannot offer. The proliferation of space assets and debris will require “eyes in the sky'' to track objects effectively. The current estimates as of 2022 state that more than 27,000 pieces of orbital debris are tracked by the Department of Defense’s global Space Surveillance Network (SSN). This number is expected to double in the next ten years with 57,000 satellites expected to be launched by 2029. Ground-based assets will not be able to track this vast number of orbital debris, and space-based monitoring capabilities will have to complement the tracking of assets and debris in the years to come. In this work, we present the Hyperspectral Thermal Imager (HyTI) CubeSat design, initially developed for Earth Observation, that can be adapted for Space Situational Awareness (SSA) applications with machine learning algorithms for fast object detection. With new advances in machine learning hardware and software, the categorization of orbital objects can help reveal features such as geometry, thermal signature, and size, among others. For example, spectral signatures can be leveraged to identify plumes of thrusters and unique characteristics of various materials used in different objects. HyTI is a 6U CubeSat funded by NASA’s Earth Science Technology Office (ESTO) In-Space Validation of Earth Science Technologies (InVEST) program. HyTI demonstrates how high spectral and spatial longwave infrared image data can be acquired from a 6U CubeSat platform. The long wave infrared detector uses a push-broom technique for producing accurate spectral and spatial data for moving targets. HyTI will demonstrate advanced on-orbit real-time data processing and the creation of scientific and operational data products. The payload uses aspatially modulated interferometric imaging technique to produce spectro-radiometrically calibrated image cubes, with 25 bands between 8-10.7 microns. The HyTI performance model indicates narrow band NEDTs of < 0.3 K. The small form factor of HyTI is made possible via the use of a no-moving-parts Fabry-Perot interferometer developedby the Hawaiʻi Institute of Geophysics and Planetology (HIGP) at the University of Hawaiʻi at Mānoa (UHM), and a Jet Propulsion Laboratory (JPL) cryogenically cooled High Operating Temperature (HOT) Barrier Infrared Detector (BIRD) focal plane array (FPA) technology. The level 0 (L0) data rate of the HyTI instrument is large. As a result, HyTI processes data from L0 to level 1 (L1, calibrated spectral radiance cubes) onboard. This is achieved using an advanced radiation-tolerant heterogeneous computer, the Unibap iX5-100 space computer, which offers CPU, GPU, and FPGA processing capability and has the option to add one or more neural network accelerators [12]. In this way, the L0 data volume is reduced by a factor of 13 before transmission to the ground as L1 data. Fully equipped, the iX5-100 can achieve several trillion computational operations per second (TOPS), which is essential for on-orbit detection of objects [13]. In this paper we provide an overview of the HyTI design and how it can be adapted for SSA observations and applications. We expand on the onboard data reduction and object detection approach, then provide an overview of the SpaceCloud Framework containerization of mission management and data applications.
Small satellites empower different applications for an affordable price. By dealing with a limited capacity for using instruments with high power consumption or high data-rate requirements, small satellite missions usually focus on specific monitoring and observation tasks. Considering that multispectral and hyperspectral sensors generate a significant amount of data subjected to communication channel impairments, bandwidth constraint is an important challenge in data transmission. That issue is addressed mainly by source and channel coding techniques aiming at an effective transmission. This paper targets a significant further bandwidth reduction by proposing an on-the-fly analysis on the satellite to decide which information is effectively useful before coding and transmitting. The images are tiled and classified using a set of detection algorithms after defining the least relevant content for general remote sensing applications. The methodology makes use of the red-band, green-band, blue-band, and near-infrared-band measurements to perform the classification of the content by managing a cloud detection algorithm, a change detection algorithm, and a vessel detection algorithm. Experiments for a set of typical scenarios of summer and winter days in Stockholm, Sweden, were conducted, and the results show that non-important content can be identified and discarded without compromising the predefined useful information for water and dry-land regions. For the evaluated images, only 22.3% of the information would need to be transmitted to the ground station to ensure the acquisition of all the important content, which illustrates the merits of the proposed method. Furthermore, the embedded platform’s constraints regarding processing time were analyzed by running the detection algorithms on Unibap’s iX10-100 space cloud platform.
Nowadays, it is a reality to launch, operate, and utilize small satellites at an affordable cost. However, bandwidth constraint is still an important challenge. For instance, multispectral and hyperspectral sensors generate a significant amount of data subjected to communication channel impairments, which is addressed mainly by source and channel coding aiming at an effective transmission. This paper targets a significant further bandwidth reduction by proposing an on-the-fly analysis technique on the satellite to decide which information is effectively useful for specific target applications, before coding and transmitting. The challenge would be detecting clouds and vessels having the measurements of red-band, green-band, blue-band, and near infrared band, aiming at sufficient probability of detection, avoiding false alarms. Furthermore, the embedded platform constraints must be satisfied. Experiments for typical scenarios of summer and winter days in Stockholm, Sweden, are conducted using data from the Mimir’s Well, the Saab AI-based data fusion system. Results show that non-relevant content can be identified and discarded, pointing out that for the cloudy scenarios evaluated, up to 73.1% percent of image content can be suppressed without compromising the useful information into the image. For the water regions in the scenarios containing vessels, results indicate that a stringent amount of data can be discarded (up to 98.5%) when transmitting only the regions of interest (ROI).
Autonomous driving is one of the main challenges of modern cars. Computer visions and intelligent on-board decision making are crucial in autonomous driving and require heterogeneous processors with high computing capability under low power consumption constraints. The progress of parallel computing using heterogeneous processing units is further supported by software frameworks like OpenCL, OpenMP, CUDA, and C++AMP. These frameworks allow the allocation of parallel computation on different compute resources. This, however, creates a difficulty in allocating the right computation segments to the right processing units in such a way that the complete system meets all its timing requirements. In this paper, we consider pre-runtime static allocations of parallel tasks to perform their execution either sequentially on CPU or in parallel using a GPU. This allows for improving any unbalanced use of GPU accelerators in a heterogeneous environment. By performing several heuristic algorithms, we show that the overuse of accelerators results in a bottle-neck of the entire system execution. The experimental results show that our allocation schemes that target a balanced use of GPU improve the system schedulability up to 90%.
During recent years, the interest in using heterogeneous computing architecture in industrial applications has increased dramatically. These architectures provide the computational power that makes them attractive for many industrial applications. However, most of these existing heterogeneous architectures suffer from the following limitations: difficulties of heterogeneous parallel programming and high communication cost between the computing units. To overcome these disadvantages, several leading hardware manufacturers have formed the HSA Foundation to develop a new hardware architecture: Heterogeneous System Architecture (HSA). In this paper, we investigate the suitability of using HSA for real-time embedded systems. A preliminary experimental study has been conducted to measure massive computing power and timing predictability of HSA.
In recent years, commercial exploitation of small satellites and CubeSats has rapidly increased. Time to market of processed customer data products is becoming an important differentiator between solution providers and satellite constellation operators. Timely and accurate data dissemination is the key to success in the commercial usage of small satellite constellations which is ultimately dependent on a high degree of autonomous fleet management and automated decision support. The traditional way for disseminating data is limited by on the communication capability of the satellite and the ground terminal availability. Even though cloud computing solutions on the ground offer high analytical performance, getting the data from the space infrastructure to the ground servers poses a bottleneck of data analysis and distribution. On the other hand, adopting advanced and intelligent algorithms onboard offers the ability of autonomy, tasking of operations, and fast customer generation of low latency conclusions, or even real-time communication with assets on the ground or other sensors in a multi-sensor configuration. In this paper, the advantages of intelligent onboard processing using advanced algorithms for Heterogeneous System Architecture (HSA) compliant onboard data processing systems are explored. The onboard data processing architecture is designed to handle a large amount of high-speed streaming data and provides hardware redundancy to be qualified for the space mission application domain. We conduct an experimental study to evaluate the performance analysis by using image recognition algorithms based on an open source intelligent machine library 'MIOpen' and an open standard 'OpenVX'. OpenVX is a cross-platform computer vision library.
Radiation effects research is crucial as it defines risk to both human bodies and spacecraft. Employing radiation-hardened products is one way to mitigate radiation effects on in-orbit systems. However, radiation effects prohibit most of the state-of-the-art commercial off-the-shelf (COTS) technologies from use in space. Furthermore, radiation effects on software components are less studied compared to hardware components. In this work, we introduce a simulation tool that analyzes the impact of radiation effects on schedulability of task sets executing on COTS system-on-chip (SoC) platforms in the in-orbit systems. In order to provide a meaningful verification environment, single-event effects (SEEs) are introduced as aleatory disturbances characterized by probability distribution of occurrence using their predefined models. The tool supports interoperability with several other tools as it uses the extensible markup language (XML) model files for input and output, i.e., for importing input task sets and radiation effects and exporting the simulation results.
Autonomous vehicular systems require computer vision and intelligent on-board decision making functionalities that include a mix of sequential and parallel workloads. The execution times of the workloads and power consumption in these functionalities can be lowered by utilizing the accelerators (e.g., GPU) instead of running the workloads entirely on the host processing units (CPU). However, allocating all the parallelizable workload to accelerators can create a computation bottleneck in the accelerators that, in turn, can have an adverse effect on schedulability of the systems. This paper presents a novel framework that can allocate the accelerate-intensive workloads to the accelerators as well as to the non-accelerated host processing units. Within the context of this framework, the paper introduces five offloading techniques to mitigate the accelerator-intensive workloads by utilizing excess capacity of non-accelerated processing units under dynamic scheduling in CPU-GPU heterogeneous processors. The proposed techniques are evaluated using simulation experiments. The evaluation results indicate that one of the proposed techniques can achieve up to 16% improvement in schedulability of the task sets compared to the traditional non-offloading technique.
On-board data processing is one of the prior on-orbit activities that improves the performance capability of in-orbit space systems such as deep-space exploration, earth and atmospheric observation satellites, and CubeSat constellations. However, on-board data processing encounters higher energy consumption compared to traditional on-board space systems. This is because the traditional space systems employ simple processing units such as single-core microprocessors as the systems do not require heavy data processing. Moreover, solving the radiation hardness problem is crucial in space, and adopting a new processing unit is challenging.
In this paper, we consider a Graphics Processing Unit (GPU) accelerated in-orbit space system for on-board data processing. According to prior works, there exist radiation-tolerant GPU, and the computing capability of systems is improved by using heterogeneous computing method. We conduct experimental observations of energy consumption and computing potential using this heterogeneous computing method in our GPU accelerated in-orbit space systems.The results show that the proper use of GPU increases computing potential with 10-140 times and consumes between 8-130 times less energy. Furthermore, the entire task system consumes 10-65% of less energy compared to the traditional use of processing units.
The technological advancements make the intelligent on-board data processing possible on a small scale of satellites and deep-space exploration spacecraft such as CubeSats. However, the operation of satellites may fall into critical conditions when the on-board data processing interferes strongly to the basic operation functionalities of satellites. In order to avoid these issues, there exist techniques such as isolation, partitioning, and virtualization. In this paper, we present an experimental study of isolation of on-board payload data processing from the basic operations of satellites using Docker. Docker is a leading technology in process level isolation as well as continuous integration and continuous deployment (CI/CD) method. This study continues with the prior study on heterogeneous computing method, which improves the schedulability of the entire system up to 90%. Based on this heterogeneous computing method, the comparison study has been conducted between the non-isolated and isolated environments.
On-board data processing is one of the prior on-orbit activities that it improves the performance capability of in-orbit space systems such as deep-space exploration, earth and atmospheric observation satellites, and CubeSat constellations. However, on-board data processing encounters with higher energy consumption compared to traditional space systems. Because traditional space systems employ simple processing units such as micro-controllers or a single-core processor as the systems require no heavy data processing on orbit. Moreover, solving the radiation hardness problem is crucial in space and adopting a new processing unit is challenging.
In this paper, we consider a GPU accelerated real-time system for on-board data processing. According to prior works, there exist radiation-tolerant GPU, and the computing capability of systems is improved by using heterogeneous computing method. We conduct experimental observations of power consumption and computing potential using this heterogeneous computing method in our GPU accelerated real-time system.The results show that the proper use of GPU increases computing potential with 10-140 times and consumes between 8-130 times less energy. Furthermore, the entire task system consumes 10-65% of less energy compared to the traditional use of processing units.