In the era of multiple industry trends and new technologies, avionics systems can benefit from several innovations. The complexity of modern electronics is increasing quickly and is being introduced as never before in new applications. At the algorithm level, the use of deep neural networks helps to solve problems that were never believed to be doable before. At the architecture level, hardware artificial intelligence accelerators, embedded graphical processing units, embedded sensors, etc., make it possible to create very powerful new functions. The list of new technologies is long. Besides technical challenges, system integrity and availability must be assured when integrating these new technologies into avionics functions. In this paper, we present emerging technologies and why a system-level approach is necessary when implementing these technologies. We also introduce supporting means for design assurance and fault-tolerance techniques. We illustrate the importance of a system-level approach through an example. Our example shows that when developing functions with new technologies and fault-tolerant architectures, the system safety assessment process is crucial for properly implementing a fail-safe design. It is also challenging due to potentially new failure modes.
In this paper, we discuss challenges when using neural networks (NNs) in safety-critical applications. We address the challenges one by one, with aviation safety in mind. We then introduce a possible implementation to overcome the challenges. Only a small portion of the solution has been implemented physically and much work is considered as future work. Our current understanding is that a real implementation in a safety-critical system would be extremely difficult. Firstly, to design the intended function of the NN, and secondly, designing monitors needed to achieve a deterministic and fail-safe behavior of the system. We conclude that only the most valuable implementations of NNs should be considered as meaningful to implement in safety-critical systems.
The five-year Master of Engineering Programme in Dependable Aerospace Systems, with dependability as its silver thread, started at Mälardalen University (MDH) in 2015. This paper presents selected ideas behind the creation of the programme, together with some preliminary analysis of current results and suggested enhancements for the programme’s fourth and fifth years.
This paper describes methods we used to improve our Master of Engineering programme in Dependable Aerospace Systems together with the industry. The target audience is mainly programme coordinators/managers who are in the process to develop their programmes for future demands. The two main questions we address are: Q1 – How do we ensure a good progression within a programme to ensure the industry’s current and future needs in engineering skills? and Q2 – How do we ensure students become acquainted with research during their studies? The results indicate that our suggested method to analyse programme progression through subject abilities supports developer of engineering programmes and that our approach to undergraduate research opportunities is a way forward to introduce students to research early.
A new assurance concept for new upcoming COTS-based computing platforms have to be based on a framework that allows to respond to various assurance challenges of different types of COTS hardware technologies. Therefore, we propose to use the generic assurance approach of the Overarching Properties, currently under research, together with assurance case as a tool to get the needed flexibility in the way to argument that the COTS assurance objectives are met. Indeed, to achieve this, it is necessary to develop a concept about COTS assurance in general which is realizable with an assurance case-based Overarching Property approach. This we have already provided in [1]. In this paper we have refined our work to integrate COTS technology specific assurance objectives and explained how their demonstration can be made within this new assurance concept in a coherent way. © 2019 IEEE.
An assurance strategy for new computing platforms in safety-critical avionics has to be flexible and take into account different types of commercial-of-the-shelf (COTS) hardware technologies. Completely new COTS technologies are already being introduced and successfully used in other domains. Good examples are heterogeneous platforms, hardware-based machine learning and approximate computing. Current avionics certification guidance material cannot cope with next generation of devices. We suggest using the generic assurance approach of the Overarching Properties (OPs) together with assurance cases to argument that COTS assurance objectives are met and to achieve the flexibility required for future computing platforms. We introduce a novel assurance cased-based OP approach in [1] and refine the work into a framework in [2]. Within this framework we are able to integrate COTS technology specific assurance objectives using a five-step process. In this paper, we show through some representative examples of emerging computing platforms that our strategy is a way forward for new platforms in safety-critical avionics.
Control systems are often an integral part of automation solutions where high reliability is crucial due to the high cost of downtime. The risk of unplanned downtime is typically reduced with redundant solutions. Additionally, safety-critical automation functions require high-integrity controllers. Today, the prevalent redundancy solution is a standby scheme, where one active primary controller drives the process while a standby backup controller is ready to take over in case of primary failure. This redundant controller pair can consist of high - integrity controllers. The automation industry is trending towards Ethernet as the sole communication medium. Our work presents an initial study of a high-integrity realization of a redundancy failure detection mechanism that guarantees only one primary controller, even in the case of network partitioning between the redundant controller pair. The failure detection is a lease-based function that leases the primary role from a central lease broker. This work discusses a high-integrity realization of the primary redundancy role leasing. We deduce and present the high-integrity-related requirements and a high-level design as an initial step towards a high-integrity realization of the redundancy role leasing.
Industrial controllers constitute the core of numerous automation solutions. Continuous control system operation is crucial in certain sectors, where hardware duplication serves as a strategy to mitigate the risk of unexpected operational halts due to hardware failures. Standby controller redundancy is a commonly adopted strategy for process automation. This approach involves an active primary controller managing the process while a passive backup is on standby, ready to resume control should the primary fail. Typically, redundant controllers are paired with redundant networks and devices to eliminate any single points of failure. The process automation domain is on the brink of a paradigm shift towards greater interconnectivity and interoperability. OPC UA is emerging as the standard that will facilitate this shift, with OPC UA PubSub as the communication standard for cyclic real-time data exchange. Our work investigates standby redundancy using OPC UA PubSub, analyzing a system with redundant controllers and devices in publisher-subscriber roles. The analysis reveals that failovers are not subscriber-transparent without synchronized publisher states. We discuss solutions and experimentally validate an internal stack state synchronization alternative.
Selecting the correct file system is critical for space applications where risks are present. This study systematically maps and tests Ext4 versus ZFS for onboard data processing on the iX10-100 and iX5-100 payload processors. The test sets are presented along with results on several performance metrics. The conclusion is that both ZFS and Ext4 are useful, but based on certain considerations of onboard data processing, Ext4 is better than the other.
In Machine Learning systems, several factors impact the performance of a trained model. The most important ones include model architecture, the amount of training time, the dataset size and diversity. In the realm of safety-critical machine learning the used datasets need to reflect the environment in which the system is intended to operate, in order to minimize the generalization gap between trained and real-world inputs. Datasets should be thoroughly prepared and requirements on the properties and characteristics of the collected data need to be specified. In our work we present a case study in which generating a synthetic dataset is accomplished based on real-world flight data from the ADS-B system, containing thousands of approaches to several airports to identify real-world statistical distributions of relevant variables to vary within our dataset sampling space. We also investigate what the effects are of training a model on synthetic data to different extents, including training on translated image sets (using domain adaptation). Our results indicate airport location to be the most critical parameter to vary. We also conclude that all experiments did benefit in performance from pre-training on synthetic data rather than using only real data, however this did not hold true in general for domain adaptation-translated images.
Machine Learning (ML) systems require representative and diverse datasets to accurately learn the objective task. Insupervised learning data needs to be accurately annotated, whichis an expensive and error-prone process. We present a methodfor generating synthetic data tailored to the use-case achievingexcellent performance in a real-world usecase. We provide amethod for producing automatically annotated synthetic visualdata of multirotor unmanned aerial vehicles (UAV) and otherairborne objects in a simulated environment with a high degreeof scene diversity, from collection of 3D models to generation ofannotated synthetic datasets (synthsets). In our data generationframework SynRender we introduce a novel method of usingNeural Radiance Field (NeRF) methods to capture photo-realistichigh-fidelity 3D-models of multirotor UAVs in order to automatedata generation for an object detection task in diverse environments. By producing data tailored to the real-world setting, ourNeRF-derived results show an advantage over generic 3D assetcollection-based methods where the domain gap between thesimulated and real-world is unacceptably large. In the spirit ofkeeping research open and accessible to the research communitywe release our dataset VISER DroneDiversity used in this project,where visual images, annotated boxes, instance segmentation anddepth maps are all generated for each image sample.
In Machine Learning systems, several factors impact the performance of a trained model. The most important ones include model architecture, the amount of training time, the dataset size and diversity. We present a method for analyzing datasets from a use-case scenario perspective, detecting and quantifying out-of-distribution (OOD) data on dataset level. Our main contribution is the novel use of similarity metrics for the evaluation of the robustness of a model by introducing relative Fréchet Inception Distance (FID) and relative Kernel Inception Distance (KID) measures. These relative measures are relative to a baseline in-distribution dataset and are used to estimate how the model will perform on OOD data (i.e. estimate the model accuracy drop). We find a correlation between our proposed relative FID/relative KID measure and the drop in Average Precision (AP) accuracy on unseen data.
The usage of complex Microcontroller Units (MCUs) in avionics systems constitutes a challenge in assuring their safety. They are not always developed according to the assurance requirements accepted by the aerospace industry. These Commercial off-the-shelf (COTS) hardware components usually target other domains like the telecommunication branch, because of the volume of sales and reduced liability. In the last years MCUs developed in compliance to the ISO 26262 have been released on the market for safety-related automotive applications. The avionics market could profit taking credit for some of the activities conducted in developing these MCUs. In this paper we present evaluation results based on comparing assurance activities from ISO 26262 that could be considered for compliance to relevant assurance guidance for COTS MCU in avionics.
In every avionics system, Commercial off-the-shelf (COTS) components play an important role by enabling more complex functions. Even in safety-critical systems, COTS hardware components are ubiquitous nowadays. Since the avionics manufacturer do not develop the COTS components themselves, traditional avionics Development Assurance (DA) methods cannot be used. Instead other assurance strategies are used. The problem is that the complexity of the COTS components continuously increase and that several different types COTS components exist, each requiring different assurance strategies. This article includes a literature review over Certification Authority (CA) materials and research reports over COTS hardware components certification related documents since the early 1990s. We then contribute by an approach that can structure this material to provide an overview on COTS assurance methods or activities through an assurance case. The early results show that assurance cases are a possible way to argument COTS assurance and that these cases reveal interconnections between the assurance methods and as such contribute to the overall goal of increased confidence in using COTS components.
In this paper, we present a novel method for detecting UAVs using diverse parallel neural networks with re-inference. The parallel networks are of type Convolutional Neural Networks (CNNs). We first set up a lowthreshold (2 respectively 20%) for each of the individual networks to detect a flying object. If all networks detecta flying object in the same area of a video frame with some overlap, we zoom into that area and redo the objectdetection and classification (re-inference step). To ensure correctness and reliability of the results from severalparallel CNNs, we introduce total confidence Tc as a measurement. We also introduce the intersection overunion for multiple parallel networks, IoUAll , and use that as threshold for calculating a reliable Tc . The resultsshow great improvements regarding accurate detection of flying drones, reduced mispredictions of otherobjects as drones, and fast response time when drones disappear from the scene.
The suggested model is developed for the purpose ofinvestigating the relationship between test coverage and itseffect on a given fault distribution in large complex safetycritical n-parameter software systems. The faults arerepresented by subspaces of the entire volume which representsthe entire input space of the system. The behavior of the systemis considered to be either correct or incorrect. Inside thesubspaces the system behaves erroneously. The shape of thesubspaces have no meaning only the size of its volume. Auniform distribution of test points leads to predictable andquantifiable fault detection.
Safety-critical software systems have traditionally been found in few domains, e.g., aerospace, nuclear and medical. As technology advances and software capability increases, such systems can be found in more and more applications, e.g., selfdriving cars, autonomous trains. This development will dramatically increase the operational exposure of such systems. All safety-critical applications need to meet exceptionally stringent criteria in terms of dependability. Proving compliance is a challenge for the industry and there is a lack of accepted methods to determine the status of safety-critical software. The regulatory bodies often require a certain amount of testing to be performed but do not, for software systems, require evidence of a given failure rate. This paper addresses quantification of test results. It examines both theoretical and practical aspects. The contribution of this paper is an equation that estimates the remaining undetected faults in the software system after testing. The equation considers partial test coverage. The theoretical results are validated with results from a large industry study (commercial military software). Additionally, the industry results are used to analyze the concept of entropy also known as Shannon information, which is shown to describe the knowledge gained from a test effort.
Safety-critical software systems need to meet exceptionally strict standards in terms of dependability. Best practice to achieve this is to follow and develop the software according to domain specific standards. These standards give guidelines on development and testing activities. The challenge is that even if you follow the steps of the appropriate standard you have no quantification of the amount of faults potentially still lingering in the system. This paper presents a way to statistically estimate the amount of undetected faults, based on test results.
Fault tree analysis is a system malfunction hazard evaluation quantitative and qualitative procedure. The method is well-known and widely used, especially in the safety systems domain, where it is a mandatory integral part of the so-called "Hazard Evaluation"documentation. This paper proposes an alternative or complementary deductive fault analysis method: it uses system topology to build a hypergraph representation of the system to identify component criticality and support loss of functionality probability evaluation. Once automated, the proposed method seems promising when the system engineers explore the different architectures. They may have indication about architecture's reliability without continuous feedback from the system safety team. The system safety team must check the solution once the engineers select the final architecture. They can also use the proposed method to validate the correctness of the fault tree analysis.