https://www.mdu.se/

Stereo correspondence is a well-established research topic and has spawned categories of algorithms combining several processing steps and strategies. One core part to stereo correspondence is to determine matching cost between the two images, or patches from the two images. Over the years several different cost metrics have been proposed, one being the Census Transform (CT). The CT is well proven for its robust matching, especially along object boundaries, with respect to outliers and radiometric differences. The CT also comes at a low computational cost and is suitable for hardware implementation. Two key developments to the CT are non-centric and sparse comparison schemas, to increase matching performance and/or save computational resources. Recent CT algorithms share both traits but are handcrafted, bounded with respect to symmetry, edge lengths and defined for a specific window size. To overcome this, a Genetic Algorithm (GA) was applied to the CT, proposing the Genetic Algorithm Census Transform (GACT), to automatically derive comparison schemas from example data. In this paper, FPGA-based hardware acceleration of GACT, has enabled evaluation of census windows of different size and shape, by significantly reducing processing time associated with training. The experiments show that lateral GACT windows produce better matching accuracy and require less resources when compared to square windows.

The Census Transform (CT) is a well proven method for stereo vision that provides robust matching, with respect to object boundaries, outliers and radiometric distortion, at a low computational cost. Recent CT methods propose patterns for pixel comparison and sparsity, to increase matching accuracy and reduce resource requirements. However, these methods are bounded with respect to symmetry and/or edge length. In this paper, a Genetic algorithm (GA) is applied to find a new and powerful CT method. The proposed method, Genetic Algorithm Census Transform (GACT), is compared with the established CT methods, showing better results for benchmarking datasets. Additional experiments have been performed to study the search space and the correlation between training and evaluation data.

Requirements engineering is a time-consuming process, and it can benefit significantly from automated tool support. Ambiguity detection in natural language requirements is a challenging problem in the requirements engineering community. Several Natural Language Processing tools and techniques have been developed to improve and solve the problem of ambiguity detection in natural language requirements. However, there is a lack of empirical evaluation of these tools. We aim to contribute the understanding of the empirical performance of such solutions by evaluating four tools using the dataset of 180 system requirements from the electric train propulsion system provided to us by our industrial partner Alstom. The tools that were selected for this study are Automated Requirements Measurement (ARM), Quality Analyzer for Requirement Specifications (QuARS), REquirements Template Analyzer (RETA), and Requirements Complexity Measurement (RCM). Our analysis showed that selected tools could achieve high recall. Two of them had the recall of 0.85 and 0.98. But they struggled to achieve high precision. The RCM, an in-house developed tool by our industrial partner Alstom, achieved the highest precision in our study of 0.68. 

Industrial control systems have been targeted by numerous cyber attacks over the past few decades which causes different problems related to data privacy, financial losses and operational failures. One potential approach to detect these attacks is by analyzing network data using machine learning and employing network anomaly detection techniques. However, the nature of these systems often involves their geographical dispersion across multiple zones, which poses a challenge in applying local machine learning methods for detecting anomalies. Additionally, there are instances where sharing complete operational data between different zones is restricted due to security concerns. As a result, a promising solution emerges by implementing a federated model for anomaly detection in these systems. In this study, we investigate the application of machine learning techniques for anomaly detection in network data, considering centralized, local, and federated approaches. We implemented the local and centralized methods using several simple machine-learning techniques and observed that Random Forest and Artificial Neural Networks exhibited superior performance compared to other methods. As a result, we extended our analysis to develop a federated version of Random Forest and Artificial Neural Network. Our findings reveal that the federated model surpasses the performance of the local models, and achieves comparable or even superior results compared to the centralized model, while it ensures data privacy and maintains the confidentiality of sensitive information.

Modular automation provides a challenge for traditional physics simulators, especially if they are used as a simulator in the loop of a development or research project looking at behavior from a systems level. In this paper, we present extensions of a previously developed simulation environment that is tailored to provide these characteristics. The extensions include simulation engine level improvements, such as including better modeling of the material flow, and sensor anomaly injections to model sensor faults or tampering, as well as system-level enhancements and functionality including certificate handling and anomaly detection methods using machine learning. This simulation environment has proven useful for education as well as research and engineering work, and with the provided extensions several new directions of use can be envisioned. The system is demonstrated in the use case of a modular ice-cream factory, including all the new and enhanced functionalities.

Differential Evolution (DE) has become one of the most important metaheuristics during the recent years, obtaining attractive results in solving many engineering optimization problems. However, the performance of DE is not always strong when seeking optimal solutions. It has two major problems in real world applications. First, it can easily get stuck in a local optimum or fail to generate better solutions before the population has converged. Secondly, its performance is significantly influenced by the control parameters, which are problem dependent and which vary in different regions of space under exploration.  It usually entails a time consuming trial-and-error procedure to set suitable parameters for DE in a specific problem, particularly for those practioners with limited knowledge and experience of using this technique.

This thesis aims to develop new DE algorithms to address the two aforementioned problems. To mitigate the first problem, we studied the hybridization of DE with local search techniques to enhance the efficiency of search. The main idea is to apply a local search mechanism to the best individual in each generation of DE to exploit the most promising regions during the evolutionary processs so as to speed up the convergence or increase the chance to scape from local optima. Four local search strategies have been integrated  and tested in the global DE framework, leading to variants of the memetic DE algorithms with different properties concerning diversification and intensification. For tackling the second problem, we propose a greedy adaptation method for dynamic adjustment of the control parameters in DE. It is implemented by conducting greedy search repeatedly during the run of DE to reach better parameter assignments in the neighborhood of a current candidate. The candidates are assessed by considering both, the success rate and also fitness improvement of trial solutions against the target ones. The incorporation of this greedy parameter adaptation method into standard DE has led to a new adaptive DE algorithm, referred to as Greedy Adaptive Differential Evolution (GADE).

The methods proposed in this thesis have been tested in different benchmark problems and compared with the state of the art algorithms, obtaining competitive results. Furthermore, the proposed GADE algorithm has been applied in an industrial scenario achieving more accurate results than those obtained by a standard DE algorithm. 

Differential Evolution (DE) is a population-based algorithm that belongs to the Evolutionary algorithm family. During recent years, DE has become a popular algorithm in optimization due to its strength solving different types of optimization problems and due to its easy usage and implementation.

However, how to choose proper mutation strategy and control parameters for DE presents a major difficulty in many real applications. Since both mutation strategy and DE control parameters are highly problem dependent, they have to be adapted to suite different search spaces and different problems. Failure in proper assignment for them will cause slow convergence in search or stagnation with a local optimum.

Many researches have been conducted to tackle the above issues. The major efforts have been made in the following three directions. First, some works have been proposed that adapt the selection between various mutation strategies. But the choice of strategies in these methods has not considered the difference of quality of individuals in the population, which means that all individuals will acquire the same probability to select a mutation strategy from the candidates. This does not seem a very desired practice since solutions of large difference would require different mutation operators to reach improvement. Second, many works have been focusing on the adaptation of the control parameters of DE (mutation factor (F) and crossover rate (CR)). They mainly rely on previous successful F and CR values to update the probability functions that are used to generate new F and CR values. By doing this, they ignore the stochastic nature of the operators in DE such that weak F and CR values can also get success in producing better trial solutions. The use of such imprecise experiences of success would prevent the DE parameters from being adapted towards the most effective values in coming generations. Third, various local search methods have been incorporated into DE to enhance exploitation in promising regions so as to speed up the convergence to optima. It is important to properly adjust the characteristics of the local search in DE to achieve well balanced exploratory/exploitative behavior to solve complex optimization problems.

This thesis aims to further improve the performance of DE by new adaptation and local search methods. The main results can be summarized in the following three aspects:

1) Proposal of a new rank-based mutation adaptation method, which takes into account the quality of solutions in the population when adapting the selection probabilities of mutation strategies. This makes possible to treat solutions with distinct ranks (in quality) differently by using different selection probabilities for mutation operators.

2) Development of improved parameter adaptation methods for DE, which emphasizes more reliable and fair evaluation of candidates (F and CR assignments) during the search process. It is suggested that greedy search being used as a fast and cheap technique to look for better parameter assignment for F and CR respectively in the neighborhood of a current candidate. Further, a joint parameter adaptation method is proposed that enables continuous update of the selection probabilities for F and CR pairs based on feedback acquired during the search.

3) Proposal of new methods for better incorporation of local search into a DE algorithm. The Eager Random Search method is investigated as local search inside DE, which exhibits different exploratory-exploitative characteristics by using different probability density functions. More importantly, we propose a novel memetic framework in which Alopex local search (ALS) is performed in collaboration with a DE algorithm. The framework favors seamless connection between exploration and exploitation in the sense that the behavior of exploitation by ALS can be controlled by the status of global exploration by DE.

The proposed methods and algorithms have been tested in a number of benchmark problems, obtaining competitive results compared with the state-of-the-art algorithms. Additionally, the Greedy Adaptive DE (GADE) algorithm (developed based on greedy search for DE parameters) has been tested in a real industrial problem, i.e., finding best component parameters to optimize the performance of harmonic filters for power transmission. GADE is shown to produce better harmonic filter systems with lower harmonic distortion than the standard DE.

Motor Imagery (MI) classification from neural activity is thought to represent valuable information that can be provided as real-time feedback during rehabilitation after for example a stroke. Previous studies have suggested that MI induces partly subject-specific EEG activation patterns, suggesting that individualized classification models should be created. However, due to fatigue of the user, only a limited number of samples can be recorded and, for EEG recordings, each sample is often composed of a large number of features. This combination leads to an undesirable input data set for classification. In order to overcome this constraint, we propose a new methodology to create and select features from the EEG signal in two steps. First, the input data is divided into different windows to reduce the cardinality of the input. Secondly, a Hierarchical Genetic Algorithm is used to select relevant features using a novel fitness function which combines the data reduction with a correlation feature selection measure. The methodology has been tested on EEG oscillatory activity recorded from 6 healthy volunteers while they performed an MI task. Results have successfully proven that a classification above 75% can be obtained in a restrictive amount of time (0.02 s), reducing the number of features by almost 90%.

Accurate system modelling is an important prerequisite for optimized process control in modern industrial scenarios. The task of parameter identification for a model can be considered as an optimization problem of searching for a set of continuous parameters to minimize the discrepancy between the model outputs and true output values. Differential Evolution (DE), as a class of population-based and global search algorithms, has strong potential to be employed here to solve this problem. Nevertheless, the performance of DE is rather sensitive to its two running parameters: scaling factor and crossover rate. Improper setting of these two parameters may cause weak performance of DE in real applications. This paper presents a new adaptive algorithm for DE, which does not require good parameter values to be specified by users in advance. Our new algorithm is established by integration of greedy search into the original DE algorithm. Greedy search is conducted repeatedly during the running of DE to reach better parameter assignments in the neighborhood. We have applied our adaptive DE algorithm for process model identification in a Furnace Optimized Control System (FOCS). The experiment results revealed that our adaptive DE algorithm yielded process models that estimated temperatures inside a furnace more precisely than those produced by using the original DE algorithm.

With the increasing use of the internet and reliance on computer-based systems for our daily lives, any vulnerability in those systems is one of the most important issues for the community. For this reason, the need for intelligent models that detect malicious intrusions is important to keep our personal information safe. In this paper, we investigate several supervised (Artificial Neural Network, Support Vector Machine, Random Forest, Linear Discriminant Analysis, and K-Nearest Neighbors) and unsupervised (K-means, Mean-shift, and DBSCAN) machine learning algorithms, in the context of anomaly-based Intrusion Detection Systems. We are using four different IDS benchmark datasets (KDD99, NSL-KDD, UNSW-NB15, and CIC-IDS-2017) to evaluate the performance of the selected machine learning algorithms for both intrusion detection and attack classification. The results have shown that Random Forest is the most suitable algorithm regarding model accuracy and execution time.

With the increasing use of computer networks and distributed systems, network security and data privacy are becoming major concerns for our society. In this paper, we present an approach based on an autoencoder trained with differential evolution for feature encoding of network data with the goal of improving security and reducing data transfers. One of the novel elements used in differential evolution for intrusion detection is the enhancements in the fitness function by adding the performance of a machine learning algorithm. We conducted an extensive evaluation of six machine learning algorithms for network intrusion detection using encoded data from well-known publicly available network datasets UNSW-NB15. The experiments clearly showed the supremacy of random forest, support vector machine, and K-nearest neighbors in terms of accuracy, and this was not affected to a high degree by reducing the number of features. Furthermore, the machine learning algorithm that was used during training (Linear Discriminant Analysis classifier) got a 14 percentage points increase in accuracy. Our results also showed clear improvements in execution times in addition to the obvious secure aspects of encoded data. Additionally, the performance of the proposed method outperformed one of the most commonly used feature reduction methods, Principal Component Analysis. 

There is a growing number of network attacks and the data on the network is more exposed than ever with the increased activity on the Internet. Applying Machine Learning (ML) techniques for cyber-security is a popular and effective approach to address this problem. However, the data which is used by ML algorithms have to be protected. In this paper, we present a framework that combines autoencoder, multi-objective optimization algorithms, and different ML algorithms to encode the network data, reduce its size, and detect and classify the network attacks. The novel element used in this framework, with respect to earlier research, is the application of multi-objective optimization algorithms, such as Multi-Objective Differential Evolution or Non-dominated Sorting Genetic Algorithm-II, to handle the different objectives in the fitness function of the autoencoder (autoencoder decoding error and accuracy of ML algorithm). We evaluated six different ML algorithms for attack detection and classification on network dataset UNSWNB15. The performance of the proposed framework is compared with single-objective Differential Evolution. The results showed that Multi-Objective Differential Evolution outperforms the counterparts for attack detection, while all the evaluated algorithms showed similar performance for attack classification.

Differential evolution (DE), as a class of biologically inspired and meta-heuristic techniques, has attained increasing popularity in solving many real world optimization problems. However, DE is not always successful. It can easily get stuck in a local optimum or an undesired stagnation condition. This paper proposes a new DE algorithm Differential Evolution with Alopex-Based Local Search (DEALS), for enhancing DE performance. Alopex uses local correlations between changes in individual parameters and changes in function values to estimate the gradient of the landscape. It also contains the idea of simulated annealing that uses temperature to control the probability of move directions during the search process. The results from experiments demonstrate that the use of Alopex as local search in DE brings substantial performance improvement over the standard DE algorithm. The proposed DEALS algorithm has also been shown to be strongly competitive (best rank) against several other DE variants with local search. 

Differential evolution (DE) presents a class of evolutionary and meta-heuristic techniques that have been applied successfully to solve many real-world problems. However, the performance of DE is significantly influenced by its control parameters such as scaling factor and crossover probability. This paper proposes a new adaptive DE algorithm by greedy adjustment of the control parameters during the running of DE. The basic idea is to perform greedy search for better parameter assignments in successive learning periods in the whole evolutionary process. Within each learning period, the current parameter assignment and its neighboring assignments are tested (used) in a number of times to acquire a reliable assessment of their suitability in the stochastic environment with DE operations. Subsequently the current assignment is updated with the best candidate identified from the neighborhood and the search then moves on to the next learning period. This greedy parameter adjustment method has been incorporated into basic DE, leading to a new DE algorithm termed as Greedy Adaptive Differential Evolution (GADE). GADE has been tested on 25 benchmark functions in comparison with five other DE variants. The results of evaluation demonstrate that GADE is strongly competitive: it obtained the best rank among the counterparts in terms of the summation of relative errors across the benchmark functions with a high dimensionality.

Differential evolution (DE) is a population-based metaheuristic algorithm that has been proved powerful in solving a wide range of real-parameter optimization tasks. However, the selection of the mutation strategy and control parameters in DE is problem dependent, and inappropriate specification of them will lead to poor performance of the algorithm such as slow convergence and early stagnation in a local optimum. This paper proposes a new method termed as Joint Adaptation of Parameters in DE (JAPDE). The key idea lies in dynamically updating the selection probabilities for a complete set of pairs of parameter generating functions based on feedback information acquired during the search by DE. Further, for mutation strategy adaptation, the Rank-Based Adaptation (RAM) method is utilized to facilitate the learning of multiple probability distributions, each of which corresponds to an interval of fitness ranks of individuals in the population. The coupling of RAM with JAPDE results in the new RAM-JAPDE algorithm that enables simultaneous adaptation of the selection probabilities for pairs of control parameters and mutation strategies in DE. The merit of RAM-JAPDE has been evaluated on the benchmark test suit proposed in CEC2014 in comparison to many well-known DE algorithms. The results of experiments demonstrate that the proposed RAM-JAPDE algorithm outperforms or is competitive to the other related DE variants that perform mutation strategy and control parameter adaptation, respectively.

Differential Evolution represents a class of evolutionary algorithms that are highly competitive for solving numerical optimization problems. In a Differential Evolution algorithm, there are a few alternative mutation strategies, which may lead to good or a bad performance depending on the property of the problem. A new mutation strategy, called DE/Alopex/1, is proposed in this paper. This mutation strategy distinguishes itself from other mutation strategies in that it uses the fitness values of the individuals in the population in order to calculate the probabilities of move directions. The performance of DE/Alopex/1 has been evaluated on the benchmark suite from CEC2013. The results of the experiments show that DE/Alopex/1 outperforms some state-of-the-art mutation strategies. © 2017 IEEE.

Differential evolution (DE) presents a class of evolutionary computing techniques that appear effective to handle real parameter optimization tasks in many practical applications. However, the performance of DE is not always perfect to ensure fast convergence to the global optimum. It can easily get stagnation resulting in low precision of acquired results or even failure. This paper proposes a new memetic DE algorithm by incorporating Eager Random Search (ERS) to enhance the performance of a basic DE algorithm. ERS is a local search method that is eager to replace the current solution by a better candidate in the neighborhood. Three concrete local search strategies for ERS are further introduced and discussed, leading to variants of the proposed memetic DE algorithm. In addition, only a small subset of randomly selected variables is used in each step of the local search for randomly deciding the next trial solution. The results of tests on a set of benchmark problems have demonstrated that the hybridization of DE with Eager Random Search can substantially augment DE algorithms to find better or more precise solutions while not requiring extra computing resources.

This paper proposes a memetic computing algorithm by incorporating Eager Random Search (ERS) into differential evolution (DE) to enhance its search ability. ERS is a local search method that is eager to move to a position that is identified as better than the current one without considering other opportunities. Forsaking optimality of moves in ERS is advantageous to increase the randomness and diversity of search for avoiding premature convergence. Three concrete local search strategies within ERS are introduced and discussed, leading to variants of the proposed memetic DE algorithm. The results of evaluations on a set of benchmark problems have demonstrated that the integration of DE with Eager Random Search can improve the performance of pure DE algorithms while not incurring extra computing expenses.

Differential evolution has many mutation strategies which are problem dependent. Some Adaptive Differential Evolution techniques have been proposed tackling this problem. But therein all individuals are treated equally without taking into account how good these solutions are. In this paper, a new method called Ranked-based Mutation Adaptation (RAM) is proposed, which takes into consideration the ranking of an individual in the whole population. This method will assign different probabilities of choosing different mutation strategies to different groups in which the population is divided. RAM has been integrated into several well-known adaptive differential evolution algorithms and its performance has been tested on the benchmark suit proposed in CEC2014. The experimental results shows the use of RAM can produce generally better quality solutions than the original adaptive algorithms.

Differential evolution (DE) is a very attractive evolutionary and meta-heuristic technique to solve many optimization problems in various real-world scenarios. However, the proper setting of control parameters of DE is highly dependent on the problem to solve as well as on the different stages of the search process. This paper proposes a new greedy adaptation method for dynamic adjustment of mutation factor and crossover rate in DE. The proposed method is based on the idea of greedy search to find better parameter assignment in the neighborhood of a current candidate. Our work emphasizes reliable evaluation of candidates via applying a candidate with a number of times in the search process. As our purpose is not merely to increase the success rate (the survival of more trial solutions) but also to accelerate the speed of fitness improvement, we suggest a new metric termed as progress rate to access the quality of candidates in support of the greedy search. This greedy parameter adaptation method has been incorporated into basic DE, leading to a new DE algorithm called Greedy Adaptive Differential Evolution (GADE). GADE was tested on 25 benchmark functions in comparison with five other DE variants. The results of evaluation demonstrate that GADE is strongly competitive: it obtains the best ranking among the counterparts in terms of the summation of relative errors across the benchmark functions.

Differential evolution (DE) is one competitive form of evolutionary algorithms. It heavily relies on mutating solutions using scaled differences of randomly selected individuals from the population to create new solutions. The choice of a proper mutation strategy is important for the success of an DE algorithm. This paper presents an empirical investigation to examine and compare the different mutation strategies for global optimization problems. Both solution quality and computational expense of DE variants were evaluated with experiments conducted on a set of benchmark problems. The results of such comparative study would offer valuable insight and information to develop optimal or adaptive mutation strategies for future DE researches and applications. 

Differential evolution (DE) is one competitive form of evolutionary algorithms. It heavily relies on mutating solutions using scaled differences of randomly selected individuals from the population to create new solutions. The choice of a proper mutation strategy is important for the success of an DE algorithm. This paper presents an empirical investigation to examine and compare the different mutation strategies for global optimization problems. Both solution quality and computational expense of DE variants were evaluated with experiments conducted on a set of benchmark problems. The results of such comparative study would offer valuable insight and information to develop optimal or adaptive mutation strategies for future DE researches and applications.

Differential Evolution is a stochastic and metaheuristic technique that has been proved powerful for solving real valued optimization problems in high dimensional spaces. However, Differential Evolution does not guarantee to con verge to the global optimum and it is easily to become trapped in a local optimum. In this paper, we aim to enhance Differential Evolution with Random Local Search to increase its ability to avoid local optimum. The proposed new algorithm is called Differential Evolution with Random Local Search (DERLS). The advantage of Random Local Search used in DERLS is that it is simple and fast in computation. The results of experiments have demonstrated that our DERLS algorithm can bring appreciable improvement for the acquired solutions in difficult optimization problems.

Differential evolution (DE) represents a class of population-based optimization techniques that uses differences of vectors to search for optimal solutions in the search space. However, promising solutions/ regions are not adequately exploited by a traditional DE algorithm. Memetic computing has been popular in recent years to enhance the exploitation of global algorithms via incorporation of local search. This paper proposes a new memetic framework to enhance DE algorithms using Alopex Local Search (MFDEALS). The novelty of the proposed MFDEALS framework lies in that the behavior of exploitation (by Alopex local search) can be controlled based on the DE global exploration status (population diversity and search stage). Additionally, an adaptive parameter inside the Alopex local search enables smooth transition of its behavior from exploratory to exploitative during the search process. A study of the important components of MFDEALS shows that there is a synergy between them. MFDEALS has been integrated with both the canonical DE method and the adaptive DE algorithm L-SHADE, leading to the MDEALS and ML-SHADEALS algorithms, respectively. Both algorithms were tested on the benchmark functions from the IEEE CEC'2014 Conference. The experiment results show that Memetic Differential Evolution with Alopex Local Search (MDEALS) not only improves the original DE algorithm but also outperforms other memetic DE algorithms by obtaining better quality solutions. Further, the comparison between ML-SHADEALS and L-SHADE demonstrates that applying the MFDEALS framework with Alopex local search can significantly enhance the performance of L-SHADE. 

Harmonic filtering has been widely applied to reduce harmonic distortion in power distribution systems. This paper investigates a new method of exploiting Differential Evolution (DE) to support the optimal design of harmonic filters. DE is a class of stochastic and population-based optimization algorithms that are expected to have stronger global ability than trajectory-based optimization techniques in locating the best component sizes for filters. However, the performance of DE is largely affected by its two control parameters: scaling factor and crossover rate, which are problem dependent. How to decide appropriate setting for these two parameters presents a practical difficulty in real applications. Greedy Adaptive Differential Evolution (GADE) algorithm is suggested in the paper as a more convenient and effective means to automatically optimize filter designs. GADE is attractive in that it does not require proper setting of the scaling factor and crossover rate prior to the running of the program. Instead it enables dynamic adjustment of the DE parameters during the course of search for performance improvement. The results of tests on several problem examples have demonstrated that the use of GADE leads to the discovery of better filter circuits facilitating less harmonic distortion than the basic DE method.

Model calibration represents the task of estimating the parameters of a process model to obtain a good match between observed and simulated behaviours. This can be considered as an optimization problem to search for model parameters that minimize the discrepancy between the model outputs and the corresponding features from the historical empirical data. This chapter investigates the use of Differential Evolution (DE), a competitive class of evolutionary algorithms, to solve calibration problems for nonlinear process models. The merits of DE include simple and compact structure, easy implementation, as well as high convergence speed. However, the good performance of DE relies on proper setting of its running parameters such as scaling factor and crossover probability, which are problem dependent and which can even vary in the different stages of the search. To mitigate this issue, we propose a new adaptive DE algorithm that dynamically adjusts its running parameters during its execution. The core of this new algorithm is the incorporated greedy local search, which is conducted in successive learning periods to continuously locate better parameter assignments in the optimization process. In case studies, we have applied our proposed adaptive DE algorithm for model calibration in a Furnace Optimized Control System. The statistical analysis of experimental results demonstrate that the proposed DE algorithm can support the creation of process models that are more accurate than those produced by standard DE.

The rapid growth of the Internet has led to the evolution of sophisticated security threats that exploit vulnerabilities within networks. The defence mechanisms must quickly adapt to these new threats to ensure that networks stay secure. One possible mechanism is to use Machine Learning (ML) algorithms to detect malicious activities. The edge devices that control and manage the network, such as routers, already have access to the data that is flowing through the network and may utilize its own computational resources to host ML algorithms and use them to detect intrusions. This paper presents a system for network intrusion detection which is deployed to an edge device and evaluated for live binary classification of network traffic. Different ML algorithms (Decision Tree, Random Forest, and Artificial Neural Network) are evaluated on existing datasets (Westermo and CIC-IDS-2017). Flow-based data pre-processing is performed and different labeling strategies and flow durations are used and compared. The most effective version of each algorithm is implemented and deployed on the Westermo Lynx- 3510 routing-capable network switch and system performance is assessed across various scenarios with simulated network attacks. The experiments showed that Random Forest is the best option, closely followed by Decision Tree.

Modern manufacturing systems collect a huge amount of data which gives an opportunity to apply various Machine Learning (ML) techniques. The focus of this paper is on the detection of anomalous behavior in industrial manufacturing systems by considering the temporal nature of the manufacturing process. Long Short-Term Memory (LSTM) networks are applied on a publicly available dataset called Modular Ice-cream factory Dataset on Anomalies in Sensors (MIDAS), which is created using a simulation of a modular manufacturing system for ice cream production. Two different problems are addressed: anomaly detection and anomaly classification. LSTM performance is analysed in terms of accuracy, execution time, and memory consumption and compared with non-time-series ML algorithms including Logistic Regression, Decision Tree, Random Forest, and Multi-Layer Perceptron. The experiments demonstrate the importance of considering the temporal nature of the manufacturing process in detecting anomalous behavior and the superiority in accuracy of LSTM over non-time-series ML algorithms. Additionally, runtime adaptation of the predictions produced by LSTM is proposed to enhance its applicability in a real system.

Vulnerability of important data is increasing everyday with the constant evolution and increase of sophisticated cyber security threats that can seriously affect the business processes. Hence, it is important for organizations to define and implement appropriate mechanisms such as intrusion detection systems to protect their valuable data. In recent years, various machine learning approaches were proposed for intrusion detection, where Random Forest (RF) is recognized as one of the most suitable algorithms. Machine learning algorithms are data-oriented and storing data for training on the centralized server can increase the vulnerability of the whole system. In this paper, we are using a federated learning approach that independently trains data subsets on multiple clients and sends only the resulting models for aggregation to a server. This considerably reduces the need for sending all data to a centralised server. Different RF-based federated learning versions were evaluated on four intrusion detection benchmark datasets (KDD, NSL-KDD, UNSW-NB15, and CIC-IDS-2017). In our experiments, the global RF on the server achieved higher accuracy than the maximum achieved with individual RFs on the clients in the case of two out of four datasets, and it was very close to the maximum for the third dataset. Even in the fourth case, the global RF performed better than the average accuracy, although it fell behind the maximum.

Communication networks are crucial components of the underlying digital infrastructure in any smart city setup. The increasing usage of computer networks brings additional cyber security concerns, and every organization has to implement preventive measures to protect valuable data and business processes. Due to the inherent distributed nature of the city infrastructures as well as the critical nature of its resources and data, any solution to the attack detection calls for distributed, efficient and privacy preserving solutions. In this paper, we extend the evaluation of our federated learning framework for network attacks detection and classification based on random forest. Previously the framework was evaluated only for attack detection using four well-known intrusion detection datasets (KDD, NSL-KDD, UNSW-NB15, and CIC-IDS-2017). In this paper, we extend the evaluation for attack classification. We also evaluate how adding differential privacy into random forest, as an additional protective mechanism, affects the framework performances. The results show that the framework outperforms the average performance of independent random forests on clients for both attack detection and classification. Adding differential privacy penalizes the performance of random forest, as expected, but the use of the proposed framework still brings benefits in comparison to the use of independent local models. The code used in this paper is publicly available, to enable transparency and facilitate reproducibility within the research community.

A small deviation in manufacturing systems can cause huge economic losses, and all components and sensors in the system must be continuously monitored to provide an immediate response. The usual industrial practice is rather simplistic based on brute force checking of limited set of parameters often with pessimistic pre-defined bounds. The usage of appropriate machine learning techniques can be very valuable in this context to narrow down the set of parameters to monitor, define more refined bounds, and forecast impending issues. One of the factors hampering progress in this field is the lack of datasets that can realistically mimic the behaviours of manufacturing systems. In this paper, we propose a new dataset called MIDAS (Modular Ice cream factory Dataset on Anomalies in Sensors) to support machine learning research in analog sensor data. MIDAS is created using a modular manufacturing simulation environment that simulates the ice cream-making process. Using MIDAS, we evaluated four different supervised machine learning algorithms (Logistic Regression, Decision Tree, Random Forest, and Multilayer Perceptron) for two different problems: anomaly detection and anomaly classification. The results showed that multilayer perceptron is the most suitable algorithm with respect to model accuracy and execution time. We have made the data set and the code for the experiments publicly available, to enable interested researchers to enhance the state of the art by conducting further studies.

The recent advances in digitalization, improved connectivity and cloud based services are making a huge revolution in manufacturing domain. In spite of the huge potential benefits in productivity, these trends also bring in some concerns related to safety and security to the traditionally closed industrial operation scenarios. This paper presents a high-level view of some of the research results and technological contributions of the InSecTT Project for meeting safety/security goals. These technology contributions are expected to support both the design and operational phases in the production life cycle. Specifically, our contributions spans (a) enforcing stricter but flexible access control, (b) evaluation of machine learning techniques for intrusion detection, (c) generation of realistic process control and network oriented datasets with injected anomalies and (d) performing safety and security analysis on automated guided vehicle platoons.

JADE is an state-of-the-art adaptive differential evolution algorithm which implements "DE/current-to-pbest" as its mutation strategy, adapts its mutation factor and crossover rate, and uses an optional external archive to keep track of potential removed individuals in previous generations. This paper proposes MPADE, which extends JADE by using a multi-populated approach to solve high dimensional real-parameter optimization problems. This mechanism helps preventing the two well-known problems affecting the differential evolution algorithm performance, which are premature convergence and stagnation. The algorithm was tested using the benchmark functions in IEEE Congress on Evolutionary Computation 2014 test suite. MPADE was compared using Wilcoxon test to JADE algorithm and with other state-of-the-art algorithms that either use a multi-population approach or adapt their parameters. The experimental results show that the proposed new algorithm improves significantly its precursor and it is also suggested that other state-of-the-art algorithms could benefit from the multi-populated based approach.

There is a growing body of knowledge on network intrusion detection, and several open data sets with network traffic and cyber-security threats have been released in the past decades. However, many data sets have aged, were not collected in a contemporary industrial communication system, or do not easily support research focusing on distributed anomaly detection. This paper presents the Westermo network traffic data set, 1.8 million network packets recorded in over 90 minutes in a network built up of twelve hardware devices. In addition to the raw data in PCAP format, the data set also contains pre-processed data in the form of network flows in CSV files. This data set can support the research community for topics such as intrusion detection, anomaly detection, misconfiguration detection, distributed or federated artificial intelligence, and attack classification. In particular, we aim to use the data set to continue work on resource-constrained distributed artificial intelligence in edge devices. The data set contains six types of events: harmless SSH, bad SSH, misconfigured IP address, duplicated IP address, port scan, and man in the middle attack. 

Convolutional neural networks (CNNs) have become increasingly important to deal with many image processing and pattern recognition problems. In order to use CNNs in music genre recognition, spectrograms (visual representation of the spectrum of frequencies of a signal as it varies with time) are usually employed as inputs of the network. Yet some other approaches used music features for genre classification as well. In this paper we propose a new deep network model combining CNN with a simple multi-layer neural network for music genre classification. Since other features are taken into account in the multi-layer network, the combined deep neural network has shown better accuracy than each of the single models in the experiments (Code available at: https://github.com/risengnom/Music-Genre-Recognition.).

ElectroEncephaloGram (EEG) spectral power has been extensively used to classify Mental Imagery (MI) of movements involving different body parts. However, there is an increasing need to enable classification of MI of movements within the same limb. In this work, EEG spectral power was recorded in seven subjects while they performed MI of closing (grip) and opening (extension of fingers) the hand. The EEG data was analyzed and the feasibility of classifying MI of the two movements were investigated using two different classification algorithms, a linear regression and a Convolutional Neural Network (CNN). Results show that only the CNN is able to significantly classify MI of opening and closing of the hand with an average classification accuracy of 60.4%. This indicates the presence of higher-order non-linear discriminatory information and demonstrates the potential of using CNN in classifying MI of same-limb movements.

OBJECTIVE: Multivariate decoding enables access to information encoded in multiple brain activity features with high temporal resolution. However, whether the strength, of which this information is represented in the brain, can be extracted across time within single trials remains largely unexplored.

APPROACH: In this study, we addressed this question by applying a Support Vector Machine (SVM) to extract Motor Imagery (MI) representations, from Electroencephalogram (EEG) data, and by performing time-resolved single-trial analyses of the multivariate decoding. EEG was recorded from a group of healthy participants during MI of opening and closing of the same hand.

MAIN RESULTS: Cross-temporal decoding revealed both dynamic and stationary MI-relevant features during the task. Specifically, features representing MI evolved dynamically early in the trial and later stabilized into a stationary network of MI features. Using a Hierarchical Genetic Algorithm (HGA) for selection of MI-relevant features, we identified primarily contralateral alpha and beta frequency features over the sensorimotor and parieto-occipital cortices as stationary which extended into a bilateral pattern in the later part of the trial. During the stationary encoding of MI, by extracting the SVM prediction scores, we analyzed MI-relevant EEG activity patterns with respect to the temporal dynamics within single trials. We show that the SVM prediction score correlates to the amplitude of univariate MI-relevant features (as documented from an extensive repertoire of previous MI studies) within single trials, strongly suggesting that these are functional variations of MI strength hidden in trial averages.

SIGNIFICANCE: Our work demonstrates a powerful approach for estimating MI strength continually within single trials, having far-reaching impact for single-trial analyses. In terms of MI neurofeedback for motor rehabilitation, these results set the ground for more refined neurofeedback reflecting the strength of MI that can be provided to patients continually in time.

Optimization problems with multiple objectives are often encountered in many scientific and engineering scenarios. The prior works on multi-objective differential evolution (DE) have mainly focused on nondominated sorting of solutions to handle different objectives at the same time. This paper suggests a new approach to differential evolution which is based on decomposition of the original problem into a set of scalar optimization subproblems. We design a decomposition-based DE algorithm to optimize these scalar subproblems simultaneously by evolving a population of solutions with proper mutation and selection operators. Since the proposed DE algorithm does not involve pairwise comparison and non-dominated sorting of solutions, it would incur lower computational complexity than the dominance-based DE algorithms.

Metaheuristics has attained increasing interest for solving complex real-world problems. This paper studies the principles and the state-of-the-art of metaheuristic methods for engineering optimization. Both the classic and emerging approaches to optimization using metaheuristics are reviewed and analyzed. All the methods are discussed in three basic types: trajectory-based, in which in each step a new solution is created from the previous one; multi-trajectory-based, in which a multi-start mechanism is used; and population-based, where multiple new solutions are created considering a population of approximate solutions. We further discuss algorithms and strategies to handle multi-modal and multi-objective optimization tasks as well as methods for parallel implementation of metaheuristic algorithms. Then, different software frameworks for metaheuristics are introduced. Finally, several interesting directions are pointed out as future research trends.

This paper gives a survey of the principles and the state-of-the-art of engineering optimization techniques. Both the classic and emerging approaches to nonlinear optimization problems are reviewed and analyzed. All the techniques are discussed in two basic types: point-based transition and population-based transition, depending on whether a single point or multiple points are generated as new approximate solution(s) in each step. We also consider multi-objective tasks as new application trend and point out the strong potential of population-based methods to tackle multiple objectives simultaneously.