https://www.mdu.se/

mdu.sePublications
Change search
Link to record
Permanent link

Direct link
Sinaei, Sima
Alternative names
Publications (10 of 10) Show all publications
Mohammadi, S., Balador, A., Sinaei, S. & Flammini, F. (2024). Balancing Privacy and Performance in Federated Learning: a Systematic Literature Review on Methods and Metrics. Journal of Parallel and Distributed Computing, 192
Open this publication in new window or tab >>Balancing Privacy and Performance in Federated Learning: a Systematic Literature Review on Methods and Metrics
2024 (English)In: Journal of Parallel and Distributed Computing, ISSN 0743-7315, E-ISSN 1096-0848, Vol. 192Article in journal (Refereed) Submitted
Abstract [en]

Federated Learning (FL) has emerged as a novel paradigm in the area of Artificial Intelligence (AI), emphasizing decentralized data utilization and bringing learning to the edge or directly on-device. While this approach eliminates the need for data centralization, ensuring enhanced privacy and protection of sensitive information, it is not without challenges. Particularly during the training phase and the exchange of model update parameters between servers and clients, new privacy challenges have arisen. While several privacy-preserving FL solutions have been developed to mitigate potential breaches in FL architectures, their integration poses its own set of challenges. Incorporating these privacy-preserving mechanisms into FL at the edge computing level can increase both communication and computational overheads, which may, in turn, compromise data utility and learning performance metrics. This paper provides a systematic literature review on essential methods and metrics to support the most appropriate trade-offs between FL privacy and other performance-related application requirements such as accuracy, loss, convergence time, utility, communication, and computation overhead. We aim to provide an extensive overview of recent privacy-preserving mechanisms in FL used across various applications, placing a particular focus on quantitative privacy assessment approaches in FL and the necessity of achieving a balance between privacy and the other requirements of real-world FL applications. This review collects, classifies, and discusses relevant papers in a structured manner, emphasizing challenges, open issues, and promising research directions. 

Place, publisher, year, edition, pages
Academic Press Inc., 2024
Keywords
Cybersecurity, Distributed artificial intelligence, Federated learning, Performance evaluation, Trustworthiness
National Category
Computer Sciences
Identifiers
urn:nbn:se:mdh:diva-64363 (URN)10.1016/j.jpdc.2024.104918 (DOI)001246744100001 ()2-s2.0-85194089881 (Scopus ID)
Available from: 2023-09-26 Created: 2023-09-26 Last updated: 2024-07-03Bibliographically approved
Mohammadi, S., Mohammadi, M., Sinaei, S., Balador, A., Nowroozi, E., Flammini, F. & Conti, M. (2023). Balancing Privacy and Accuracy in Federated Learning for Speech Emotion Recognition. In: Proceedings of the 18th Conference on Computer Science and Intelligence Systems: . Paper presented at 18th Conference on Computer Science and Intelligence Systems, September 17–20, 2023. Warsaw, Poland (pp. 191-200). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Balancing Privacy and Accuracy in Federated Learning for Speech Emotion Recognition
Show others...
2023 (English)In: Proceedings of the 18th Conference on Computer Science and Intelligence Systems, Institute of Electrical and Electronics Engineers (IEEE), 2023, p. 191-200Conference paper, Published paper (Refereed)
Abstract [en]

Speech Emotion Recognition (SER) is a valuable technology that identifies human emotions from spoken language, enabling the development of context-aware and personalized intelligent systems. To protect user privacy, Federated Learning (FL) has been introduced, enabling local training of models on user devices. However, FL raises concerns about the potential exposure of sensitive information from local model parameters, which is especially critical in applications like SER that involve personal voice data. Local Differential Privacy (LDP) has been successful in preventing privacy leaks in image and video data. However, it encounters notable accuracy degradation when applied to speech data, especially in the presence of high noise levels. In this paper, we propose an approach called LDP-FL with CSS, which combines LDP with a novel client selection strategy (CSS). By leveraging CSS, we aim to improve the representatives of updates and mitigate the adverse effects of noise on SER accuracy while ensuring client privacy through LDP. Furthermore, we conducted model inversion attacks to evaluate the robustness of LDP-FL in preserving privacy. These attacks involved an adversary attempting to reconstruct individuals' voice samples using the output labels provided by the SER model. The evaluation results reveal that LDP-FL with CSS achieved an accuracy of 65-70%, which is 4% lower than the initial SER model accuracy. Furthermore, LDP-FL demonstrated exceptional resilience against model inversion attacks, outperforming the non-LDP method by a factor of 10. Overall, our analysis emphasizes the importance of achieving a balance between privacy and accuracy in accordance with the requirements of the SER application

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
National Category
Computer Sciences
Identifiers
urn:nbn:se:mdh:diva-64364 (URN)10.15439/2023F444 (DOI)2-s2.0-85179177296 (Scopus ID)9788396744784 (ISBN)
Conference
18th Conference on Computer Science and Intelligence Systems, September 17–20, 2023. Warsaw, Poland
Available from: 2023-09-26 Created: 2023-09-26 Last updated: 2023-12-20Bibliographically approved
Mohammadi, S., Sinaei, S., Balador, A. & Flammini, F. (2023). Secure and Efficient Federated Learning by Combining Homomorphic Encryption and Gradient Pruning in Speech Emotion Recognition. In: International Conference on Information Security Practice and Experience: . Paper presented at 18th International Conference on Information Security Practice and Experience (pp. 1-16). Springer Berlin/Heidelberg
Open this publication in new window or tab >>Secure and Efficient Federated Learning by Combining Homomorphic Encryption and Gradient Pruning in Speech Emotion Recognition
2023 (English)In: International Conference on Information Security Practice and Experience, Springer Berlin/Heidelberg, 2023, p. 1-16Conference paper, Published paper (Refereed)
Abstract [en]

Speech Emotion Recognition (SER) detects human emotions expressed in spoken language. SER is highly valuable in diverse fields; however, privacy concerns arise when analyzing speech data, as it reveals sensitive information like biometric identity. To address this, Federated Learning (FL) has been developed, allowing models to be trained locally and just sharing model parameters with servers. However, FL introduces new privacy concerns when transmitting local model parameters between clients and servers, as third parties could exploit these parameters and disclose sensitive information. In this paper, we introduce a novel approach called Secure and Efficient Federated Learning (SEFL) for SER applications. Our proposed method combines Paillier homomorphic encryption (PHE) with a novel gradient pruning technique. This approach enhances privacy and maintains confidentiality in FL setups for SER applications while minimizing communication and computation overhead and ensuring model accuracy. As far as we know, this is the first paper that implements PHE in FL setup for SER applications. Using a public SER dataset, we evaluated the SEFL method. Results show substantial efficiency gains with a key size of 1024, reducing computation time by up to 25% and communication traffic by up to 70%. Importantly, these improvements have minimal impact on accuracy, effectively meeting the requirements of SER applications. 

Place, publisher, year, edition, pages
Springer Berlin/Heidelberg, 2023
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349
National Category
Computer Sciences
Identifiers
urn:nbn:se:mdh:diva-64365 (URN)10.1007/978-981-99-7032-2_1 (DOI)001166763200001 ()9789819970315 (ISBN)
Conference
18th International Conference on Information Security Practice and Experience
Available from: 2023-09-26 Created: 2023-09-26 Last updated: 2024-12-20Bibliographically approved
Mirsalari, S. A., Nazari, N., Sinaei, S., Salehi, M. E. & Daneshtalab, M. (2022). FaCT-LSTM: Fast and Compact Ternary Architecture for LSTM Recurrent Neural Networks. IEEE design & test, 39(3), 45-53
Open this publication in new window or tab >>FaCT-LSTM: Fast and Compact Ternary Architecture for LSTM Recurrent Neural Networks
Show others...
2022 (English)In: IEEE design & test, ISSN 2168-2356, E-ISSN 2168-2364, Vol. 39, no 3, p. 45-53Article in journal (Refereed) Published
Abstract [en]

Long Short-Term Memory (LSTM) achieved great success in healthcare applications. However, its extensive computation cost and massive model size have become the major obstacles for the deployment of such a powerful algorithm in resource-limited embedded systems such as wearable devices. Quantization is a promising way to reduce the memory footprint and computational cost of neural networks. Although quantization achieved remarkable success in convolutional neural networks (CNNs), it still suffers from large accuracy loss in LSTM networks, especially in extremely low bitwidths. In this paper, we propose Fast and Compact Ternary LSTM (FaCT-LSTM), which bridges the accuracy gap between the full precision and quantized neural networks. We propose a hardware-friendly circuit to implement ternarized LSTM and eliminate computation-intensive floating-point operations. With the proposed ternarized LSTM architectures, our experiments on the ECG and EMG signals show ~0.88 to 2.04% accuracy loss in comparison to the full-precision counterparts while reducing latency and area for ~111× to 116× and ~29× to 33×, respectively. The proposed architectures also improves the memory footprint and bandwidth of the full precision signal classification, by 17×, and 31×, respectively.

Keywords
Encoding;Computer architecture;Quantization (signal);Logic gates;Computational modeling;Multiplexing;Long short term memory;Long Short -Term Memory (LSTM);ECG;EMG;Wearable Devices;Quantization
National Category
Computer Sciences
Identifiers
urn:nbn:se:mdh:diva-56397 (URN)10.1109/MDAT.2021.3070245 (DOI)000803107400010 ()2-s2.0-85103775592 (Scopus ID)
Available from: 2021-11-09 Created: 2021-11-09 Last updated: 2022-11-25Bibliographically approved
Mirsalari, S. A., Nazari, N., Ansarmohammadi, S. A., Sinaei, S., Salehi, M. E. & Daneshtalab, M. (2021). ELC-ECG: Efficient LSTM cell for ECG classification based on quantized architecture. In: Proceedings - IEEE International Symposium on Circuits and Systems: . Paper presented at 53rd IEEE International Symposium on Circuits and Systems, ISCAS 2021, 22 May 2021 through 28 May 2021. Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>ELC-ECG: Efficient LSTM cell for ECG classification based on quantized architecture
Show others...
2021 (English)In: Proceedings - IEEE International Symposium on Circuits and Systems, Institute of Electrical and Electronics Engineers Inc. , 2021Conference paper, Published paper (Refereed)
Abstract [en]

Long Short-Term Memory (LSTM) is one of the most popular and effective Recurrent Neural Network (RNN) models used for sequence learning in applications such as ECG signal classification. Complex LSTMs could hardly be deployed on resource-limited bio-medical wearable devices due to the huge amount of computations and memory requirements. Binary LSTMs are introduced to cope with this problem. However, naive binarization leads to significant accuracy loss in ECG classification. In this paper, we propose an efficient LSTM cell along with a novel hardware architecture for ECG classification. By deploying 5-level binarized inputs and just 1-level binarization for weights, output, and in-memory cell activations, the delay of one LSTM cell operation is reduced 50x with about 0.004% accuracy loss in comparison with full precision design of ECG classification.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2021
Keywords
Electrocardiogram (ECG) Signal Classification, Long Short -Term Memory (LSTM), Wearable Devices, Cells, Cytology, Electrocardiography, Memory architecture, Network architecture, Cell operation, Ecg classifications, Memory requirements, Novel hardware, Precision design, Recurrent neural network (RNN), Sequence learning, Long short-term memory
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:mdh:diva-55485 (URN)10.1109/ISCAS51556.2021.9401261 (DOI)000696765400207 ()2-s2.0-85108992062 (Scopus ID)9781728192017 (ISBN)
Conference
53rd IEEE International Symposium on Circuits and Systems, ISCAS 2021, 22 May 2021 through 28 May 2021
Available from: 2021-07-15 Created: 2021-07-15 Last updated: 2022-11-25Bibliographically approved
Loni, M., Sinaei, S., Zoljodi, A., Daneshtalab, M. & Sjödin, M. (2020). DeepMaker: A multi-objective optimization framework for deep neural networks in embedded systems. Microprocessors and microsystems, 73, Article ID 102989.
Open this publication in new window or tab >>DeepMaker: A multi-objective optimization framework for deep neural networks in embedded systems
Show others...
2020 (English)In: Microprocessors and microsystems, ISSN 0141-9331, E-ISSN 1872-9436, Vol. 73, article id 102989Article in journal (Refereed) Published
Abstract [en]

Deep Neural Networks (DNNs) are compute-intensive learning models with growing applicability in a wide range of domains. Due to their computational complexity, DNNs benefit from implementations that utilize custom hardware accelerators to meet performance and response time as well as classification accuracy constraints. In this paper, we propose DeepMaker framework that aims to automatically design a set of highly robust DNN architectures for embedded devices as the closest processing unit to the sensors. DeepMaker explores and prunes the design space to find improved neural architectures. Our proposed framework takes advantage of a multi-objective evolutionary approach that exploits a pruned design space inspired by a dense architecture. DeepMaker considers the accuracy along with the network size factor as two objectives to build a highly optimized network fitting with limited computational resource budgets while delivers an acceptable accuracy level. In comparison with the best result on the CIFAR-10 dataset, a generated network by DeepMaker presents up to a 26.4x compression rate while loses only 4% accuracy. Besides, DeepMaker maps the generated CNN on the programmable commodity devices, including ARM Processor, High-Performance CPU, GPU, and FPGA. 

Place, publisher, year, edition, pages
Elsevier B.V., 2020
National Category
Computer Systems
Identifiers
urn:nbn:se:mdh:diva-46792 (URN)10.1016/j.micpro.2020.102989 (DOI)000520940000032 ()2-s2.0-85077516447 (Scopus ID)
Available from: 2020-01-23 Created: 2020-01-23 Last updated: 2022-11-25Bibliographically approved
Sinaei, S. & Daneshtalab, M. (2020). Hardware acceleration for recurrent neural networks. In: Hardware Architectures for Deep Learning: (pp. 27-52). Institution of Engineering and Technology
Open this publication in new window or tab >>Hardware acceleration for recurrent neural networks
2020 (English)In: Hardware Architectures for Deep Learning, Institution of Engineering and Technology , 2020, p. 27-52Chapter in book (Other academic)
Abstract [en]

This chapter focuses on the LSTM model and is concerned with the design of a high-performance and energy-efficient solution to implement deep learning inference. The chapter is organized as follows: Section 2.1 introduces Recurrent Neural Networks (RNNs). In this section Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU) network models are discussed as special kind of RNNs. Section 2.2 discusses inference acceleration with hardware. In Section 2.3, a survey on various FPGA designs is presented within the context of the results of previous related works and after which Section 2.4 concludes the chapter.

Place, publisher, year, edition, pages
Institution of Engineering and Technology, 2020
Keywords
Deep learning inference, Energy-efficient solution, Field programmable gate arrays, FPGA designs, Gated recurrent unit network models, GRU network models, Hardware acceleration, High-performance solution, Long short term memory, LSTM model, Recurrent neural nets, Recurrent neural networks, RNN
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:mdh:diva-62517 (URN)10.1049/PBCS055E_ch2 (DOI)2-s2.0-85153645311 (Scopus ID)9781785617683 (ISBN)
Available from: 2023-05-31 Created: 2023-05-31 Last updated: 2023-05-31Bibliographically approved
Mirsalari, S. A., Sinaei, S., Salehi, M. E. & Daneshtalab, M. (2020). MuBiNN: Multi-level binarized recurrent neural network for EEG signal classification. In: Proceedings - IEEE International Symposium on Circuits and Systems: . Paper presented at 52nd IEEE International Symposium on Circuits and Systems, ISCAS 2020, 10 October 2020 through 21 October 2020. Institute of Electrical and Electronics Engineers Inc., Article ID 102250.
Open this publication in new window or tab >>MuBiNN: Multi-level binarized recurrent neural network for EEG signal classification
2020 (English)In: Proceedings - IEEE International Symposium on Circuits and Systems, Institute of Electrical and Electronics Engineers Inc. , 2020, article id 102250Conference paper, Published paper (Refereed)
Abstract [en]

Recurrent Neural Networks (RNN) are widely used for learning sequences in applications such as EEG classification. Complex RNNs could be hardly deployed on wearable devices due to their computation and memory-intensive processing patterns. Generally, reduction in precision leads much more efficiency and binarized RNNs are introduced as energy-efficient solutions. However, naive binarization methods lead to significant accuracy loss in EEG classification. In this paper, we propose a multi-level binarized LSTM, which significantly reduces computations whereas ensuring an accuracy pretty close to the full precision LSTM. Our method reduces the delay of the 3-bit LSTM cell operation 47× with less than 0.01% accuracy loss.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2020
Keywords
EEG signal classification, Long short - Term memory (LSTM), Recurrent Neural Networks (RNNs), Wearable devices, Energy efficiency, Long short-term memory, Accuracy loss, Cell operation, EEG classification, Energy efficient, Learning sequences, Recurrent neural network (RNN), Biomedical signal processing
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:mdh:diva-55486 (URN)000696570700247 ()2-s2.0-85108991421 (Scopus ID)9781728133201 (ISBN)
Conference
52nd IEEE International Symposium on Circuits and Systems, ISCAS 2020, 10 October 2020 through 21 October 2020
Available from: 2021-07-15 Created: 2021-07-15 Last updated: 2022-11-25Bibliographically approved
Mirsalari, S. A., Sinaei, S., Salehi, M. E. & Daneshtalab, M. (2020). MuBiNN: Multi-Level Binarized Recurrent Neural Network for EEG Signal Classification. In: 2020 IEEE International Symposium on Circuits and Systems (ISCAS): . Paper presented at 2020 IEEE International Symposium on Circuits and Systems (ISCAS, Seville, Spain, 12-14 October 2020.
Open this publication in new window or tab >>MuBiNN: Multi-Level Binarized Recurrent Neural Network for EEG Signal Classification
2020 (English)In: 2020 IEEE International Symposium on Circuits and Systems (ISCAS), 2020Conference paper, Published paper (Refereed)
Abstract [en]

Recurrent Neural Networks (RNN) are widely used for learning sequences in applications such as EEG classification. Complex RNNs could be hardly deployed on wearable devices due to their computation and memory-intensive processing patterns. Generally, reduction in precision leads much more efficiency and binarized RNNs are introduced as energy-efficient solutions. However, naive binarization methods lead to significant accuracy loss in EEG classification. In this paper, we propose a multi-level binarized LSTM, which significantly reduces computations whereas ensuring an accuracy pretty close to the full precision LSTM. Our method reduces the delay of the 3-bit LSTM cell operation 47× with less than 0.01% accuracy loss.

National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:mdh:diva-61138 (URN)10.1109/iscas45731.2020.9180634 (DOI)978-1-7281-3320-1 (ISBN)
Conference
2020 IEEE International Symposium on Circuits and Systems (ISCAS, Seville, Spain, 12-14 October 2020
Available from: 2022-12-07 Created: 2022-12-07 Last updated: 2022-12-07Bibliographically approved
Loni, M., Zoljodi, A., Sinaei, S., Daneshtalab, M. & Nolin, M. (2019). NeuroPower: Designing Energy Efficient Convolutional Neural Network Architecture for Embedded Systems. In: Lecture Notes in Computer Science, Volume 11727: . Paper presented at The 28th International Conference on Artificial Neural Networks ICANN 2019, 17 Sep 2019, Munich, Germany (pp. 208-222). Munich, Germany: Springer
Open this publication in new window or tab >>NeuroPower: Designing Energy Efficient Convolutional Neural Network Architecture for Embedded Systems
Show others...
2019 (English)In: Lecture Notes in Computer Science, Volume 11727, Munich, Germany: Springer , 2019, p. 208-222Conference paper, Published paper (Refereed)
Abstract [en]

Convolutional Neural Networks (CNNs) suffer from energy-hungry implementation due to their computation and memory intensive processing patterns. This problem is even more significant by the proliferation of CNNs on embedded platforms. To overcome this problem, we offer NeuroPower as an automatic framework that designs a highly optimized and energy efficient set of CNN architectures for embedded systems. NeuroPower explores and prunes the design space to find improved set of neural architectures. Toward this aim, a multi-objective optimization strategy is integrated to solve Neural Architecture Search (NAS) problem by near-optimal tuning network hyperparameters. The main objectives of the optimization algorithm are network accuracy and number of parameters in the network. The evaluation results show the effectiveness of NeuroPower on energy consumption, compacting rate and inference time compared to other cutting-edge approaches. In comparison with the best results on CIFAR-10/CIFAR-100 datasets, a generated network by NeuroPower presents up to 2.1x/1.56x compression rate, 1.59x/3.46x speedup and 1.52x/1.82x power saving while loses 2.4%/-0.6% accuracy, respectively.

Place, publisher, year, edition, pages
Munich, Germany: Springer, 2019
Series
Lecture Notes in Computer Science, ISSN 0302-9743 ; 11727
Keywords
Convolutional neural networks (CNNs), Neural Architecture Search (NAS), Embedded Systems, Multi-Objective Optimization
National Category
Engineering and Technology Computer Systems
Identifiers
urn:nbn:se:mdh:diva-45043 (URN)10.1007/978-3-030-30487-4_17 (DOI)000546494000017 ()2-s2.0-85072863572 (Scopus ID)9783030304867 (ISBN)
Conference
The 28th International Conference on Artificial Neural Networks ICANN 2019, 17 Sep 2019, Munich, Germany
Projects
DPAC - Dependable Platforms for Autonomous systems and ControlDeepMaker: Deep Learning Accelerator on Commercial Programmable Devices
Available from: 2019-08-23 Created: 2019-08-23 Last updated: 2022-11-25Bibliographically approved
Organisations

Search in DiVA

Show all publications