https://www.mdu.se/

mdu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Analysing Robustness of Tiny Deep Neural Networks
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
Mälardalen University, School of Innovation, Design and Engineering, Innovation and Product Realisation.ORCID iD: 0000-0001-6889-5005
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Tallinn University of Technology (Taltech), Tallinn, Estonia.
2023 (English)In: Commun. Comput. Info. Sci., Springer Science and Business Media Deutschland GmbH , 2023, p. 150-159Conference paper, Published paper (Refereed)
Abstract [en]

Real-world applications that are safety-critical and resource-constrained necessitate using compact and robust Deep Neural Networks (DNNs) against adversarial data perturbation. MobileNet-tiny has been introduced as a compact DNN to deploy on edge devices to reduce the size of networks. To make DNNs more robust against adversarial data, adversarial training methods have been proposed. However, recent research has investigated the robustness of large-scale DNNs (such as WideResNet), but the robustness of tiny DNNs has not been analysed. In this paper, we analyse how the width of the blocks in MobileNet-tiny affects the robustness of the network against adversarial data perturbation. Specifically, we evaluate natural accuracy, robust accuracy, and perturbation instability metrics on the MobileNet-tiny with various inverted bottleneck blocks with different configurations. We generate configurations for inverted bottleneck blocks using different width-multipliers and expand-ratio hyper-parameters. We discover that expanding the width of the blocks in MobileNet-tiny can improve the natural and robust accuracy but increases perturbation instability. In addition, after a certain threshold, increasing the width of the network does not have significant gains in robust accuracy and increases perturbation instability. We also analyse the relationship between the width-multipliers and expand-ratio hyper-parameters with the Lipchitz constant, both theoretically and empirically. It shows that wider inverted bottleneck blocks tend to have significant perturbation instability. These architectural insights can be useful in developing adversarially robust tiny DNNs for edge devices.

Place, publisher, year, edition, pages
Springer Science and Business Media Deutschland GmbH , 2023. p. 150-159
Keywords [en]
Adversarial data perturbation, Adversarial training, Lipchitz constant, Robustness analysis, Safety engineering, Stability, Data perturbation, Hyper-parameter, Large-scales, Real-world, Recent researches, Training methods, Deep neural networks
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:mdh:diva-64432DOI: 10.1007/978-3-031-42941-5_14Scopus ID: 2-s2.0-85171970100ISBN: 9783031429408 (print)OAI: oai:DiVA.org:mdh-64432DiVA, id: diva2:1803377
Conference
Communications in Computer and Information Science
Available from: 2023-10-09 Created: 2023-10-09 Last updated: 2023-10-09Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Mousavi, HamidZoljodi, AliDaneshtalab, Masoud

Search in DiVA

By author/editor
Mousavi, HamidZoljodi, AliDaneshtalab, Masoud
By organisation
Embedded SystemsInnovation and Product Realisation
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 10 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf