https://www.mdu.se/

mdu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
DASS: Differentiable Architecture Search for Sparse Neural Networks
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.
Zenseact Ab, Göteborg, Sweden.
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems. Computer Systems, Tallinn University of Technology, Tallinn, Estonia.
2023 (English)In: ACM Transactions on Embedded Computing Systems, ISSN 1539-9087, E-ISSN 1558-3465, Vol. 22, no 5 s, article id 105Article in journal (Refereed) Published
Abstract [en]

The deployment of Deep Neural Networks (DNNs) on edge devices is hindered by the substantial gap between performance requirements and available computational power. While recent research has made significant strides in developing pruning methods to build a sparse network for reducing the computing overhead of DNNs, there remains considerable accuracy loss, especially at high pruning ratios. We find that the architectures designed for dense networks by differentiable architecture search methods are ineffective when pruning mechanisms are applied to them. The main reason is that the current methods do not support sparse architectures in their search space and use a search objective that is made for dense networks and does not focus on sparsity.This paper proposes a new method to search for sparsity-friendly neural architectures. It is done by adding two new sparse operations to the search space and modifying the search objective. We propose two novel parametric SparseConv and SparseLinear operations in order to expand the search space to include sparse operations. In particular, these operations make a flexible search space due to using sparse parametric versions of linear and convolution operations. The proposed search objective lets us train the architecture based on the sparsity of the search space operations. Quantitative analyses demonstrate that architectures found through DASS outperform those used in the state-of-the-art sparse networks on the CIFAR-10 and ImageNet datasets. In terms of performance and hardware effectiveness, DASS increases the accuracy of the sparse version of MobileNet-v2 from 73.44% to 81.35% (+7.91% improvement) with a 3.87× faster inference time.

Place, publisher, year, edition, pages
Association for Computing Machinery , 2023. Vol. 22, no 5 s, article id 105
Keywords [en]
image classification, network sparsification, Neural architecture search, optimization, Deep neural networks, Network architecture, Dense network, Images classification, Neural architectures, Optimisations, Search spaces, Sparse network, Sparse neural networks, Sparsification
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
URN: urn:nbn:se:mdh:diva-64424DOI: 10.1145/3609385ISI: 001074334300008Scopus ID: 2-s2.0-85171744110OAI: oai:DiVA.org:mdh-64424DiVA, id: diva2:1803414
Available from: 2023-10-09 Created: 2023-10-09 Last updated: 2023-10-25Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Mousavi, HamidLoni, MohammadDaneshtalab, Masoud

Search in DiVA

By author/editor
Mousavi, HamidLoni, MohammadDaneshtalab, Masoud
By organisation
Embedded Systems
In the same journal
ACM Transactions on Embedded Computing Systems
Electrical Engineering, Electronic Engineering, Information Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 23 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf