https://www.mdu.se/

On-device transfer learning suggests fine-tuning pretrained neural networks on new input data directly on edge devices. The memory limitation of edge devices necessitates using memory-efficient fine-tuning methods. Fine-tuning involves two primary phases: the forward-pass phase and the backwardpass phase. The forward-pass phase generates output activations, and the backward-pass phase computes gradients and updates the parameters accordingly. Although the forward-pass phase demands a temporary memory to store a layer’s input and output activations, the backward-pass phase may require storing the output activations from all layers to compute gradients. This fact introduces the memory cost of the backward-pass phase as the main contributor to the huge training memory demands of deep neural networks (DNNs), which has been the focus of many studies. However, little attention has been made to how the temporary activation memory involved in the forward-pass phase may also act as the memory bottleneck, which is the main focus of this paper. This paper aims to mitigate this memory bottleneck by pruning unimportant channels from layers that require significant temporary activation memory. Experimental results demonstrate how the proposed method effectively reduces peak activation memory and total memory costs of MobileNetV2 by 65% and 59%, respectively, at the cost of 3% accuracy drop.