Early Exiting with Compressible Activations for Efficient Neural Network Inference

Simple item page
dc.contributor.authorBeysens, Jona
dc.contributor.authorSénéclauze, Martin
dc.contributor.authorDallemagne, Philippe
dc.contributor.authorBigdeli, Siavash
dc.date.accessioned2023-03-31T07:41:36Z
dc.date.available2023-03-31T07:41:36Z
dc.date.issued2023
dc.description.abstractCloud-based Machine Learning (ML) incurs high latency and high sensitivity to connectivity failures. This can be improved by maximizing the ML processing on the local device. However, since these low-power devices typically have limited resources, the early-exit mechanism has been used to hierarchically split a neural network in parts over multiple devices, trading off computation with communication costs. But, the intermediate activations can be significantly large at the split point. In this paper, we present a novel entropy-based technique that learns to intelligently compress activations during training and efficiently encodes them during inference. We show that in an early-exit configuration, entropy regularization with Huffman coding can save up to 54% in communication cost, while keeping the classification accuracy of MobileNetV2 on CIFAR-10 above 85%.en_US
dc.description.sponsorshipInternal research funding by DATAPROG (RES3)en_US
dc.identifier.citationIEEE, Smart Systems Integration, Bruges, Belgiumen_US
dc.identifier.urihttps://hdl.handle.net/20.500.12839/1173
dc.language.isoenen_US
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/4.0/*
dc.subjectHierarchical machine learningen_US
dc.subjectActivation compressionen_US
dc.subjectEntropy regularizationen_US
dc.subjectEarly exitingen_US
dc.titleEarly Exiting with Compressible Activations for Efficient Neural Network Inferenceen_US
dc.typeConferenceen_US
dc.type.csemdivisionsDiv-Men_US
dc.type.csemresearchareasData & AIen_US
dc.type.csemresearchareasIoT & Visionen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
20230213_SSI2023_manuscript_submission_preprint_footer.pdf
Size:
480.02 KB
Format:
Adobe Portable Document Format
Description:
Main article
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
2.82 KB
Format:
Item-specific license agreed upon to submission
Description:
Download
Collections
Research Publications