합성곱 신경망에서 구조적 희소 학습을 위한 three operator splitting 알고리즘

초록

As deep learning-based image processing technologies have gained attention, the use of large-scale convolutional neural networks has also increased. However, as models become more complex, computational cost and memory requirements grow, leading to limitations in applying them to real-time environments. Accordingly, model compression techniques have been actively studied, but there still exist limitations, such as fixed structures or the need for an additional retraining process after training. In this study, an optimization method based on a three operator splitting technique was applied to simultaneously induce structured sparsity at the channel and filter levels during the training process. This method applies separate proximal operators to different regularization terms, enabling natural sparsity induction during training without the need for additional threshold tuning. In addition, experiments were conducted on the cases where the regularizations were applied individually, in order to compare the effects of structure-specific sparsity methods. As a result, the proposed method showed less than 1% accuracy loss while maintaining accuracy similar to the original model, achieving significant computational cost reduction and enabling efficient sparsity without repeated experiments.

키워드