Pac Net a Model Pruning Approach
1. Abstract When using an over-parameterized model, the author found the model can be pruned without losing the most accuracy. The author identifies the essential weight using LWM to obtain the mask. For pruned model, train on source domain with regularization. Then, transfer the model to the target domain, freeze the un-pruned parameter on the source domain, and train only the pruned parameter on the target domain. 2. Contribution Very first using pruning in transfer learning....