Skip to content
/ DA2Lite Public

DA2Lite is an automated model compression toolkit for PyTorch.

License

Notifications You must be signed in to change notification settings

da2so/DA2Lite

Repository files navigation

DA2Lite (Deep Architecture to Lite) is a toolkit to compress and accelerate deep network models.

MIT licensed Python version support Pytorch version support

⭐ Star us on GitHub — it helps!!

Frameworks & Libraries Algorithms
Built-in
  • Supported Framework
    • PyTorch

Install

git clone https://github.com/da2so/DA2Lite.git

You will need a machine with a GPU and CUDA installed.
Then, you prepare runtime environment:

pip install -r requirements.txt

Use

Run

main.py(DA2Lite) runs with two main configurations like as follows:

CUDA_VISIBLE_DEVICES=0 python main.py --train_config_file=./configs/train/cifar10/cifar10/vgg16.yaml --compress_config_file=./configs/compress/tucker.yaml

The first one is train_config_file, which indicates training configurations and the other is compress_config_file, which represents compress configurations. The details of available configurations are described in Here.

After you run DA2Lite to compress a DNN model, logging and compressed model are saved in ./log directory.

The following shows the format of saving:

  • YYYY-MM-DD.HH.MM.SS : format of saved directory for an instance.
    • models
      • origin_{dataset}_{model}.pt : The original model is saved.
      • compress_1_{dataset}_{model}.pt : The first compressed model is saved.
      • ...
    • process.log : The inevitable log is only logged.
    • specific_process.log : The training procedure log is added with process.log

Example

  • Run the CIFAR10 example with resnet18 using tucker decomposition.
    • The pretrained-model are decomposed and right after fine-tuned: Here

Result

Cifar10 dataset

Model Acc(%) Param num(M) MACs(G) File size(MB) Download
ResNet18 94.74% -> 94.14% (-0.6) 11.17M -> 0.75M (14.81x) 0.56G -> 0.19G (2.96x) 42.70MB -> 2.96MB (14.44x) Here
Vgg16 90.83% -> 88.37% (-2.46) 14.72M -> 0.38M (39.12x) 0.31G -> 0.1G (3.29x) 56.16MB -> 1.45MB (38.71x) Here
Vgg16_bn 93.22% -> 92.74% (-0.48) 14.73M -> 0.71M (20.7x) 0.31G -> 0.11G (2.93x) 56.25MB -> 2.77MB (20.29x) Here

TODO

  • Multi-GPU training
  • PyTorchMobile conversion
  • Train a model based on a custom dataset
  • Rand-augmentation for improving an accuracy
  • Make a model zoo
  • Up-to-date model architectures.
  • Train a model for object detection tasks (further future...)
  • Compression methods for object detection tasks (further future...)

License

The entire codebase is under MIT license