FastDeploy, based on PaddleSlim's Auto Compression Toolkit(ACT), provides developers with a one-click model auto compression tool that supports post-training quantization and knowledge distillation training. We take the Yolov5 series as an example to demonstrate how to install and execute FastDeploy's one-click model auto compression.
1.Install PaddlePaddle 2.4 version
https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html
2.Install PaddleSlim 2.4 version
pip install paddleslim==2.4.0FastDeploy One-Click Model Automation compression does not require a separate installation, users only need to properly install the FastDeploy Toolkit
Fastdeploy Auto Compression can include multiple strategies, At present, offline quantization and quantization distillation are mainly used for training. The following will introduce how to use it from two strategies, offline quantization and quantitative distillation.
Developers need to prepare the model to be quantized and the Calibration dataset on their own. In this demo, developers can execute the following command to download the yolov5s.onnx model to be quantized and calibration data set.
# Download yolov5.onnx
wget https://paddle-slim-models.bj.bcebos.com/act/yolov5s.onnx
# Download dataset. This Calibration dataset is the first 320 images from COCO val2017
wget https://bj.bcebos.com/paddlehub/fastdeploy/COCO_val_320.tar.gz
tar -xvf COCO_val_320.tar.gzThe following command is to quantize the yolov5s model, if developers want to quantize other models, replace the config_path with other model configuration files in the configs folder.
fastdeploy compress --config_path=./configs/detection/yolov5s_quant.yaml --method='PTQ' --save_dir='./yolov5s_ptq_model/'[notice] PTQ is short for post-training quantization
To complete the quantization, developers only need to provide a customized model config file, specify the quantization method, and the path to save the quantized model.
| Parameter | Description |
|---|---|
| --config_path | Quantization profiles needed for one-click quantization.Configs |
| --method | Quantization method selection, PTQ for post-training quantization, QAT for quantization distillation training |
| --save_dir | Output of quantized model paths, which can be deployed directly in FastDeploy |
FastDeploy currently supports quantized distillation training only for images without annotation. It does not support evaluating model accuracy during training. The datasets are images from inference application, and the number of images is determined by the size of the dataset, covering all deployment scenarios as much as possible. In this demo, we prepare the first 320 images from the COCO2017 validation set for users. Note: If users want to obtain a more accurate quantized model through quantized distillation training, feel free to prepare more data and train more rounds.
# Download yolov5.onnx
wget https://paddle-slim-models.bj.bcebos.com/act/yolov5s.onnx
# Download dataset. This Calibration dataset is the first 320 images from COCO val2017
wget https://bj.bcebos.com/paddlehub/fastdeploy/COCO_val_320.tar.gz
tar -xvf COCO_val_320.tar.gzThe following command is to quantize the yolov5s model, if developers want to quantize other models, replace the config_path with other model configuration files in the configs folder.
# Please specify the single card GPU before training, otherwise it may get stuck during the training process.
export CUDA_VISIBLE_DEVICES=0
fastdeploy compress --config_path=./configs/detection/yolov5s_quant.yaml --method='QAT' --save_dir='./yolov5s_qat_model/'To complete the quantization, developers only need to provide a customized model config file, specify the quantization method, and the path to save the quantized model.
| Parameter | Description |
|---|---|
| --config_path | Quantization profiles needed for one-click quantization.Configs |
| --method | Quantization method selection, PTQ for post-training quantization, QAT for quantization distillation training |
| --save_dir | Output of quantized model paths, which can be deployed directly in FastDeploy |
FastDeploy currently provides users with compression config files of multiple models, and the corresponding FP32 model, Users can directly download and experience it.
| Config file | FP32 model | Note |
|---|---|---|
| mobilenetv1_ssld_quant | mobilenetv1_ssld | |
| resnet50_vd_quant | resnet50_vd | |
| efficientnetb0_quant | efficientnetb0 | |
| mobilenetv3_large_x1_0_quant | mobilenetv3_large_x1_0 | |
| pphgnet_tiny_quant | pphgnet_tiny | |
| pplcnetv2_base_quant | pplcnetv2_base | |
| yolov5s_quant | yolov5s | |
| yolov6s_quant | yolov6s | |
| yolov7_quant | yolov7 | |
| ppyoloe_withNMS_quant | ppyoloe_l | Support PPYOLOE's s,m,l,x series models, export the model normally when exporting the model from PaddleDetection, do not remove NMS |
| ppyoloe_plus_withNMS_quant | ppyoloe_plus_s | Support PPYOLOE+'s s,m,l,x series models, export the model normally when exporting the model from PaddleDetection, do not remove NMS |
| pp_liteseg_quant | pp_liteseg | |
| deeplabv3_resnet_quant | deeplabv3_resnet101 | |
| fcn_hrnet_quant | fcn_hrnet | |
| unet_quant | unet |
Once obtained the quantized model, developers can deploy it on FastDeploy. Please refer to the following docs for more details