mct-model-optimization
Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient, constrained hardware. This project provides res
What is mct-model-optimization?
Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient, constrained hardware. This project provides researchers, developers, and engineers advanced quantization and compression tools for deploying state-of-the-art neural networks.
Overview of mct-model-optimization
What it is
Model Compression Toolkit (MCT) is an open source project designed for neural network model optimization under efficient, constrained hardware. It provides researchers, developers, and engineers advanced quantization and compression tools for deploying state-of-the-art neural networks.
Key features
- Mixed-precision search: Assigns optimal quantization bit-width per layer for weights and activations.
- Graph optimizations: Transforms the model to be best fitted for the quantization process.
- Quantization parameter search: Minimizes expected quantization noise during thresholds search using methods such as MSE, No-Clipping, and MAE.
- Advanced quantization algorithms: Includes Shift negative correction, Outliers filtering, and clustering to enhance quantization performance.
- Hardware-aware optimization (TPC): Describes the target hardware’s constraints for which the model optimization is targeted.
- Data-free quantization (Data Generation): Generates synthetic images based on statistics stored in the model's batch normalization layers for when image data isn’t available.
- Structured Pruning: Reduces model size/complexity by removing redundant input channels from layers and reconstructing layer weights.
Use cases
Deploying state-of-the-art neural networks on efficient, constrained hardware such as edge devices. Researchers and engineers can use MCT to quantize and compress models for reduced memory footprint, lower latency, and energy efficiency. Typical workflows include post-training quantization (PTQ), gradient-based post-training quantization (GPTQ) with optional EPTQ algorithm, quantization-aware training (QAT), and structured pruning.
Getting started
pip install model-compression-toolkitRequires Python>=3.10 with PyTorch>=2.3 or TensorFlow>=2.14. A pre-trained floating point model (PyTorch/Keras) is needed as input. See the repository README for additional installation options and tutorials.
License and pricing
Licensed under Apache-2.0. The project is open source; pricing is not stated.
Based on the project's public README and repository metadata.