MIT HAN Lab
MIT HAN Lab is an MIT research lab focused on efficient generative AI, including large language models, multimodal models, and diffusion models. It uses algorithm–system co-design across pre-training, post-training, compression, deployment, and token-factory infrastructure to make generative AI faster, greener, more accessible, and practical on resource-constrained hardware.
At a Glance
- AI researchers and open-source developers
- LLM/VLM infrastructure and inference-serving teams
- Cloud and edge AI developers
- Organizations deploying generative AI on constrained or low-power hardware
- +1 more
AI Tools by MIT HAN Lab
(1)
AWQ (llm-awq)
LLM Weight Quantization Toolkit
Discussions
No discussions yet
Be the first to start a discussion about MIT HAN Lab
Latest News
Lightning Weave introduced an offline post-training framework that composes accuracy- and efficiency-oriented capabilities for reasoning models.
Lightning OPD 2.0 introduced cross-fitted residualization to mitigate style bias in cross-teacher on-policy distillation.
Quant VideoGen introduced training-free 2-bit KV-cache quantization for autoregressive long-video generation, cutting KV memory by up to 7x with under 4% latency overhead.
FourTune introduced an end-to-end W4A4G4 post-training framework for diffusion models, reporting 2.25x lower memory overhead and 2.27x higher throughput than BF16 LoRA on FLUX.1-dev.
Products & Services
An open-source, hardware-friendly low-bit weight-only quantization method for LLM compression and acceleration. It protects salient weights identified through activations, supports INT3/INT4 quantization, instruction-tuned and multimodal models, and is accompanied by a model zoo and efficient inference kernels.
A turnkey, lightweight chatbot and on-device inference framework built around AWQ for running large language and vision-language models on cloud GPUs and resource-constrained edge devices, including NVIDIA Jetson Orin.
The newer TinyChat release with faster prefilling for edge LLMs and VLMs; the project reports 1.5–1.7x higher prefilling speed than the prior version.
Pre-computed AWQ search results and quantized-model support covering model families including Llama, OPT, Vicuna, LLaVA, CodeLlama, StarCoder, VILA/NVILA, Qwen, and DeepSeek-R1-Distill.
Market Position
MIT HAN Lab positions itself as an academic leader in efficient generative-AI computing through algorithm–system co-design. AWQ differentiates its low-bit quantization by using activation observations to protect salient weights, avoiding backpropagation/reconstruction, preserving generalization across instruction-tuned and multimodal models, and pairing the algorithm with optimized inference kernels and TinyChat for practical edge deployment. Its impact is evidenced by adoption in major serving and inference ecosystems including TensorRT-LLM, vLLM, Transformers, TGI, FastChat, LMDeploy, Vertex AI, and SageMaker.
Leadership
Executive Team
Song Han
Principal Investigator; Associate Professor, MIT EECS
Received his PhD from Stanford advised by Bill Dally. His work includes Deep Compression, the Efficient Inference Engine, TinyML, hardware-aware neural architecture search, and LLM quantization/acceleration including SmoothQuant, AWQ, and StreamingLLM.
Founding Story
The lab's stated motivation is that foundation models are powerful but prohibitively costly in computation, energy, and scalability. It was organized around Song Han's research program in efficient AI computing, with an initial and continuing vision of co-designing algorithms, software, and hardware to democratize advanced AI deployment.
Target Markets
- AI researchers and open-source developers
- LLM/VLM infrastructure and inference-serving teams
- Cloud and edge AI developers
- Organizations deploying generative AI on constrained or low-power hardware
- Academic and industry partners working on efficient AI, model compression, acceleration, and physical AI
- Compressing and accelerating LLM inference
- Serving instruction-tuned chat models
- On-device and offline LLM/VLM chatbots
- Efficient multimodal and video-language inference
- Deploying large models on laptops, desktop GPUs, mobile GPUs, and low-power edge devices
- Reducing memory footprint and memory-bandwidth bottlenecks in model serving