AI Journal Club
Paper study focused on Efficient NLP and mathematical optimization
A journal club for reading, summarizing, and sharing the latest AI research papers.
Covers topics including model compression, efficient training, and structural optimization.
Course Schedule
| Week | Paper Title | Summary | Difficulty | Resources | Video |
|---|---|---|---|---|---|
| Week 0 | OT (Orientation) | Study introduction and operational guidelines | ★☆☆☆ | 📄 Download PDF | 📹 Watch Video |
| Week 1 | Efficient Methods for Natural Language Processing: A Survey | Comprehensive overview of NLP efficiency. Covers quantization, pruning, compression, and PEFT as optimization problems. | ★★☆☆ | 📄 Download PDF | Coming Soon |
| Week 2 | A Survey on Efficient Inference for Large Language Models | LLM optimization techniques from data, model, and system perspectives. Provides constrained optimization framework. | ★★☆☆ | 📄 Download PDF | Coming Soon |
| Week 3 | DRPruning: Efficient LLM Pruning through Distributionally Robust Optimization | DRO-based pruning — robust loss optimization against data distribution shifts. | ★★★☆ | 📄 Download PDF | Coming Soon |
| Week 4 | QPruner: Probabilistic Decision Quantization for Structured Pruning in LLMs | Bayesian optimization-based mixed-precision quantization + pruning. | ★★★☆ | 📄 Download PDF | 📹 Watch Video |
| Week 5 | L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on LLMs | Combines QAT with efficient fine-tuning approaches. | ★★★☆ | 📄 Download PDF | Coming Soon |
| Week 6 | ResLoRA: Identity Residual Mapping in Low-Rank Adaptation | Adds Residual Mapping to LoRA → rank-constrained optimization stability. | ★★☆☆ | 📄 Download PDF | Coming Soon |
| Week 7 | MeMoTune: Measure & Moment-Driven Fine-Tuning for Quantized LLMs | Statistical measure-based optimization for improved quantized LLM fine-tuning performance. | ★★★☆ | 📄 Download PDF | Coming Soon |
| Week 8 | DiffSkip: Differential Layer Skipping in LLMs | Formalizes layer skipping as differentiable optimization. | ★★★★ | 📄 Download PDF | Coming Soon |
| Week 9 | Constrained Decoding with Speculative Lookaheads | Approaches speculative decoding as constrained optimization problem. | ★★★★ | 📄 Download PDF | Coming Soon |
| Week 10 | DB-LLM: Accurate Dual-Binarization for Efficient LLMs | Weight + activation binarization → projected gradient-based optimization. | ★★★☆ | 📄 Download PDF | Coming Soon |
| Week 11 | PruneVid: Visual Token Pruning for Efficient Video LLMs | Visual token pruning optimization in multimodal LLMs. | ★★★☆ | 📄 Download PDF | Coming Soon |
| Week 12 | Efficient Continual Pre-training for Domain-Specific LLMs | Models domain adaptation pre-training as efficient optimization problem. | ★★☆☆ | 📄 Download PDF | Coming Soon |
| Week 13 | Efficient Large Language Models: A Survey | Latest efficient LLM survey. Wrap-up from multi-objective optimization perspective. | ★★☆☆ | 📄 Download PDF | Coming Soon |
Study Format
- Individual paper presentations (~10 minutes each)
- Shared paper preparation with rotating presentations
- Instructor-led lecture format on selected papers
Key Topics
- Model Compression: Pruning, Quantization, Low-Rank Adaptation (LoRA)
- Efficient Training: PEFT, Quantization-Aware Training (QAT)
- Structural Optimization: Layer Skipping, Speculative Decoding
- Optimization Surveys: Balancing Performance, Efficiency, and Fairness