gptq
Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning.
Installation
SKILL.md
Post-training quantization method that compresses LLMs to 4-bit with minimal accuracy loss using group-wise quantization.
| Group Size | Model Size | Accuracy | Speed | Recommendation |
| -1 (per-column) | Smallest | Best | Slowest | Research only | | 32 | Smaller | Better | Slower | High accuracy needed | | 128 | Medium | Good | Fast | Recommended default | | 256 | Larger | Lower | Faster | Speed critical | | 1024 | Largest | Lowest | Fastest | Not recommended |
Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning. Source: ovachiever/droid-tings.
Facts (cite-ready)
Stable fields and commands for AI/search citations.
- Install command
npx skills add https://github.com/ovachiever/droid-tings --skill gptq- Source
- ovachiever/droid-tings
- Category
- </>Dev Tools
- Verified
- —
- First Seen
- 2026-02-01
- Updated
- 2026-02-18
Quick answers
What is gptq?
Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning. Source: ovachiever/droid-tings.
How do I install gptq?
Open your terminal or command line tool (Terminal, iTerm, Windows Terminal, etc.) Copy and run this command: npx skills add https://github.com/ovachiever/droid-tings --skill gptq Once installed, the skill will be automatically configured in your AI coding environment and ready to use in Claude Code or Cursor
Where is the source repository?
https://github.com/ovachiever/droid-tings
Details
- Category
- </>Dev Tools
- Source
- user
- First Seen
- 2026-02-01