Quark Torch PTQ

Purpose

Take a PyTorch / Hugging Face LLM from model identification through confirmed AMD Quark PTQ. Perform intake, planning, manifest generation, execution, and output verification as one self-contained workflow.

This skill stops at the quantized model. It does not accept .onnx model input, train or fine-tune a model, or modify Quark package/source files.

Prerequisites

• Python 3.11 to 3.13, accelerator-matched PyTorch 2.2 or newer, amd-quark[cli], and datasets.
• The required ROCm version and GPU architecture depend on the PyTorch build and selected quantization scheme. Record the actual runtime, GPU architecture (gfx... from gcnArchName on AMD), host kernel, and driver, and verify support for the confirmed plan.
• No container image is required. If running in a container, record its image name or digest when available and verify GPU device access.
• Inspect and preserve HIP_VISIBLE_DEVICES, CUDA_VISIBLE_DEVICES, HSA_OVERRIDE_GFX_VERSION, PYTORCH_ROCM_ARCH, and PYTORCH_HIP_ALLOC_CONF. Include any required changes to device visibility, architecture, or memory allocation in the confirmed plan.

Inputs