Tag
model-placement
- Sep 20, 2026 Adapter Residency When One Replica Serves Many Fine-Tunes Low-rank adapters let dozens of fine-tunes share a base model, but each token still needs one adapter applied. Residency, batch grouping, and routing decide whether that sharing pays. inference systems
- Feb 26, 2026 Replica Packing and Topology Constraints Tensor parallelism turns a model into a group of devices that must be co-located and upgraded together. That group, not the individual replica, is the unit a scheduler can place. inference systems