Prefill/decode disaggregation

Technique · Text & code

Predicted to be the next big provider-side optimization, since prefill is compute-bound and decode memory-bound