Prefill/decode disaggregation
Technique · Text & code
Predicted to be the next big provider-side optimization, since prefill is compute-bound and decode memory-bound
Technique · Text & code
Predicted to be the next big provider-side optimization, since prefill is compute-bound and decode memory-bound