MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing. Su, Z., Zhang, Z., Lan, T., Wang, Z., Shen, H., Yang, J., & Cheng, Y. In Proceedings of the Ninth Annual Conference on Machine Learning and Systems, Bellevue, WA, May, 2026. Version Number: 2![MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper doi abstract bibtex Efficiently serving large language models (LLMs) under dynamic and bursty workloads remains a key challenge for real-world deployment. Existing serving frameworks and static model compression techniques fail to adapt to workload fluctuations, leading to either service-level objective (SLO) violations under full-precision serving or persistent accuracy degradation with static quantization. To deal with these issues, we present MorphServe, a dynamic, workload-aware LLM serving framework based on morphological adaptation. MorphServe introduces two asynchronous, token-level runtime mechanisms: quantized layer swapping, which selectively replaces less impactful layers with quantized alternatives during high-load periods, and pressure-aware KV cache resizing, which dynamically adjusts KV cache capacity in response to memory pressure. These mechanisms enable state-preserving transitions with minimum runtime overhead and are fully compatible with modern scheduling and attention techniques. Extensive experiments on Vicuna and Llama family models with real-world workloads demonstrate that MorphServe reduces average SLO violations by 92.45% and improves the P95 TTFT latency by 2.2-3.9× compared to full-precision serving, without compromising generation quality. These results establish MorphServe as a practical and elastic solution for LLM deployment in dynamic environments.
@inproceedings{su_morphserve_2026,
address = {Bellevue, WA},
title = {{MorphServe}: {Efficient} and {Workload}-{Aware} {LLM} {Serving} via {Runtime} {Quantized} {Layer} {Swapping} and {KV} {Cache} {Resizing}},
copyright = {Creative Commons Attribution 4.0 International},
shorttitle = {{MorphServe}},
url = {https://mlsys.org/virtual/2026/oral/3816},
doi = {10.48550/ARXIV.2506.02006},
abstract = {Efficiently serving large language models (LLMs) under dynamic and bursty workloads remains a key challenge for real-world deployment. Existing serving frameworks and static model compression techniques fail to adapt to workload fluctuations, leading to either service-level objective (SLO) violations under full-precision serving or persistent accuracy degradation with static quantization. To deal with these issues, we present MorphServe, a dynamic, workload-aware LLM serving framework based on morphological adaptation. MorphServe introduces two asynchronous, token-level runtime mechanisms: quantized layer swapping, which selectively replaces less impactful layers with quantized alternatives during high-load periods, and pressure-aware KV cache resizing, which dynamically adjusts KV cache capacity in response to memory pressure. These mechanisms enable state-preserving transitions with minimum runtime overhead and are fully compatible with modern scheduling and attention techniques. Extensive experiments on Vicuna and Llama family models with real-world workloads demonstrate that MorphServe reduces average SLO violations by 92.45\% and improves the P95 TTFT latency by 2.2-3.9× compared to full-precision serving, without compromising generation quality. These results establish MorphServe as a practical and elastic solution for LLM deployment in dynamic environments.},
language = {en},
urldate = {2026-07-07},
booktitle = {Proceedings of the {Ninth} {Annual} {Conference} on {Machine} {Learning} and {Systems}},
author = {Su, Zhaoyuan and Zhang, Zeyu and Lan, Tingfeng and Wang, Zirui and Shen, Haiying and Yang, Juncheng and Cheng, Yue},
month = may,
year = {2026},
note = {Version Number: 2},
keywords = {Distributed, Parallel, and Cluster Computing (cs.DC), FOS: Computer and information sciences, Machine Learning (cs.LG), WG: Explorable},
}
Downloads: 0
{"_id":"2d9pNA2WSajhFdxTC","bibbaseid":"su-zhang-lan-wang-shen-yang-cheng-morphserveefficientandworkloadawarellmservingviaruntimequantizedlayerswappingandkvcacheresizing-2026","author_short":["Su, Z.","Zhang, Z.","Lan, T.","Wang, Z.","Shen, H.","Yang, J.","Cheng, Y."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","address":"Bellevue, WA","title":"MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing","copyright":"Creative Commons Attribution 4.0 International","shorttitle":"MorphServe","url":"https://mlsys.org/virtual/2026/oral/3816","doi":"10.48550/ARXIV.2506.02006","abstract":"Efficiently serving large language models (LLMs) under dynamic and bursty workloads remains a key challenge for real-world deployment. Existing serving frameworks and static model compression techniques fail to adapt to workload fluctuations, leading to either service-level objective (SLO) violations under full-precision serving or persistent accuracy degradation with static quantization. To deal with these issues, we present MorphServe, a dynamic, workload-aware LLM serving framework based on morphological adaptation. MorphServe introduces two asynchronous, token-level runtime mechanisms: quantized layer swapping, which selectively replaces less impactful layers with quantized alternatives during high-load periods, and pressure-aware KV cache resizing, which dynamically adjusts KV cache capacity in response to memory pressure. These mechanisms enable state-preserving transitions with minimum runtime overhead and are fully compatible with modern scheduling and attention techniques. Extensive experiments on Vicuna and Llama family models with real-world workloads demonstrate that MorphServe reduces average SLO violations by 92.45% and improves the P95 TTFT latency by 2.2-3.9× compared to full-precision serving, without compromising generation quality. These results establish MorphServe as a practical and elastic solution for LLM deployment in dynamic environments.","language":"en","urldate":"2026-07-07","booktitle":"Proceedings of the Ninth Annual Conference on Machine Learning and Systems","author":[{"propositions":[],"lastnames":["Su"],"firstnames":["Zhaoyuan"],"suffixes":[]},{"propositions":[],"lastnames":["Zhang"],"firstnames":["Zeyu"],"suffixes":[]},{"propositions":[],"lastnames":["Lan"],"firstnames":["Tingfeng"],"suffixes":[]},{"propositions":[],"lastnames":["Wang"],"firstnames":["Zirui"],"suffixes":[]},{"propositions":[],"lastnames":["Shen"],"firstnames":["Haiying"],"suffixes":[]},{"propositions":[],"lastnames":["Yang"],"firstnames":["Juncheng"],"suffixes":[]},{"propositions":[],"lastnames":["Cheng"],"firstnames":["Yue"],"suffixes":[]}],"month":"May","year":"2026","note":"Version Number: 2","keywords":"Distributed, Parallel, and Cluster Computing (cs.DC), FOS: Computer and information sciences, Machine Learning (cs.LG), WG: Explorable","bibtex":"@inproceedings{su_morphserve_2026,\n\taddress = {Bellevue, WA},\n\ttitle = {{MorphServe}: {Efficient} and {Workload}-{Aware} {LLM} {Serving} via {Runtime} {Quantized} {Layer} {Swapping} and {KV} {Cache} {Resizing}},\n\tcopyright = {Creative Commons Attribution 4.0 International},\n\tshorttitle = {{MorphServe}},\n\turl = {https://mlsys.org/virtual/2026/oral/3816},\n\tdoi = {10.48550/ARXIV.2506.02006},\n\tabstract = {Efficiently serving large language models (LLMs) under dynamic and bursty workloads remains a key challenge for real-world deployment. Existing serving frameworks and static model compression techniques fail to adapt to workload fluctuations, leading to either service-level objective (SLO) violations under full-precision serving or persistent accuracy degradation with static quantization. To deal with these issues, we present MorphServe, a dynamic, workload-aware LLM serving framework based on morphological adaptation. MorphServe introduces two asynchronous, token-level runtime mechanisms: quantized layer swapping, which selectively replaces less impactful layers with quantized alternatives during high-load periods, and pressure-aware KV cache resizing, which dynamically adjusts KV cache capacity in response to memory pressure. These mechanisms enable state-preserving transitions with minimum runtime overhead and are fully compatible with modern scheduling and attention techniques. Extensive experiments on Vicuna and Llama family models with real-world workloads demonstrate that MorphServe reduces average SLO violations by 92.45\\% and improves the P95 TTFT latency by 2.2-3.9× compared to full-precision serving, without compromising generation quality. These results establish MorphServe as a practical and elastic solution for LLM deployment in dynamic environments.},\n\tlanguage = {en},\n\turldate = {2026-07-07},\n\tbooktitle = {Proceedings of the {Ninth} {Annual} {Conference} on {Machine} {Learning} and {Systems}},\n\tauthor = {Su, Zhaoyuan and Zhang, Zeyu and Lan, Tingfeng and Wang, Zirui and Shen, Haiying and Yang, Juncheng and Cheng, Yue},\n\tmonth = may,\n\tyear = {2026},\n\tnote = {Version Number: 2},\n\tkeywords = {Distributed, Parallel, and Cluster Computing (cs.DC), FOS: Computer and information sciences, Machine Learning (cs.LG), WG: Explorable},\n}\n\n\n\n","author_short":["Su, Z.","Zhang, Z.","Lan, T.","Wang, Z.","Shen, H.","Yang, J.","Cheng, Y."],"key":"su_morphserve_2026","id":"su_morphserve_2026","bibbaseid":"su-zhang-lan-wang-shen-yang-cheng-morphserveefficientandworkloadawarellmservingviaruntimequantizedlayerswappingandkvcacheresizing-2026","role":"author","urls":{"Paper":"https://mlsys.org/virtual/2026/oral/3816"},"keyword":["Distributed","Parallel","and Cluster Computing (cs.DC)","FOS: Computer and information sciences","Machine Learning (cs.LG)","WG: Explorable"],"metadata":{"authorlinks":{}},"downloads":0},"bibtype":"inproceedings","biburl":"https://bibbase.org/zotero-group/pratikmhatre/5933976","dataSources":["yJr5AAtJ5Sz3Q4WT4"],"keywords":["distributed","parallel","and cluster computing (cs.dc)","fos: computer and information sciences","machine learning (cs.lg)","wg: explorable"],"search_terms":["morphserve","efficient","workload","aware","llm","serving","via","runtime","quantized","layer","swapping","cache","resizing","su","zhang","lan","wang","shen","yang","cheng"],"title":"MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing","year":2026}