<\/p>\n\n\n\n
Training trillion-parameter models has made collective communication a dominant bottleneck in GPU clusters. Existing approaches face a fundamental tension: optimization-based synthesis techniques suffer from combinatorial complexity that can require minutes to hours, while analytic heuristics rely on rigid assumptions that often fail under production variability. To the best of our knowledge, we present OptiFlow, among the first LLM-driven frameworks for automated design of high-performance collective communication algorithms. Our key insight is a two-layer decomposition: the LLM generates compact data-movement intent expressed in a domain-specific language, while deterministic scheduling algorithms compile these programs into executable schedules. We further employ an iterative pipeline that uses real-hardware feedback to refine LLM-generated proposals, without updating model parameters or relying on manually designed search heuristics. Evaluated on a 32-GPU NVIDIA A100 cluster, OptiFlow discovers All-Gather schedules that outperform NCCL by up to 3.75 \u00d7 , while it also surpasses TACCL and TE-CCL with gains up to 5.12 \u00d7 and 3.28 \u00d7 , and shows better optimization-time scalability.<\/p>\n","protected":false},"excerpt":{"rendered":"
Training trillion-parameter models has made collective communication a dominant bottleneck in GPU clusters. Existing approaches face a fundamental tension: optimization-based synthesis techniques suffer from combinatorial complexity that can require minutes to hours, while analytic heuristics rely on rigid assumptions that often fail under production variability. To the best of our knowledge, we present OptiFlow, among […]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[{"type":"text","value":"Fei Long","user_id":0},{"type":"user_nicename","value":"Ziyue Yang","user_id":"41653"},{"type":"text","value":"Kaihui Gao","user_id":0},{"type":"user_nicename","value":"Shuai Wang","user_id":"33641"},{"type":"text","value":"Li Chen","user_id":0},{"type":"user_nicename","value":"Zhixiong Niu","user_id":"38118"},{"type":"edited_text","value":"Ran Shu","user_id":"37926"},{"type":"user_nicename","value":"Wenxue Cheng","user_id":"43500"},{"type":"user_nicename","value":"Peng Cheng","user_id":"33225"},{"type":"user_nicename","value":"Yongqiang Xiong","user_id":"35049"},{"type":"user_nicename","value":"Dan Li","user_id":"31531"}],"msr_publishername":"Asia-Pacific Workshop on Networking","msr_publisher_other":"","msr_booktitle":"Asia-Pacific Workshop on Networking","msr_chapter":"","msr_edition":"Asia-Pacific Workshop on Networking","msr_editors":"","msr_how_published":"","msr_isbn":"","msr_issue":"","msr_journal":"Proceedings of the 10th Asia-Pacific Workshop on Networking","msr_number":"","msr_organization":"","msr_pages_string":"","msr_page_range_start":"","msr_page_range_end":"","msr_series":"","msr_volume":"","msr_copyright":"","msr_conference_name":"Asia-Pacific Workshop on Networking","msr_doi":"10.1145\/3820441.3820452","msr_arxiv_id":"","msr_mag_id":"","msr_other_authors":"","msr_other_contributors":"","msr_speaker":"","msr_award":"","msr_affiliation":"","msr_institution":"","msr_host":"","msr_version":"","msr_duration":"","msr_release_tracker_id":"","msr_highlight_type":"","msr_date_display_format":"","msr_main_download_label":"","msr_external_link_label":"","msr_doi_label":"","msr_published_date":"2026-08-05","msr_startdate":"","msr_presentation_date":"","msr_highlight_text":"","msr_notes":"","msr_longbiography":"","msr_publicationurl":"","msr_external_url":"","msr_secondary_video_url":"","msr_conference_url":"","msr_journal_url":"","msr_year":2026,"msr_month":8,"msr_day":5,"msr_microsoftintellectualproperty":false,"msr_pub_id":"222db898736f47295a9e56fa7b38c431e0448c3c","msr_publication_uploader":[{"type":"url","title":"https:\/\/dblp.org\/rec\/conf\/apnet\/LongYGWCNSCCXL26.html","label_id":243109,"id":false,"viewUrl":false}],"msr_related_uploader":[],"msr_original_fields_of_study":[],"msr_s2_paper_id":"222db898736f47295a9e56fa7b38c431e0448c3c","msr_s2_pdf_url":"https:\/\/doi.org\/10.1145\/3820441.3820452","msr_citation_count_updated":"","msr_citation_count":0,"msr_influential_citations":0,"msr_reference_count":13,"msr_s2_open_access":true,"msr_s2_author_ids":[],"msr_pub_ids":[{"provider":"s2","id":"222db898736f47295a9e56fa7b38c431e0448c3c"},{"provider":"doi","id":"10.1145\/3820441.3820452"},{"provider":"dblp","id":"conf\/apnet\/LongYGWCNSCCXL26"},{"provider":"corpusid","id":"290372923"}],"msr_hide_image_in_river":0,"footnotes":""},"msr-research-highlight":[],"research-area":[13556,13547],"msr-publication-type":[193715,193719],"msr-publisher":[],"msr-publication-cta":[],"msr-focus-area":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[256039,246691],"msr-conference":[270498],"msr-journal":[270493],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-1181181","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-artificial-intelligence","msr-research-area-systems-and-networking","msr-locale-en_us","msr-field-of-study-artifcial-intelligence","msr-field-of-study-computer-science"],"msr_publishername":"Asia-Pacific Workshop on Networking","msr_edition":"Asia-Pacific Workshop on Networking","msr_affiliation":"","msr_published_date":"2026-08-05","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"Asia-Pacific Workshop on Networking","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"Proceedings of the 10th Asia-Pacific Workshop on Networking","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":0,"msr_main_download":"","msr_publicationurl":"","msr_doi":"10.1145\/3820441.3820452","msr_publication_uploader":[{"type":"url","title":"https:\/\/dblp.org\/rec\/conf\/apnet\/LongYGWCNSCCXL26.html","label_id":243109,"id":false,"viewUrl":false}],"msr_related_uploader":[],"msr_citation_count":0,"msr_citation_count_updated":"","msr_s2_paper_id":"222db898736f47295a9e56fa7b38c431e0448c3c","msr_influential_citations":0,"msr_reference_count":13,"msr_arxiv_id":"","msr_s2_author_ids":[],"msr_s2_open_access":true,"msr_s2_pdf_url":"https:\/\/doi.org\/10.1145\/3820441.3820452","msr_attachments":[],"msr-author-ordering":[{"type":"text","value":"Fei Long","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Ziyue Yang","user_id":41653,"rest_url":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Ziyue Yang"},{"type":"text","value":"Kaihui Gao","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Shuai Wang","user_id":33641,"rest_url":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Shuai Wang"},{"type":"text","value":"Li Chen","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Zhixiong Niu","user_id":38118,"rest_url":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Zhixiong Niu"},{"type":"edited_text","value":"Ran Shu","user_id":37926,"rest_url":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Ran Shu"},{"type":"user_nicename","value":"Wenxue Cheng","user_id":43500,"rest_url":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Wenxue Cheng"},{"type":"user_nicename","value":"Peng Cheng","user_id":33225,"rest_url":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Peng Cheng"},{"type":"user_nicename","value":"Yongqiang Xiong","user_id":35049,"rest_url":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Yongqiang Xiong"},{"type":"user_nicename","value":"Dan Li","user_id":31531,"rest_url":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Dan Li"}],"msr_impact_theme":[],"msr_research_lab":[199560,1012650],"msr_event":[],"msr_group":[],"msr_project":[],"publication":[],"video":[],"msr-tool":[],"msr_publication_type":"article","related_content":[],"_links":{"self":[{"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1181181","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":3,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1181181\/revisions"}],"predecessor-version":[{"id":1181649,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1181181\/revisions\/1181649"}],"wp:attachment":[{"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1181181"}],"wp:term":[{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=1181181"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1181181"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=1181181"},{"taxonomy":"msr-publisher","embeddable":true,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-publisher?post=1181181"},{"taxonomy":"msr-publication-cta","embeddable":true,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-cta?post=1181181"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=1181181"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1181181"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1181181"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=1181181"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=1181181"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=1181181"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1181181"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/research.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=1181181"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}