Zijie Zhao / 赵子杰

Main Contributions

1. AAAI submitted

   TrustLoco: Robust Quadrupedal Parkour via Trust-Aware Vision-Proprioception Fusion

   Shengqian Chen^*, Zijie Zhao^*, Xiaoxu Wang, Yuanheng Zhu, Dongbin Zhao

   AAAI Conference on Artificial Intelligence (AAAI), 2027.


2. CoRL submitted

   LocoWM: High-Precision Locomotion through World-Model-Guided Residual Adaptation

   Zijie Zhao^*, Shengqian Chen^*, Xiaoxu Wang, Han Jiang, Yuanheng Zhu, Dongbin Zhao

   Conference on Robot Learning (CoRL), 2026.


3. ICLR

   Empowering Multi-Robot Cooperation via Sequential World Models

   Zijie Zhao, Honglei Guo, Shengqian Chen, Kaixuan Xu, Bo Jiang, Yuanheng Zhu, Dongbin Zhao

   International Conference on Learning Representations (ICLR), 2026.

   Abstract

   Model-based reinforcement learning (MBRL) has shown significant potential in robotics due to its high sample efficiency and planning capability. However, extending MBRL to multi-robot cooperation remains challenging due to the complexity of joint dynamics. To address this, we propose the Sequential World Model (SeqWM), a novel framework that integrates the sequential paradigm into model-based multi-agent reinforcement learning. SeqWM employs independent, sequentially structured agent-wise world models to decompose complex joint dynamics. Latent rollouts and decision-making are performed through sequential communication, where each agent generates its future trajectory and plans its actions based on the predictions of its predecessors. This design enables explicit intention sharing, enhancing cooperative performance, and reduces communication overhead to linear complexity. Results in challenging simulated environments (Bi-DexHands and Multi-Quad) show that SeqWM outperforms existing state-of-the-art model-free and model-based baselines in both overall performance and sample efficiency, while exhibiting advanced cooperative behaviors such as predictive adaptation and role division. Furthermore, SeqWM has been success fully deployed on physical quadruped robots, demonstrating its effectiveness in real-world multi-robot systems.
   

   PDF Code BibTeX

   @inproceedings{zhao2025seqwm,
       title={Empowering Multi-Robot Cooperation via Sequential World Models}, 
       author={Zijie Zhao and Honglei Guo and Shengqian Chen and Kaixuan Xu and Bo Jiang and Yuanheng Zhu and Dongbin Zhao},
       year={2025},
       eprint={2509.13095},
       archivePrefix={arXiv},
       primaryClass={cs.RO},
       url={https://arxiv.org/abs/2509.13095}, 
   }
   



4. NeurIPS

   Learning and Planning Multi-Agent Tasks via a MoE-based World Model

   Zijie Zhao, Zhongyue Zhao, Kaixuan Xu, Yuqian Fu, Jiajun Chai, Yuanheng Zhu, Dongbin Zhao

   Advances in Neural Information Processing Systems (NeurIPS), 2025.

   Abstract

   Multi-task multi-agent reinforcement learning (MT-MARL) aims to develop a single model capable of solving a diverse set of tasks. However, existing methods often fall short due to the substantial variation in optimal policies across tasks, making it challenging for a single policy model to generalize effectively. In contrast, we find that many tasks exhibit bounded similarity in their underlying dynamics—highly similar within certain groups (e.g., door-open/close) diverge significantly between unrelated tasks (e.g., door-open & object-catch). To leverage this property, we reconsider the role of modularity in multi-task learning, and propose M3W, a novel approach that applies mixture-of-experts (MoE) to world model instead of policy, enabling both learning and planning. For learning, it uses a SoftMoE-based dynamics model alongside a SparseMoE-based predictor to facilitate knowledge reuse across similar tasks while avoiding gradient conflicts across dissimilar tasks. For planning, it evaluates and optimizes actions using the predicted rollouts from the world model, without relying directly on a explicit policy model, thereby overcoming the limitations of policy-centric methods. As the first MoE-based multi-task world model, M3W demonstrates superior performance, sample efficiency, and multi-task adaptability, as validated on Bi-DexHands with 14 tasks and MA-Mujoco with 24 tasks.
   

   PDF Code BibTeX

   @inproceedings{zhao2025m3w,
     title     = {Learning and Planning Multi-Agent Tasks via a MoE-based World Model},
     author    = {Zhao, Zijie and Zhao, Zhongyue and Xu, Kaixuan and Fu, Yuqian and Chai, Jiajun and Zhu, Yuanheng and Zhao, Dongbin},
     booktitle = {Advances in Neural Information Processing Systems},
     year      = {2025}
   }
   



5. IEEE TNNLS

   Meta Learning Task Representation in Multiagent Reinforcement Learning: From Global Inference to Local Inference

   Zijie Zhao, Yuqian Fu, Jiajun Chai, Yuanheng Zhu, Dongbin Zhao

   IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2025.

   Abstract

   Multiagent meta reinforcement learning (MAMRL) enables multiagent systems (MASs) to adapt to multiple tasks. However, partial observability poses a significant challenge by hindering efficient task inference from agents' limited local experiences. To address this, we propose MG2L, a novel algorithm featuring a global-to-local (G2L) training scheme based on mutual information optimization (MIO). We first extend the centralized training and decentralized execution (CTDE) framework to MAMRL, and introduce a multilevel task encoder for joint global and local task inference. Building on this encoder, the MG2L scheme employs tailored loss functions to optimize task representations. For global inference, the MAS learns a centralized global representation by maximizing the MI between the representation and the task context. For local inference, we formulate conditional MI reduction to quantify the G2L gap. Agents then learn the local representation by minimizing this reduction. The MG2L scheme effectively harmonizes centralized training with decentralized execution, offering a versatile solution for MAMRL challenges. Additionally, we integrate a permutation-invariant attention (PIA) module into the task encoder to reduce sensitivity to behavior policy variations. Extensive experiments—including comparative analyses, ablation studies, meta-test evaluations, and visualizations—demonstrate MG2L's effectiveness.
   

   PDF Code BibTeX

   @article{zhao2025mg2l,
     author={Zhao, Zijie and Fu, Yuqian and Chai, Jiajun and Zhu, Yuanheng and Zhao, Dongbin},
     journal={IEEE Transactions on Neural Networks and Learning Systems}, 
     title={Meta Learning Task Representation in Multiagent Reinforcement Learning: From Global Inference to Local Inference}, 
     year={2025},
     volume={36},
     number={8},
     pages={14908-14921}
   }
   



6. IEEE TCSS

   Opinion dynamics of social networks with intermittent-influence leaders

   Zijie Zhao, Lei Shi, Tong Li, Jinliang Shao, Yuhua Cheng

   IEEE Transactions on Computational Social Systems (TCSS), 2023.

   Abstract

   This article constructs a leader-follower architecture by introducing intermittent-influence opinion leaders to the DeGroot model and analyzes the influence of this type of leaders on the evolution of opinions. Different from the existing studies where the leaders can convey their opinions to the followers uninterruptedly, the leaders in this article can only convey its opinion by broadcasting at some intermittent moments. First, we analyze the relationship between the leaders' broadcast moments and the consensus opinion of followers and explain that the marginal revenue of the broadcasts is diminishing. Second, we describe the connotation of assimilation and calculate the minimum number of broadcasts required for the leaders to assimilate the followers' opinions. Finally, aiming to make the consensus opinion of the followers and the leaders' as close as possible, we give an optimal strategy on how to select followers to broadcast. The correctness of theoretical results is verified by numerical simulations.
   

   PDF BibTeX

   @article{zhao2022opinion,
     author={Zhao, Zijie and Shi, Lei and Li, Tong and Shao, Jinliang and Cheng, Yuhua},
     journal={IEEE Transactions on Computational Social Systems}, 
     title={Opinion Dynamics of Social Networks With Intermittent-Influence Leaders}, 
     year={2023},
     volume={10},
     number={3},
     pages={1073-1082}
   }
   

Others

1. Tech Report

   G0.5: One Autoregressive Stream for Robot Reasoning and Action

   Galaxea AI Research

   Technical Report, 2026.

   Abstract

   "The prevailing recipe for Vision-Language-Action (VLA) models couples a pretrained VLM with a separately trained flow-matching action expert. This makes the VLM a context encoder rather than a decision-maker. Instead, we argue for focusing on the VLM backbone: a unified model with a single set of weights that generates both reasoning and actions within a single autoregressive token stream. We introduce G0.5, a pretrained autoregressive VLA in which a single transformer decoder emits reasoning and action tokens under a single objective. Three components make this tractable at foundation-model scale: a learnable cross-embodiment action tokenizer that maps heterogeneous robot actions into a shared vocabulary; a native chain-of-thought stream interleaving task decomposition, object grounding, and action hints with action tokens; and a visual memory module that injects multi-second history through the vision encoder. Because reasoning and action share a single set of weights, the pretrained VLM’s capabilities carry over to physical behavior: the model follows instructions closely, and prompts directly steer action granularity, task horizon, and out-of-distribution scene handling without further training. Pretrained on a large collection of robot datasets together with VQA samples, G0.5 surpasses state-of-the-art models across 7 independent regimes: real-world fine-tuning on R1-Lite/R1-Pro robots (76.7% vs. 53.3% for 𝜋0.5 and 24.4% for GR00T-N1.7), the 2025 BEHAVIOR Challenge on 50 long-horizon household mobile manipulation tasks using a generalist policy (31.4% vs. 26.3% for 𝜋0.5 and 26.1% for the challenge winner), DROID post-training followed by zero-shot transfer to an unseen environment and objects (82.5%), a language-following Pick-and-Place benchmark, LIBERO (98.9%), RoboTwin 2.0 (93.3%), and SimplerEnv-Bridge (87.3%)."
   

   PDF Code BibTeX

   @misc{liu2026g05autoregressivestreamrobot,
     title={G0.5: One Autoregressive Stream for Robot Reasoning and Action},
     author={Yicheng Liu and Zibin Dong and Baijun Ye and Tianyuan Yuan and Tao Jiang and Anqi Yang and Shicheng Cao and Haonan Liu and Yue Sun and Zihan Guo and Xiao Liu and Ke Dong and Changxun Pan and Chenru Wu and Tailai Cheng and Xiaoshu Ren and Xinlei Zhang and Jianning Cui and Zijie Zhao and Haoyu Zhang and Kaiming Xu and Haodong Yang and Bowen Zhang and Jiahui Niu and Shaoting Zhu and Shiduo Zhang and Hang Zhao},
     year={2026},
     eprint={2608.11739},
     archivePrefix={arXiv},
     primaryClass={cs.RO},
     url={https://arxiv.org/abs/2608.11739}
   }
   



2. AISE

   A Survey of Cooperative Multi-Agent Reinforcement Learning for Multi-Task Scenarios

   Jiajun Chai, Zijie Zhao, Yuanheng Zhu, Dongbin Zhao

   Artificial Intelligence Science and Engineering (AISE), 2025.

   Abstract

   Cooperative multi-agent reinforcement learning (MARL) is a key technology for enabling cooperation in complex multi-agent systems. It has achieved remarkable progress in areas such as gaming, autonomous driving, and multi-robot control. Empowering cooperative MARL with multi-task decision-making capabilities is expected to further broaden its application scope. In multi-task scenarios, cooperative MARL algorithms need to address 3 types of multi-task problems: reward-related multi-task, arising from different reward functions; multi-domain multi-task, caused by differences in state and action spaces, state transition functions; and scalability-related multi-task, resulting from the dynamic variation in the number of agents. Most existing studies focus on scalability-related multi-task problems. However, with the increasing integration between large language models (LLMs) and multi-agent systems, a growing number of LLM-based multi-agent systems have emerged, enabling more complex multi-task cooperation. This paper provides a comprehensive review of the latest advances in this field. By combining multi-task reinforcement learning with cooperative MARL, we categorize and analyze the 3 major types of multi-task problems under multi-agent settings, offering more fine-grained classifications and summarizing key insights for each. In addition, we summarize commonly used benchmarks and discuss future directions of research in this area, which hold promise for further enhancing the multi-task cooperation capabilities of multi-agent systems and expanding their practical applications in the real world.
   

   PDF BibTeX

   @ARTICLE{chai2025survey,
     author={Chai, Jiajun and Zhao, Zijie and Zhu, Yuanheng and Zhao, Dongbin},
     journal={Artificial Intelligence Science and Engineering}, 
     title={A Survey of Cooperative Multi-Agent Reinforcement Learning for Multi-Task Scenarios}, 
     year={2025},
     volume={1},
     number={2},
     pages={98-121},
   }
   



3. IEEE TCDS

   CPIG: Leveraging Consistency Policy with Intention Guidance for Multi-agent Exploration

   Yuqian Fu, Yuanheng Zhu, Haoran Li, Zijie Zhao, Jiajun Chai, Dongbin Zhao

   IEEE Transactions on Cognitive and Developmental Systems (TCDS), 2025.

   Abstract

   Efficient exploration is crucial in cooperative multi-agent reinforcement learning (MARL), especially in sparse-reward settings. However, due to the reliance on the unimodal policy, existing methods are prone to falling into the local optima, hindering the effective exploration of better policies. Furthermore, in sparse-reward settings, each agent tends to receive a scarce reward, which poses significant challenges to inter-agent cooperation. This not only increases the difficulty of policy learning but also degrades the overall performance of multi-agent tasks. To address these issues, we propose a Consistency Policy with Intention Guidance (CPIG), with two primary components: (a) introducing a multimodal policy to enhance the agent’s exploration capability, and (b) sharing the intention among agents to foster agent cooperation. For component (a), CPIG incorporates a Consistency model as the policy, leveraging its multimodal nature and stochastic characteristics to facilitate exploration. Regarding component (b), we introduce an Intention Learner to deduce the intention on the global state from each agent’s local observation. This intention then serves as a guidance for the Consistency Policy, promoting cooperation among agents. The proposed method is evaluated in multi-agent particle environments (MPE) and multi-agent MuJoCo (MAMuJoCo). Empirical results demonstrate that our method not only achieves comparable performance to various baselines in dense-reward environments but also significantly enhances performance in sparse-reward settings, outperforming state-of-the-art (SOTA) algorithms by 20%.
   

   PDF BibTeX

   @article{fu2025cpig,
     author={Fu, Yuqian and Zhu, Yuanheng and Li, Haoran and Zhao, Zijie and Chai, Jiajun and Zhao, Dongbin},
     journal={IEEE Transactions on Cognitive and Developmental Systems}, 
     title={CPIG: Leveraging Consistency Policy with Intention Guidance for Multi-agent Exploration}, 
     year={2025},
     volume={},
     number={},
     pages={1-13},
   }
   



4. ACL 2026

   CriticSearch: Fine-Grained Credit Assignment for Search Agents via a Retrospective Critic

   Yaocheng Zhang, Haohuan Huang, Zijun Song, Yuanheng Zhu, Zijie Zhao, Qichao Zhang, Dongbin Zhao

   The 64th Annual Meeting of the Association for Computational Linguistics (ACL), 2026.

   Abstract

   Tool-Integrated Reasoning (TIR) with search engines enables large language models to iteratively retrieve up-to-date external knowledge, enhancing adaptability and generalization in complex question-answering tasks. However, existing search agent pipelines typically depend on reinforcement learning based optimization, which often suffers from sparse outcome rewards, leading to inefficient exploration and unstable training. We introduce CriticSearch, a fine-grained credit-assignment framework that supplies dense, turn-level feedback via a retrospective critic mechanism. During training, a frozen, asymmetric critique LLM retrospectively evaluates each turn using privileged information from the full trajectory and gold answers, converting these assessments into stable, dense rewards that guide policy improvement. Experimental results across diverse multi-hop reasoning benchmarks demonstrate that CriticSearch consistently outperforms existing baselines, achieving faster convergence, improved training stability, and higher performance.
   

   PDF BibTeX

   @article{zhang2025criticsearch,
     title={CriticSearch: Fine-Grained Credit Assignment for Search Agents via a Retrospective Critic},
     author={Zhang, Yaocheng and Huang, Haohuan and Song, Zijun and Zhu, Yuanheng and Zhang, Qichao and Zhao, Zijie and Zhao, Dongbin},
     journal={arXiv preprint arXiv:2511.12159},
     year={2025}
   }