A Survey of Multi-Agent Reinforcement Learning from the Perspective of Task Characteristics

Yongli Li; Shuochen Ji; Mingyao Gong; Yiping Zhu; Xu Zhao; Lirong Zhou

doi:10.6919/ICJE.202609_12(9).0024

Authors

Yongli Li School of Aeronautical Engineering,JILIN UNIVERSITY OF CHEMICAL TECHNOLOGY, Jilin 132000, China
Shuochen Ji School of Aeronautical Engineering,JILIN UNIVERSITY OF CHEMICAL TECHNOLOGY, Jilin 132000, China
Mingyao Gong School of Aeronautical Engineering,JILIN UNIVERSITY OF CHEMICAL TECHNOLOGY, Jilin 132000, China
Yiping Zhu School of Aeronautical Engineering,JILIN UNIVERSITY OF CHEMICAL TECHNOLOGY, Jilin 132000, China
Xu Zhao School of Aeronautical Engineering,JILIN UNIVERSITY OF CHEMICAL TECHNOLOGY, Jilin 132000, China
Lirong Zhou School of Aeronautical Engineering,JILIN UNIVERSITY OF CHEMICAL TECHNOLOGY, Jilin 132000, China

DOI:

https://doi.org/10.6919/ICJE.202609_12(9).0024

Keywords:

Agents; Reinforcement Learning; Multi-agent Reinforcement Learning; Task Characteristics.

Abstract

Multi-agent reinforcement learning (MARL), the relationship between agents' rewards determines part of the learning problem. Shared rewards give rise to credit assignment and coordinated action selection; opposed rewards require responses to an adapting opponent. Both relationships occur in mixed tasks. This survey takes these task differences as the starting point for reviewing MARL, complementing classifications based on algorithms, training frameworks, or application domains. The analysis follows task characteristics, interaction types, and algorithmic mechanisms across cooperative, competitive, and mixed cooperative-competitive settings. It covers the basic formulations and training and execution paradigms, then compares representative methods in terms of their mechanisms, applicability, and limitations. Path planning, task allocation, pursuit-evasion and adversarial engagement, autonomous driving, and resource allocation illustrate how information access and operational constraints affect method selection. The concluding discussion concerns non-stationarity, scaling, safety, and the evidence needed for deployment. The comparisons support method selection under stated task conditions, without treating applicability as independent of the task being addressed.

Downloads

Download data is not yet available.

References

[1] Wong, A., Bäck, T., Kononova, A. V., et al. (2023). Deep multiagent reinforcement learning: challenges and directions. Artificial Intelligence Review, 56(6).

[2] Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2019). A survey and critique of multiagent deep reinforcement learning. Autonomous Agents and Multi-Agent Systems, 33(6), 750–797.

[3] Oroojlooy, A., & Hajinezhad, D. (2023). A review of cooperative multi-agent deep reinforcement learning. Applied Intelligence, 53(11), 13677–13722.

[4] Zhang, K., Yang, Z., & Başar, T. (2021). Multi-agent reinforcement learning: A selective overview of theories and algorithms. In Handbook of reinforcement learning and control (pp. 321–384).

[5] Li, T., Zhu, K., Luong, N. C., et al. (2022). Applications of multi-agent reinforcement learning in future internet: A comprehensive survey. IEEE Communications Surveys & Tutorials, 24(2), 1240–1279.

[6] Orr, J., & Dutta, A. (2023). Multi-agent deep reinforcement learning for multi-robot applications: A survey. Sensors, 23(7), 3625.

[7] Gronauer, S., & Diepold, K. (2022). Multi-agent deep reinforcement learning: a survey. Artificial Intelligence Review, 55(2), 895–943.

[8] Ning, Z., & Xie, L. (2024). A survey on multi-agent reinforcement learning and its application. Journal of Automation and Intelligence, 3(2), 73–91.

[9] Buşoniu, L., Babuška, R., & De Schutter, B. (2010). Multi-agent reinforcement learning: An overview. In Innovations in multi-agent systems and applications-1 (pp. 183–221).

[10] Yu, C., Velu, A., Vinitsky, E., et al. (2022). The surprising effectiveness of PPO in cooperative multi-agent games. Advances in Neural Information Processing Systems, 35, 24611–24624.

[11] Lowe, R., Wu, Y., Tamar, A., et al. (2017). Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in Neural Information Processing Systems, 30.

[12] Amato, C. (2024). An introduction to centralized training for decentralized execution in cooperative multi-agent reinforcement learning. arXiv. [https://arxiv.org/abs/2409.03052](https://arxiv.org/abs/2409.03052)

[13] Sunehag, P., Lever, G., Gruslys, A., et al. (2017). Value-decomposition networks for cooperative multi-agent learning. arXiv. [https://arxiv.org/abs/1706.05296](https://arxiv.org/abs/1706.05296)

[14] Rashid, T., Samvelyan, M., De Witt, C. S., et al. (2020). Monotonic value function factorisation for deep multi-agent reinforcement learning. Journal of Machine Learning Research, 21(178), 1–51.

[15] Son, K., Kim, D., Kang, W. J., et al. (2019). QTRAN: Learning to factorize with transformation. In International Conference on Machine Learning (pp. 5887–5896). PMLR.

[16] Wang, J., Ren, Z., Liu, T., et al. (2020). QPLEX: Duplex dueling multi-agent Q-learning. arXiv. [https://arxiv.org/abs/2008.01062](https://arxiv.org/abs/2008.01062)

[17] Kuba, J. G., Chen, R., Wen, M., et al. (2021). Trust region policy optimisation in multi-agent reinforcement learning. arXiv. [https://arxiv.org/abs/2109.11251](https://arxiv.org/abs/2109.11251)

[18] Littman, M. L. (1994). Markov games as a framework for multi-agent reinforcement learning. In Machine learning proceedings 1994 (pp. 157–163). Morgan Kaufmann.

[19] Heinrich, J., Lanctot, M., & Silver, D. (2015). Fictitious self-play in extensive-form games. In International Conference on Machine Learning (pp. 805–813). PMLR.

[20] Heinrich, J., & Silver, D. (2016). Deep reinforcement learning from self-play in imperfect-information games. arXiv. [https://arxiv.org/abs/1603.01121](https://arxiv.org/abs/1603.01121)

[21] Lanctot, M., Zambaldi, V., Gruslys, A., et al. (2017). A unified game-theoretic approach to multiagent reinforcement learning. Advances in Neural Information Processing Systems, 30.

[22] Ackermann, J., Gabler, V., Osa, T., et al. (2019). Reducing overestimation bias in multi-agent domains using double centralized critics. arXiv. [https://arxiv.org/abs/1910.01465](https://arxiv.org/abs/1910.01465)

[23] Iqbal, S., & Sha, F. (2019). Actor-attention-critic for multi-agent reinforcement learning. In International Conference on Machine Learning (pp. 2961–2970). PMLR.

[24] Sartoretti, G., Kerr, J., Shi, Y., et al. (2019). PRIMAL: Pathfinding via reinforcement and imitation multi-agent learning. IEEE Robotics and Automation Letters, 4(3), 2378–2385.

[25] Kong, X., Yang, J., Chai, X., et al. (2025). An advantage duPLEX dueling multi-agent Q-learning algorithm for multi-UAV cooperative target search in unknown environments. Simulation Modelling Practice and Theory, 142, 103118.

[26] Pham, H. X., La, H. M., Feil-Seifer, D., et al. (2018). Cooperative and distributed reinforcement learning of drones for field coverage. arXiv. [https://arxiv.org/abs/1803.07250](https://arxiv.org/abs/1803.07250)

[27] Liu, Y., Li, X., Wang, J., et al. (2024). Reinforcement-learning-based multi-UAV cooperative search for moving targets in 3D scenarios. Drones, 8(8), 378.

[28] Qin, W., Sun, Y. N., Zhuang, Z. L., et al. (2021). Multi-agent reinforcement learning-based dynamic task assignment for vehicles in urban transportation system. International Journal of Production Economics, 240, 108251.

[29] Yin, Y., Guo, Y., Su, Q., et al. (2022). Task allocation of multiple unmanned aerial vehicles based on deep transfer reinforcement learning. Drones, 6(8), 215.

[30] Wu, X., Yan, Q., Wang, J., et al. (2025). Dynamic task allocation for UAV swarms in maritime rescue scenarios based on PG-MAPPO. IEEE Internet of Things Journal.

[31] Wang, H., Qiu, T., Liu, Z., et al. (2020). Multi-agent formation control with obstacle avoidance under restricted communication through graph reinforcement learning. IFAC-PapersOnLine, 53(2), 8150–8156.

[32] Gu, S., Kuba, J. G., Chen, Y., et al. (2023). Safe multi-agent reinforcement learning for multi-robot control. Artificial Intelligence, 319, 103905.

[33] Kun, Y., Ao, S., Xu, N., et al. (2025). A review of reinforcement learning approaches for pursuit-evasion games. Chinese Journal of Aeronautics, 103940.

[34] Wan, K., Wu, D., Zhai, Y, et al. (2021). An improved approach towards multi-agent pursuit-evasion game decision-making using deep reinforcement learning. Entropy, 23(11), 1433.

[35] Peng, Z., Wu, G., Luo, B., et al. (2025). Multi-UAV cooperative pursuit strategy with limited visual field in urban airspace: A multi-agent reinforcement learning approach. IEEE/CAA Journal of Automatica Sinica.

[36] Lin, Y., Gao, H., & Xia, Y. (2025). Distributed pursuit-evasion game decision-making based on multi-agent deep reinforcement learning. Electronics, 14(11), 2141.

[37] Silver, D., Schrittwieser, J., Simonyan, K., et al. (2017). Mastering the game of Go without human knowledge. Nature, 550(7676), 354–359.

[38] Berner, C., Brockman, G., Chan, B., et al. (2019). Dota 2 with large scale deep reinforcement learning. arXiv. [https://arxiv.org/abs/1912.06680](https://arxiv.org/abs/1912.06680)

[39] Vinyals, O., Babuschkin, I., Czarnecki, W. M., et al. (2019). Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature, 575(7782), 350–354.

[40] Bighashdel, A., Wang, Y., McAleer, S., et al. (2024). Policy space response oracles: A survey. arXiv. [https://arxiv.org/abs/2403.02227](https://arxiv.org/abs/2403.02227)

[41] Yang, J., Yang, X., & Yu, T. (2024). Multi-unmanned aerial vehicle confrontation in intelligent air combat: A multi-agent deep reinforcement learning approach. Drones, 8(8), 382.

[42] Li, B., Wang, J., Song, C., et al. (2024). Multi-UAV roundup strategy method based on deep reinforcement learning CEL-MADDPG algorithm. Expert Systems with Applications, 245, 123018.

[43] Dinneweth, J., Boubezoul, A., Mandiau, R., et al. (2022). Multi-agent reinforcement learning for autonomous vehicles: A survey. Autonomous Intelligent Systems, 2(1), 27.

[44] Zheng, Z., & Gu, S. (2025). Safe multiagent reinforcement learning with bilevel optimization in autonomous driving. IEEE Transactions on Artificial Intelligence, 6(4), 829–842. [https://doi.org/10.1109/TAI.2024.3497919](https://doi.org/10.1109/TAI.2024.3497919)

[45] Hady, M. A., Hu, S., Pratama, M., et al. (2025). Multi-agent reinforcement learning for resource allocation optimization: A survey. Artificial Intelligence Review, 58(11), 354.

[46] Liu, Q., & Ma, Y. (2025). Communication resource allocation method in vehicular networks based on federated multi-agent deep reinforcement learning. Scientific Reports, 15(1), 30866.

[47] Xu, S., Liu, Q., Gong, C., et al. (2025). Energy-efficient multi-agent deep reinforcement learning task offloading and resource allocation for UAV edge computing. Sensors, 25(11), 3403.

[48] Gao, Z., Yang, L., & Dai, Y. (2023). Large-scale computation offloading using a multi-agent reinforcement learning in heterogeneous multi-access edge computing. IEEE Transactions on Mobile Computing, 22(6), 3425–3443. [https://doi.org/10.1109/TMC.2022.3141080](https://doi.org/10.1109/TMC.2022.3141080)