Top Open-source Projects
Quadrotor DRL Simulation Platform
AirGym is an open souce Python quadrotor simulator based on IsaacGym, a part of AirGym series Sim-to-Real working flow. It provides a high-fidelity dynamics and Deep Reinforcement Learning (DRL) framework for quadrotor robot learning research. Furthermore, we also provide toolkits for transferring policy from AirGym simulator to the real quadrotor emNavi-X152b, making Sim-to-Real possible.
Book
Embodied Multi-Agent Systems
Huaping Liu, Xinzhu Liu, Kangyao Huang, Di Guo
This book focuses on active perception and interactive learning for embodied multi-agent systems. The remarkable reasoning, perception, and decision-making capabilities demonstrated by LLM in recent years have brought significant opportunities for the exploration of artificial general intelligence (AGI). This results in the development of increasingly larger models and a higher consumption of data. The ultimate goal is to achieve AGI through a unified brain model. However, when it comes to embodied agents, this strategy encounters considerable challenges due to the variety in morphology and function among these agents. It is neither feasible nor desirable to expect all embodied agents to conform to a single morphology. Instead, we should embrace the principles of biodiversity, promoting the existence, collaboration, and interaction of various forms. This recognition has motivated our research into embodied multi-agent systems. During this process, we have realized that active perception, along with the interactive learning capabilities that stem from it, plays a crucial role in fostering collaboration and synergy among multiple embodied agents.
We would like to thank Prof. Angelo Cangelosi, Prof. David Hsu, Prof. John Aloimonos who provide lots of support. We would like to express our gratitude to Dr. Hongbo Li from Geek+ and Dr. Tianlei Zhang from TrunkTech for their invaluable support in our study of embodied multi-agent collaboration. Additionally, we would like to express our sincere gratitude to Chenxu Wang, Xinghang Li, Juan Wang, Peiyan Li, Pingcheng Jian, and Chuye Hong for their significant assistance in preparing the figures and proofreading the book.
Selected Publications
LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integration
Wen Jiang, Li Wang, Kangyao Huang, Wei Fan, Jinyuan Liu, Shaoyu Liu, Hongwei Duan, Bin Xu, Xiangyang Ji
arXiv
Competitive Learning for Autonomous Flight
Kangyao Huang*,
Hao Wang*,
Di Guo,
Xiangkui Zhang,
Xiangyang Ji,
Huaping Liu
* contribute equally to this work
IEEE Robotics and Automation Letters (RA-L) 2025
Learning for Embodiment and Embodiment for Learning
Huaping Liu
Nature Reviews Electrical Engineering 2025 Comment Article
Towards Task-Oriented Flying: Framework, Infrastructure, and Principles
Kangyao Huang*,
Hao Wang*,
Jingyu Chen,
Jintao Chen,
Yu Luo,
Di Guo,
Xiangkui Zhang,
Xiangyang Ji,
Huaping Liu
* contribute equally to this work
arXiv
Learning a Distributed Hierarchical Locomotion Controller for Embodied Cooperation
Chuye Hong*,
Kangyao Huang*,
Huaping Liu
* contribute equally to this work
Conference on Robot Learning (CoRL) 2024
CompetEvo: Towards Morphological Evolution from Competition
Kangyao Huang,
Di Guo,
Xinyu Zhang,
Xiangyang Ji,
Huaping Liu
International Joint Conference on Artificial Intelligence (IJCAI) 2024
Stimulate the Potential of Robots via Competition
Kangyao Huang,
Di Guo,
Xinyu Zhang,
Xiangyang Ji,
Huaping Liu
IEEE International Conference on Robotics and Automation (ICRA) 2024
A Multi-modal Deformable Land-air Robot for Complex Environments
Xinyu Zhang,
Yuanhao Huang,
Kangyao Huang
IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) 2023
Coupled Modeling and Fusion Control for a Multi-modal Deformable Land-air Robot
Xinyu Zhang,
Yuanhao Huang,
Kangyao Huang
Submitted to RAS
Path Planning for Air-Ground Robot Considering Modal Switching Point Optimization
Xiaoyu Wang, Kangyao Huang, Xinyu Zhang, Honglin Sun, Wenzhuo Liu, Huaping Liu, Jun Li
International Conference on Unmanned Aircraft Systems (ICUAS) 2023
State-of-the-art and Technical Trends of Intelligent Flying Cars
Xinyu Zhang
中国科学（技术科学）, SCIENTIA SINICA Techinologica, 2024, TH-CPL-A, 2024年4月-第54卷第4期封面文章-Cover Paper
Wind-Resistant Flight Control for Amphibious Flying Car
Kangyao Huang,
Baoshang Zhou,
Songsong Rong,
Xinyu Zhang
International Conference on Guidance, Navigation and Control (ICGNC) 2024
Intelligent Amphibious Ground-aerial Vehicles: State of the Art Technology for Future Transportation
Xinyu Zhang,
Jiangeng Huang,
Yuanhao Huang,
Kangyao Huang
IEEE Transactions on Intelligent Vehicles (T-IV) 2022 IF: 14
Bio-inspired Multi-agent Model and Optimization Strategy for Collaborative Aerial Transport
Kangyao Huang,
Jingyu Chen,
John Oyekan
Chinese Intelligent Automation Conference (CIAC) 2021 Best Student Paper
Decentralised Aerial Swarm for Adaptive and Energy Efficient Transport of Unknown Loads
Kangyao Huang,
Jingyu Chen,
John Oyekan
Swarm and Evolutionary Computation (SWEVO) 2021 IF: 8.2
Projects
emNavi derives from "Embodied Navigation". The vision of emNavi is to make navigation more intelligent. Besides, emNavi is an open-sourced project that re-construct and optimize the navigation-related SoTA (state of the art) algorithms and apply them on robots, especially aerial robots, to promote the implementation of Embodied AI on mobile robots.
GhostLoc: Geospatial High-precise Optical System for Tactical Localization
Work with teammates @emNavi: Kai Tang(CV), Hao Wang (System), Wantong Qin (Design), Tao Peng (Hardware)
Brief: GhostLoc is a compact, high-altitude visual positioning system that achieves GPS-level precision in denied environments by fusing Visual-Inertial Odometry (VIO) with a Visual Positioning System (VPS). Requiring only lowcost fisheye cameras, IMU, and barometer, it provides smooth 6-DoF pose estimation at over 15 Hz with meter-level accuracy, enabling robust navigation for various aerial platforms without relying on GNSS signals. We integrated all learning-based models in one robust framework running on an onboard computer with RK3588s.
the Implementation of Flying Car for Ground-aerial Transportation
Partners: @School of Vehicle and Mobility, THU @Department of Computer Science and Technology, THU @Suzhou Automobile Research Institute-CN @Singapore University of Technology and Design @Bingo Intelligence Co., Ltd. @Fullymax Co., Ltd. @XY-UAV Co., Ltd. @RoboSense Co., Ltd.
Project Principal: Xinyu Zhang, Project Technical Leader: Kangyao Huang, Director: Jun Li
Other participants: Qihao Zhu, Qingjing Meng, Bo Cui, Songsong Rong, Haowen Shen, Guole Li, Huaping Liu, Jianxi Luo, Dafeng Jin, Jun Yang, Shuzhi Ge, Weiguo Yang, Yu Wan, Zhiqiang Yang, Zhenlong Ding, Xiaofeng Xu, Jiang Qian, Chaoyang Ha, Yuanhao Huang, Qiujiang Wu, Xingang Wu, Qifan Tan, Mo Zhou, Yang Shen, Li Wang, Yan Han, Zhaosheng Huang, Zhiwei Li, Lei Yang, Linxun Shi, Dazhong Xu, Kai Tang, et cetera.
In 2021, we successfully developed the first generation of Tsinghua Mengshi intelligent flying car. This vehicle is the world first electric manned rotorcraft amphibious ground-aerial vehicle with integrated intelligent driving function.
Roadside Autonomous Driving Architecture towards SCSTSV
Director: Qifan Tan, Xinyu Zhang, Jun Li @New Technology Concept Vehicles, THU
Brief: Towards the Smart City-Smart Transportation-Smart Vehicles(SCSTSV), we focus on the key areas of autonomous driving technology, particularly in the design and optimization of architectures for perception, decision-making, and control. I propose a comprehensive architecture that integrates perception, decision-making, and control into a unified framework, further enhancing system performance through the integration of road testing equipment.
Solar-powered UAV for Long Duration Cruising
Work with Colleagues @Bingo Intelligence Co., Ltd.
Brief: In 2017, we built a solar-powered unmanned aerial vehicles and focused on enhancing the endurance. Additionally, our team developed a dynamic power system with low vortex drag variable-pitch propellers to adapt to varying wind speeds during flight, enabling them to operate efficiently across diverse conditions. This research contributes to the advancement of renewable energy-powered UAV technology.
We honor the memory of Zhaoxi Wang, one of co-founders of the team, whose dedication and vision continue to inspire us as we move forward, and his spirit will forever remain an integral part of our journey,just like his aircraft did.
Auto-Patrolling eVTOL UAV and Tracking System
Work with Colleagues @Bingo Intelligence Co., Ltd.
Brief: This project developed an Auto-Patrolling eVTOL UAV and Tracking System, integrating the vertical takeoff and landing capability of an electric Vertical Take-Off and Landing (eVTOL) aircraft with advanced computer vision for autonomous detection and real-time tracking of targets, enabling automated surveillance and monitoring missions in dynamic environments such as security, traffic, and infrastructure inspection. For this purpose, we designed and developed this UAV entirely from scratch, encompassing its aerodynamic profile, airframe structure and composite materials, flight control systems, and cv detection capabilities based on third party optical device.
Oil-powered Variable-pitch Quadrotor
My dissertation for BEng degree. Notably, this is the first successful flight of oil-powered rotary-wing UAV utilizing variable-pitch control in China. We finished it at the end of 2015. Work with teammates Fanjie Kong, Jingdong Ma.
Brief: In 2015, I participated a groundbreaking project focused on the development of an oil-powered variable-pitch multirotor unmanned aerial vehicle (UAV). This project involved the comprehensive design and prototyping testing of the UAV. Our efforts encompassed the integration of advanced variable-pitch technology with traditional oil-powered UAV systems, leading to a novel and efficient aerial platform. We use gears and belts as the power transmission method, where belts can effectively reduce the vibrations generated by the engine. The successful flight testing signifies a significant milestone in the domestic UAV industry, showcasing the feasibility and potential of variable-pitch control in enhancing the performance and versatility of oil-powered UAVs.
Education
|
2022~now, | PhD candidate in Computer Science and Technology, Tsinghua University, China Dissertation: Robot Interactive Learning |
|
2019~2020, | MRes in Control & Systems Engineering, ACSE, the University of Sheffield, UK Dissertation: Cooperative Transport by Swarm Robots |
|
2012~2016, | BEng in Aircraft Design and Engineering, School of Astronautics, Northwestern Polytechnical University, China Dissertation: Oil-powered Quadrotor |