A Hierarchical Reinforcement Learning Framework for UAV Path Planning in Tactical Environments

Authors : Mahmut Nedim ALPDEMİR

Pages : 243-259

Doi:10.55525/tjst.1219845

View : 53 | Download : 15

Publication Date : 2023-03-29

Article Type : Research Paper

Abstract :Tactical UAV path planning under radar threat using reinforcement learning involves particular challenges ranging from modeling related difficulties to sparse feedback problem. Learning goal-directed behavior with sparse feedback from complex environments is a fundamental challenge for reinforcement learning algorithms. In this paper we extend our previous work in this area to provide a solution to the problem setting stated above, using Hierarchical Reinforcement Learning insert ignore into journalissuearticles values(HRL); in a novel way that involves a meta controller for higher level goal assignment and a controller that determines the lower-level actions of the agent. Our meta controller is based on a regression model trained using a state transition scheme that defines the evolution of goal designation, whereas our lower-level controller is based on a Deep Q Network insert ignore into journalissuearticles values(DQN); and is trained via reinforcement learning iterations. This two-layer framework ensures that an optimal plan for a complex path, organized as multiple goals, is achieved gradually, through piecewise assignment of sub-goals, and thus as a result of a staged, efficient and rigorous procedure.
Keywords : hierarchical reinforcement learning, tactical UAV path planning, machine learning, path optimization

ORIGINAL ARTICLE URL