A Combined Deep Q-Network and Graph Search for Three Dimensional Route Planning Problems for Multiple Mobile Robots

A Combined Deep Q-Network and Graph Search for Three Dimensional Route Planning Problems for Multiple Mobile Robots. Fukushima, K., Nishi, T., & Liu, Z. In 2023 IEEE 19th International Conference on Automation Science and Engineering (CASE), pages 1-6, 8, 2023. IEEE.

Website doi abstract bibtex

In recent years, automated multiple mobile robots are introduced for transporting loads and inspecting final products in factories to reduce the burden of human labor shortage. Mobile robots are required to develop automated systems that can make decisions as flexibly like human operators. Most conventional route planning problems for mobile robots have been utilizing either, optimization methods or learning methods. However, those conventional methods have a difficulty in applying it to the conflict-free route planning problems with a large number of states with three dimensional environment. We propose a method that combines deep reinforcement learning and graph search methods. In the proposed method, the routing is firstly determined by a graph search algorithm, and Deep Q-Network (DQN). A deep reinforcement learning method is used to avoid collisions. A route planning problem in a three dimensional environment is successfully solved by using DQN that can process multi dimensional states. The proposed method is also applied to the multiple drones route planning problem. The performance of the proposed method is compared with that of the optimization methods. As a result, it was found that a near optimal route planning was obtained in approximately 6% of the computation time required to find the optimal solution.

@inproceedings{
 title = {A Combined Deep Q-Network and Graph Search for Three Dimensional Route Planning Problems for Multiple Mobile Robots},
 type = {inproceedings},
 year = {2023},
 pages = {1-6},
 websites = {https://ieeexplore.ieee.org/document/10260638/},
 month = {8},
 publisher = {IEEE},
 day = {26},
 id = {421ab32c-7f43-381c-baca-812847420f2a},
 created = {2024-08-31T23:34:51.713Z},
 file_attached = {false},
 profile_id = {c1ff1b97-6df6-3341-988d-56e0a24fa7a2},
 last_modified = {2024-09-01T13:46:16.890Z},
 read = {false},
 starred = {false},
 authored = {true},
 confirmed = {false},
 hidden = {false},
 source_type = {inproceedings},
 private_publication = {false},
 abstract = {In recent years, automated multiple mobile robots are introduced
for transporting loads and inspecting final products in factories
to reduce the burden of human labor shortage. Mobile robots are
required to develop automated systems that can make decisions as
flexibly like human operators. Most conventional route planning
problems for mobile robots have been utilizing either,
optimization methods or learning methods. However, those
conventional methods have a difficulty in applying it to the
conflict-free route planning problems with a large number of
states with three dimensional environment. We propose a method
that combines deep reinforcement learning and graph search
methods. In the proposed method, the routing is firstly
determined by a graph search algorithm, and Deep Q-Network (DQN).
A deep reinforcement learning method is used to avoid collisions.
A route planning problem in a three dimensional environment is
successfully solved by using DQN that can process multi
dimensional states. The proposed method is also applied to the
multiple drones route planning problem. The performance of the
proposed method is compared with that of the optimization
methods. As a result, it was found that a near optimal route
planning was obtained in approximately 6% of the computation
time required to find the optimal solution.},
 bibtype = {inproceedings},
 author = {Fukushima, Konosuke and Nishi, Tatsushi and Liu, Ziang},
 doi = {10.1109/CASE56687.2023.10260638},
 booktitle = {2023 IEEE 19th International Conference on Automation Science and Engineering (CASE)}
}

Downloads: 0

{"_id":"nppQE7fuqPmasgDbe","bibbaseid":"fukushima-nishi-liu-acombineddeepqnetworkandgraphsearchforthreedimensionalrouteplanningproblemsformultiplemobilerobots-2023","author_short":["Fukushima, K.","Nishi, T.","Liu, Z."],"bibdata":{"title":"A Combined Deep Q-Network and Graph Search for Three Dimensional Route Planning Problems for Multiple Mobile Robots","type":"inproceedings","year":"2023","pages":"1-6","websites":"https://ieeexplore.ieee.org/document/10260638/","month":"8","publisher":"IEEE","day":"26","id":"421ab32c-7f43-381c-baca-812847420f2a","created":"2024-08-31T23:34:51.713Z","file_attached":false,"profile_id":"c1ff1b97-6df6-3341-988d-56e0a24fa7a2","last_modified":"2024-09-01T13:46:16.890Z","read":false,"starred":false,"authored":"true","confirmed":false,"hidden":false,"source_type":"inproceedings","private_publication":false,"abstract":"In recent years, automated multiple mobile robots are introduced\nfor transporting loads and inspecting final products in factories\nto reduce the burden of human labor shortage. Mobile robots are\nrequired to develop automated systems that can make decisions as\nflexibly like human operators. Most conventional route planning\nproblems for mobile robots have been utilizing either,\noptimization methods or learning methods. However, those\nconventional methods have a difficulty in applying it to the\nconflict-free route planning problems with a large number of\nstates with three dimensional environment. We propose a method\nthat combines deep reinforcement learning and graph search\nmethods. In the proposed method, the routing is firstly\ndetermined by a graph search algorithm, and Deep Q-Network (DQN).\nA deep reinforcement learning method is used to avoid collisions.\nA route planning problem in a three dimensional environment is\nsuccessfully solved by using DQN that can process multi\ndimensional states. The proposed method is also applied to the\nmultiple drones route planning problem. The performance of the\nproposed method is compared with that of the optimization\nmethods. As a result, it was found that a near optimal route\nplanning was obtained in approximately 6% of the computation\ntime required to find the optimal solution.","bibtype":"inproceedings","author":"Fukushima, Konosuke and Nishi, Tatsushi and Liu, Ziang","doi":"10.1109/CASE56687.2023.10260638","booktitle":"2023 IEEE 19th International Conference on Automation Science and Engineering (CASE)","bibtex":"@inproceedings{\n title = {A Combined Deep Q-Network and Graph Search for Three Dimensional Route Planning Problems for Multiple Mobile Robots},\n type = {inproceedings},\n year = {2023},\n pages = {1-6},\n websites = {https://ieeexplore.ieee.org/document/10260638/},\n month = {8},\n publisher = {IEEE},\n day = {26},\n id = {421ab32c-7f43-381c-baca-812847420f2a},\n created = {2024-08-31T23:34:51.713Z},\n file_attached = {false},\n profile_id = {c1ff1b97-6df6-3341-988d-56e0a24fa7a2},\n last_modified = {2024-09-01T13:46:16.890Z},\n read = {false},\n starred = {false},\n authored = {true},\n confirmed = {false},\n hidden = {false},\n source_type = {inproceedings},\n private_publication = {false},\n abstract = {In recent years, automated multiple mobile robots are introduced\nfor transporting loads and inspecting final products in factories\nto reduce the burden of human labor shortage. Mobile robots are\nrequired to develop automated systems that can make decisions as\nflexibly like human operators. Most conventional route planning\nproblems for mobile robots have been utilizing either,\noptimization methods or learning methods. However, those\nconventional methods have a difficulty in applying it to the\nconflict-free route planning problems with a large number of\nstates with three dimensional environment. We propose a method\nthat combines deep reinforcement learning and graph search\nmethods. In the proposed method, the routing is firstly\ndetermined by a graph search algorithm, and Deep Q-Network (DQN).\nA deep reinforcement learning method is used to avoid collisions.\nA route planning problem in a three dimensional environment is\nsuccessfully solved by using DQN that can process multi\ndimensional states. The proposed method is also applied to the\nmultiple drones route planning problem. The performance of the\nproposed method is compared with that of the optimization\nmethods. As a result, it was found that a near optimal route\nplanning was obtained in approximately 6% of the computation\ntime required to find the optimal solution.},\n bibtype = {inproceedings},\n author = {Fukushima, Konosuke and Nishi, Tatsushi and Liu, Ziang},\n doi = {10.1109/CASE56687.2023.10260638},\n booktitle = {2023 IEEE 19th International Conference on Automation Science and Engineering (CASE)}\n}","author_short":["Fukushima, K.","Nishi, T.","Liu, Z."],"urls":{"Website":"https://ieeexplore.ieee.org/document/10260638/"},"biburl":"https://bibbase.org/service/mendeley/c1ff1b97-6df6-3341-988d-56e0a24fa7a2","bibbaseid":"fukushima-nishi-liu-acombineddeepqnetworkandgraphsearchforthreedimensionalrouteplanningproblemsformultiplemobilerobots-2023","role":"author","metadata":{"authorlinks":{}}},"bibtype":"inproceedings","biburl":"https://bibbase.org/service/mendeley/c1ff1b97-6df6-3341-988d-56e0a24fa7a2","dataSources":["jZMMqeTQx4XTrhw8L","2252seNhipfTmjEBQ"],"keywords":[],"search_terms":["combined","deep","network","graph","search","three","dimensional","route","planning","problems","multiple","mobile","robots","fukushima","nishi","liu"],"title":"A Combined Deep Q-Network and Graph Search for Three Dimensional Route Planning Problems for Multiple Mobile Robots","year":2023}