Grounding Verbs of Motion in Natural Language Commands to Robots. Kollar, T., Tellex, S., Roy, D., & Roy, N. ![Grounding Verbs of Motion in Natural Language Commands to Robots [pdf]](https://bibbase.org/img/filetypes/pdf.svg "pdf")
Paper abstract bibtex To be useful teammates to human partners, robots must be able to follow spoken instructions given in natural language. An important class of instructions in-volve interacting with people, such as " Follow the person to the kitchen " or " Meet the person at the elevators. " These instructions require that the robot fluidly react to changes in the environment, not simply follow a pre-computed plan. We present an algorithm for understanding natural language commands with three components. First, we create a cost function that scores the language according to how well it matches a candidate plan in the environment, defined as the log-likelihood of the plan given the command. Components of the cost function include novel models for the meanings of motion verbs such as " follow, " " meet, " and " avoid, " as well as spa-tial relations such as " to " and landmark phrases such as " the kitchen. " Second, an inference method uses this cost function to perform forward search, finding a plan that matches the natural language command. Third, a high-level controller repeat-edly calls the inference method at each timestep to compute a new plan in response to changes in the environment such as the movement of the human partner or other people in the scene. When a command consists of more than a single task, the con-troller switches to the next task when an earlier one is satisfied. We evaluate our approach on a set of example tasks that require the ability to follow both simple and complex natural language commands.
@article{
title = {Grounding Verbs of Motion in Natural Language Commands to Robots},
type = {article},
id = {7f0544a7-0932-3ce1-802d-50cb4888b6c3},
created = {2017-09-01T15:53:36.922Z},
file_attached = {true},
profile_id = {80da7853-f7b7-36a9-8e4c-d7ddb2d9e538},
group_id = {a2333ea3-15a4-3d40-8d36-f0d9590ca926},
last_modified = {2017-09-01T15:53:37.109Z},
read = {false},
starred = {false},
authored = {false},
confirmed = {false},
hidden = {false},
abstract = {To be useful teammates to human partners, robots must be able to follow spoken instructions given in natural language. An important class of instructions in-volve interacting with people, such as " Follow the person to the kitchen " or " Meet the person at the elevators. " These instructions require that the robot fluidly react to changes in the environment, not simply follow a pre-computed plan. We present an algorithm for understanding natural language commands with three components. First, we create a cost function that scores the language according to how well it matches a candidate plan in the environment, defined as the log-likelihood of the plan given the command. Components of the cost function include novel models for the meanings of motion verbs such as " follow, " " meet, " and " avoid, " as well as spa-tial relations such as " to " and landmark phrases such as " the kitchen. " Second, an inference method uses this cost function to perform forward search, finding a plan that matches the natural language command. Third, a high-level controller repeat-edly calls the inference method at each timestep to compute a new plan in response to changes in the environment such as the movement of the human partner or other people in the scene. When a command consists of more than a single task, the con-troller switches to the next task when an earlier one is satisfied. We evaluate our approach on a set of example tasks that require the ability to follow both simple and complex natural language commands.},
bibtype = {article},
author = {Kollar, Thomas and Tellex, Stefanie and Roy, Deb and Roy, Nicholas}
}
Downloads: 0
{"_id":"LYxENPfPf4NrvquHt","bibbaseid":"kollar-tellex-roy-roy-groundingverbsofmotioninnaturallanguagecommandstorobots","downloads":0,"creationDate":"2017-09-01T15:59:40.169Z","title":"Grounding Verbs of Motion in Natural Language Commands to Robots","author_short":["Kollar, T.","Tellex, S.","Roy, D.","Roy, N."],"year":null,"bibtype":"article","biburl":null,"bibdata":{"title":"Grounding Verbs of Motion in Natural Language Commands to Robots","type":"article","id":"7f0544a7-0932-3ce1-802d-50cb4888b6c3","created":"2017-09-01T15:53:36.922Z","file_attached":"true","profile_id":"80da7853-f7b7-36a9-8e4c-d7ddb2d9e538","group_id":"a2333ea3-15a4-3d40-8d36-f0d9590ca926","last_modified":"2017-09-01T15:53:37.109Z","read":false,"starred":false,"authored":false,"confirmed":false,"hidden":false,"abstract":"To be useful teammates to human partners, robots must be able to follow spoken instructions given in natural language. An important class of instructions in-volve interacting with people, such as \" Follow the person to the kitchen \" or \" Meet the person at the elevators. \" These instructions require that the robot fluidly react to changes in the environment, not simply follow a pre-computed plan. We present an algorithm for understanding natural language commands with three components. First, we create a cost function that scores the language according to how well it matches a candidate plan in the environment, defined as the log-likelihood of the plan given the command. Components of the cost function include novel models for the meanings of motion verbs such as \" follow, \" \" meet, \" and \" avoid, \" as well as spa-tial relations such as \" to \" and landmark phrases such as \" the kitchen. \" Second, an inference method uses this cost function to perform forward search, finding a plan that matches the natural language command. Third, a high-level controller repeat-edly calls the inference method at each timestep to compute a new plan in response to changes in the environment such as the movement of the human partner or other people in the scene. When a command consists of more than a single task, the con-troller switches to the next task when an earlier one is satisfied. We evaluate our approach on a set of example tasks that require the ability to follow both simple and complex natural language commands.","bibtype":"article","author":"Kollar, Thomas and Tellex, Stefanie and Roy, Deb and Roy, Nicholas","bibtex":"@article{\n title = {Grounding Verbs of Motion in Natural Language Commands to Robots},\n type = {article},\n id = {7f0544a7-0932-3ce1-802d-50cb4888b6c3},\n created = {2017-09-01T15:53:36.922Z},\n file_attached = {true},\n profile_id = {80da7853-f7b7-36a9-8e4c-d7ddb2d9e538},\n group_id = {a2333ea3-15a4-3d40-8d36-f0d9590ca926},\n last_modified = {2017-09-01T15:53:37.109Z},\n read = {false},\n starred = {false},\n authored = {false},\n confirmed = {false},\n hidden = {false},\n abstract = {To be useful teammates to human partners, robots must be able to follow spoken instructions given in natural language. An important class of instructions in-volve interacting with people, such as \" Follow the person to the kitchen \" or \" Meet the person at the elevators. \" These instructions require that the robot fluidly react to changes in the environment, not simply follow a pre-computed plan. We present an algorithm for understanding natural language commands with three components. First, we create a cost function that scores the language according to how well it matches a candidate plan in the environment, defined as the log-likelihood of the plan given the command. Components of the cost function include novel models for the meanings of motion verbs such as \" follow, \" \" meet, \" and \" avoid, \" as well as spa-tial relations such as \" to \" and landmark phrases such as \" the kitchen. \" Second, an inference method uses this cost function to perform forward search, finding a plan that matches the natural language command. Third, a high-level controller repeat-edly calls the inference method at each timestep to compute a new plan in response to changes in the environment such as the movement of the human partner or other people in the scene. When a command consists of more than a single task, the con-troller switches to the next task when an earlier one is satisfied. We evaluate our approach on a set of example tasks that require the ability to follow both simple and complex natural language commands.},\n bibtype = {article},\n author = {Kollar, Thomas and Tellex, Stefanie and Roy, Deb and Roy, Nicholas}\n}","author_short":["Kollar, T.","Tellex, S.","Roy, D.","Roy, N."],"urls":{"Paper":"http://bibbase.org/service/mendeley/80da7853-f7b7-36a9-8e4c-d7ddb2d9e538/file/1ac84ed6-4072-cacb-4549-244c6e44d09c/Grounding_Verbs_of_Motion_in_Natural_Language_Commands_to_Robots.pdf.pdf"},"bibbaseid":"kollar-tellex-roy-roy-groundingverbsofmotioninnaturallanguagecommandstorobots","role":"author","downloads":0},"search_terms":["grounding","verbs","motion","natural","language","commands","robots","kollar","tellex","roy","roy"],"keywords":[],"authorIDs":[]}