Unsupervised Semantic Perception, Summarization, and Autonomous Exploration for Robots in Unstructured Environments. Girdhar, Y. Ph.D. Thesis, McGill University, 2014. ![Unsupervised Semantic Perception, Summarization, and Autonomous Exploration for Robots in Unstructured Environments [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper abstract bibtex This thesis explores the challenges involved in building autonomous exploration and monitoring systems, and makes contributions on four fronts: describing the semantic context of the collected data, summarizing this information, deciding where to collect this data, and making optimal online irrevocable decisions for physical sample collection. Making high level decisions based on the environmental context of a robot's location requires that we first describe what is being observed in a semantic space with higher level of abstraction than the low level sensor reading. ROST, a realtime online spatiotemporal topic modeling technique that we develop in this thesis solves the problem of obtaining such high level descriptors. Topics in this case represent the latent causes (such as objects and terrains), which produce these observations. ROST extends previous work on topic modeling by efficiently taking into account the spatiotemporal context of an observation, and using a novel Gibbs sampling technique to refine the topic label assignment in realtime, making it suitable for processing streaming sensor data such as video and audio observed by a robot. Our experiments suggest that taking into account the spatiotemporal context of observations results in better topic labels that have higher mutual information with ground truth labels, compared to topic modeling without taking into account the spatiotemporal context. Moreover we show that the perplexity of the online topic model using the proposed Gibbs sampler is competitive with batch Gibbs sampler. Given a scene descriptor such as bag-of-words, location, or topic distribution, the thesis then proposes a novel online summarization algorithm, which unlike previous techniques focuses on building a navigation summary containing all the surprising scenes observed by the robot. We argue that the summaries produced by the algorithm (called extremum summaries) are ideal for monitoring and inspections tasks, where the goal is to maintain a small set of images that is representative of the diversity of what has been observed. Although computation of an optimal summary, even in the batch case is NP-hard, we empirically show that the approximate online algorithm presented in the thesis produces summaries with cost that is statistically indistinguishable from batch summaries, while running on natural datasets. Cost was measured as the distance of the farthest sample from a sample in the summary. Collecting data from an environment to build a topic model or a summary requires a robot to traverse this environment. If the geographic size of this region of interest is small then we can simply use any space filling curve to plan this path. However, for larger areas this might not be possible, and hence we propose an information theoretic exploration technique which biases the path towards locations with high information gain in topic space. The resulting topic models were empirically shown to perform better than topic models learned with other competing exploration algorithms, such as free space exploration. Performance was measured in terms of mutual information with ground truth labels, and mutual information with topic labels computed in batch mode with complete knowledge of the environment. Many exploration robots are required to collect samples and perform some chemical or physical analysis. Often such a task requires making irrevocable decisions on whether to select the current sample or not. This thesis presents a novel formulation of this task as an instance of the secretaries hiring problem. We examine several existing variants of this problem, and present an optimal solution to a new variant of the secretaries hiring problem, where the goal is to maximize the probability of identifying the top K samples online and irrevocably. Together, the contributions of this thesis are a step towards developing fully autonomous robotic agents that can be used in collaboration with humans to explore dangerous unknown environments.
@phdthesis{Girdhar2014,
title = {Unsupervised {Semantic} {Perception}, {Summarization}, and {Autonomous} {Exploration} for {Robots} in {Unstructured} {Environments}},
copyright = {All rights reserved},
url = {http://digitool.library.mcgill.ca:80/R/-?func=dbin-jump-full&object_id=129641&silo_library=GEN01},
abstract = {This thesis explores the challenges involved in building autonomous exploration and monitoring systems, and makes contributions on four fronts: describing the semantic context of the collected data, summarizing this information, deciding where to collect this data, and making optimal online irrevocable decisions for physical sample collection. Making high level decisions based on the environmental context of a robot's location requires that we first describe what is being observed in a semantic space with higher level of abstraction than the low level sensor reading. ROST, a realtime online spatiotemporal topic modeling technique that we develop in this thesis solves the problem of obtaining such high level descriptors. Topics in this case represent the latent causes (such as objects and terrains), which produce these observations. ROST extends previous work on topic modeling by efficiently taking into account the spatiotemporal context of an observation, and using a novel Gibbs sampling technique to refine the topic label assignment in realtime, making it suitable for processing streaming sensor data such as video and audio observed by a robot. Our experiments suggest that taking into account the spatiotemporal context of observations results in better topic labels that have higher mutual information with ground truth labels, compared to topic modeling without taking into account the spatiotemporal context. Moreover we show that the perplexity of the online topic model using the proposed Gibbs sampler is competitive with batch Gibbs sampler. Given a scene descriptor such as bag-of-words, location, or topic distribution, the thesis then proposes a novel online summarization algorithm, which unlike previous techniques focuses on building a navigation summary containing all the surprising scenes observed by the robot. We argue that the summaries produced by the algorithm (called extremum summaries) are ideal for monitoring and inspections tasks, where the goal is to maintain a small set of images that is representative of the diversity of what has been observed. Although computation of an optimal summary, even in the batch case is NP-hard, we empirically show that the approximate online algorithm presented in the thesis produces summaries with cost that is statistically indistinguishable from batch summaries, while running on natural datasets. Cost was measured as the distance of the farthest sample from a sample in the summary. Collecting data from an environment to build a topic model or a summary requires a robot to traverse this environment. If the geographic size of this region of interest is small then we can simply use any space filling curve to plan this path. However, for larger areas this might not be possible, and hence we propose an information theoretic exploration technique which biases the path towards locations with high information gain in topic space. The resulting topic models were empirically shown to perform better than topic models learned with other competing exploration algorithms, such as free space exploration. Performance was measured in terms of mutual information with ground truth labels, and mutual information with topic labels computed in batch mode with complete knowledge of the environment. Many exploration robots are required to collect samples and perform some chemical or physical analysis. Often such a task requires making irrevocable decisions on whether to select the current sample or not. This thesis presents a novel formulation of this task as an instance of the secretaries hiring problem. We examine several existing variants of this problem, and present an optimal solution to a new variant of the secretaries hiring problem, where the goal is to maximize the probability of identifying the top K samples online and irrevocably. Together, the contributions of this thesis are a step towards developing fully autonomous robotic agents that can be used in collaboration with humans to explore dangerous unknown environments.},
school = {McGill University},
author = {Girdhar, Yogesh},
year = {2014},
}
Downloads: 0
{"_id":"PdNiHy6tYaqQw4C62","bibbaseid":"girdhar-unsupervisedsemanticperceptionsummarizationandautonomousexplorationforrobotsinunstructuredenvironments-2014","downloads":0,"creationDate":"2016-03-07T00:57:28.874Z","title":"Unsupervised Semantic Perception, Summarization, and Autonomous Exploration for Robots in Unstructured Environments","author_short":["Girdhar, Y."],"year":2014,"bibtype":"phdthesis","biburl":"https://bibbase.org/zotero-mypublications/girdhar","bibdata":{"bibtype":"phdthesis","type":"phdthesis","title":"Unsupervised Semantic Perception, Summarization, and Autonomous Exploration for Robots in Unstructured Environments","copyright":"All rights reserved","url":"http://digitool.library.mcgill.ca:80/R/-?func=dbin-jump-full&object_id=129641&silo_library=GEN01","abstract":"This thesis explores the challenges involved in building autonomous exploration and monitoring systems, and makes contributions on four fronts: describing the semantic context of the collected data, summarizing this information, deciding where to collect this data, and making optimal online irrevocable decisions for physical sample collection. Making high level decisions based on the environmental context of a robot's location requires that we first describe what is being observed in a semantic space with higher level of abstraction than the low level sensor reading. ROST, a realtime online spatiotemporal topic modeling technique that we develop in this thesis solves the problem of obtaining such high level descriptors. Topics in this case represent the latent causes (such as objects and terrains), which produce these observations. ROST extends previous work on topic modeling by efficiently taking into account the spatiotemporal context of an observation, and using a novel Gibbs sampling technique to refine the topic label assignment in realtime, making it suitable for processing streaming sensor data such as video and audio observed by a robot. Our experiments suggest that taking into account the spatiotemporal context of observations results in better topic labels that have higher mutual information with ground truth labels, compared to topic modeling without taking into account the spatiotemporal context. Moreover we show that the perplexity of the online topic model using the proposed Gibbs sampler is competitive with batch Gibbs sampler. Given a scene descriptor such as bag-of-words, location, or topic distribution, the thesis then proposes a novel online summarization algorithm, which unlike previous techniques focuses on building a navigation summary containing all the surprising scenes observed by the robot. We argue that the summaries produced by the algorithm (called extremum summaries) are ideal for monitoring and inspections tasks, where the goal is to maintain a small set of images that is representative of the diversity of what has been observed. Although computation of an optimal summary, even in the batch case is NP-hard, we empirically show that the approximate online algorithm presented in the thesis produces summaries with cost that is statistically indistinguishable from batch summaries, while running on natural datasets. Cost was measured as the distance of the farthest sample from a sample in the summary. Collecting data from an environment to build a topic model or a summary requires a robot to traverse this environment. If the geographic size of this region of interest is small then we can simply use any space filling curve to plan this path. However, for larger areas this might not be possible, and hence we propose an information theoretic exploration technique which biases the path towards locations with high information gain in topic space. The resulting topic models were empirically shown to perform better than topic models learned with other competing exploration algorithms, such as free space exploration. Performance was measured in terms of mutual information with ground truth labels, and mutual information with topic labels computed in batch mode with complete knowledge of the environment. Many exploration robots are required to collect samples and perform some chemical or physical analysis. Often such a task requires making irrevocable decisions on whether to select the current sample or not. This thesis presents a novel formulation of this task as an instance of the secretaries hiring problem. We examine several existing variants of this problem, and present an optimal solution to a new variant of the secretaries hiring problem, where the goal is to maximize the probability of identifying the top K samples online and irrevocably. Together, the contributions of this thesis are a step towards developing fully autonomous robotic agents that can be used in collaboration with humans to explore dangerous unknown environments.","school":"McGill University","author":[{"propositions":[],"lastnames":["Girdhar"],"firstnames":["Yogesh"],"suffixes":[]}],"year":"2014","bibtex":"@phdthesis{Girdhar2014,\n\ttitle = {Unsupervised {Semantic} {Perception}, {Summarization}, and {Autonomous} {Exploration} for {Robots} in {Unstructured} {Environments}},\n\tcopyright = {All rights reserved},\n\turl = {http://digitool.library.mcgill.ca:80/R/-?func=dbin-jump-full&object_id=129641&silo_library=GEN01},\n\tabstract = {This thesis explores the challenges involved in building autonomous exploration and monitoring systems, and makes contributions on four fronts: describing the semantic context of the collected data, summarizing this information, deciding where to collect this data, and making optimal online irrevocable decisions for physical sample collection. Making high level decisions based on the environmental context of a robot's location requires that we first describe what is being observed in a semantic space with higher level of abstraction than the low level sensor reading. ROST, a realtime online spatiotemporal topic modeling technique that we develop in this thesis solves the problem of obtaining such high level descriptors. Topics in this case represent the latent causes (such as objects and terrains), which produce these observations. ROST extends previous work on topic modeling by efficiently taking into account the spatiotemporal context of an observation, and using a novel Gibbs sampling technique to refine the topic label assignment in realtime, making it suitable for processing streaming sensor data such as video and audio observed by a robot. Our experiments suggest that taking into account the spatiotemporal context of observations results in better topic labels that have higher mutual information with ground truth labels, compared to topic modeling without taking into account the spatiotemporal context. Moreover we show that the perplexity of the online topic model using the proposed Gibbs sampler is competitive with batch Gibbs sampler. Given a scene descriptor such as bag-of-words, location, or topic distribution, the thesis then proposes a novel online summarization algorithm, which unlike previous techniques focuses on building a navigation summary containing all the surprising scenes observed by the robot. We argue that the summaries produced by the algorithm (called extremum summaries) are ideal for monitoring and inspections tasks, where the goal is to maintain a small set of images that is representative of the diversity of what has been observed. Although computation of an optimal summary, even in the batch case is NP-hard, we empirically show that the approximate online algorithm presented in the thesis produces summaries with cost that is statistically indistinguishable from batch summaries, while running on natural datasets. Cost was measured as the distance of the farthest sample from a sample in the summary. Collecting data from an environment to build a topic model or a summary requires a robot to traverse this environment. If the geographic size of this region of interest is small then we can simply use any space filling curve to plan this path. However, for larger areas this might not be possible, and hence we propose an information theoretic exploration technique which biases the path towards locations with high information gain in topic space. The resulting topic models were empirically shown to perform better than topic models learned with other competing exploration algorithms, such as free space exploration. Performance was measured in terms of mutual information with ground truth labels, and mutual information with topic labels computed in batch mode with complete knowledge of the environment. Many exploration robots are required to collect samples and perform some chemical or physical analysis. Often such a task requires making irrevocable decisions on whether to select the current sample or not. This thesis presents a novel formulation of this task as an instance of the secretaries hiring problem. We examine several existing variants of this problem, and present an optimal solution to a new variant of the secretaries hiring problem, where the goal is to maximize the probability of identifying the top K samples online and irrevocably. Together, the contributions of this thesis are a step towards developing fully autonomous robotic agents that can be used in collaboration with humans to explore dangerous unknown environments.},\n\tschool = {McGill University},\n\tauthor = {Girdhar, Yogesh},\n\tyear = {2014},\n}\n\n","author_short":["Girdhar, Y."],"key":"Girdhar2014","id":"Girdhar2014","bibbaseid":"girdhar-unsupervisedsemanticperceptionsummarizationandautonomousexplorationforrobotsinunstructuredenvironments-2014","role":"author","urls":{"Paper":"http://digitool.library.mcgill.ca:80/R/-?func=dbin-jump-full&object_id=129641&silo_library=GEN01"},"metadata":{"authorlinks":{"girdhar, y":"https://www2.whoi.edu/"}},"downloads":0},"search_terms":["unsupervised","semantic","perception","summarization","autonomous","exploration","robots","unstructured","environments","girdhar"],"keywords":[],"authorIDs":["KMWsQiocBDitEciM8"],"dataSources":["KqS8yj4KdK5TgR3vo","ya2CyA73rpZseyrZ8","Aba7SDZ3i2zffh3rY","XFrKPG99s5t3W7xuW"]}