Efficient Learning Control via Structural Policy Priors, Hierarchical Abstraction, and Latent World Models
Speaker
Tim Seyde
CSAIL MIT
Host
Daniela Rus
CSAIL MIT
Zoom Link: https://mit.zoom.us/j/2994010277
Abstract: Learning control will enable deployment of autonomous robots in unstructured real-world settings. Solving the associated complex decision processes under real-time constraints will require intuition, guiding current actions by prior experience to anticipate long-horizon environment interactions and integrating with optimal control to ground action selection in short-horizon system constraints. Ensuring tractability of the underlying learning process is conditional upon maximizing task-aligned information extracted from environment interactions, while minimizing the required guidance via human interventions. In this thesis, we develop novel learning control algorithms that enable efficient acquisition of complex behaviors while limiting prior knowledge, direct human supervision, and computational requirements. Our study particularly focuses on learning from interaction through reinforcement learning, combining insights from model-free, model-based, as well as hierarchical techniques. We design decoupled discrete policy structures to yield memory-efficient agent representations, leverage hierarchical abstraction over diverse behavior components to enable time-efficient optimization, and build latent world models for multi-step prediction and sample-efficient interaction selection. In sum, this thesis develops scalable and efficient robot learning algorithms by addressing representational challenges across layers of abstractions, providing agents with an intrinsic ability to set implicit exploration goals under high-level guidance, and facilitating information propagation in limited data regimes.
Thesis committee:
Daniela Rus, Sertac Karaman, Pulkit Agrawal
Abstract: Learning control will enable deployment of autonomous robots in unstructured real-world settings. Solving the associated complex decision processes under real-time constraints will require intuition, guiding current actions by prior experience to anticipate long-horizon environment interactions and integrating with optimal control to ground action selection in short-horizon system constraints. Ensuring tractability of the underlying learning process is conditional upon maximizing task-aligned information extracted from environment interactions, while minimizing the required guidance via human interventions. In this thesis, we develop novel learning control algorithms that enable efficient acquisition of complex behaviors while limiting prior knowledge, direct human supervision, and computational requirements. Our study particularly focuses on learning from interaction through reinforcement learning, combining insights from model-free, model-based, as well as hierarchical techniques. We design decoupled discrete policy structures to yield memory-efficient agent representations, leverage hierarchical abstraction over diverse behavior components to enable time-efficient optimization, and build latent world models for multi-step prediction and sample-efficient interaction selection. In sum, this thesis develops scalable and efficient robot learning algorithms by addressing representational challenges across layers of abstractions, providing agents with an intrinsic ability to set implicit exploration goals under high-level guidance, and facilitating information propagation in limited data regimes.
Thesis committee:
Daniela Rus, Sertac Karaman, Pulkit Agrawal