Quick Facts

Organizers:	Jan Peters, Marc Toussaint
Conference:	NIPS 2007
Date:	December 7, 2007
Room:	Hilton: Black Tusk (90)
Location:	Westin Resort and Spa and Westin Hilton, Whistler, B.C., Canada
Website:	http://www.robot-learning.de
Program:	Official program as PDF
Abstracts:	All abstracts as PDF

Abstract

Creating autonomous robots that can assist humans in situations of daily life is a great challenge for machine learning. While this aim has been a long standing vision of robotics, artificial intelligence, and the cognitive sciences, we have yet to achieve the first step of creating robots that can accomplish a multitude of different tasks, triggered by environmental context or higher level instruction. Despite the wide range of machine learning problems encountered in robotics, the main bottleneck towards this goal has been a lack of interaction between the core robotics and the machine learning communities. To date, many roboticists still discard machine learning approaches as generally inapplicable or inferior to classical, hand-crafted solutions. Similarly, machine learning researchers do not yet acknowledge that robotics can play the same role for machine learning which for instance physics had for mathematics: as a major application as well as a driving force for new ideas, algorithms and approaches.

Some fundamental problems we encounter in robotics that equally inspire current research directions in Machine Learning are:

learning and handling models, (e.g., of robots, task or environments)
learning deep hierarchies or levels of representations (e.g., from sensor & motor representations to task abstractions)
regression in very high-dimensional spaces for model and policy learning
finding low-dimensional embeddings of movement as an implicit generative model
methods for probabilistic inference of task parameters from vision,e.g., 3D geometry of manipulated objects
the integration of multi-modal information (e.g., proprioceptive, tactile, vison) for state estimation and causal inference
probabilistic inference in non-linear, non-Gaussian stochastic systems (e.g., for planning as well as optimal or adaptive control)

Robotics challenges can inspire and motivate new Machine Learning research as well as being an interesting field of application of standard ML techniques.

Inversely, with the current rise of real, physical humanoid robots in robotics research labs around the globe, the need for machine learning in robotics has grown significantly. Only if machine learning can succeed at making robots fully adaptive, it is likely that we will be able to take real robots out of the research labs into real, human inhabited environments. To do so, we future robots will need to be able to make proper use of perceptual stimuli such as vision, proprioceptive & tactile feedback and translate these into motor commands.

To close this complex loop, machine learning will be needed on various stages ranging from sensory-based action determination over high-level plan generation to motor control on torque level. Among the important problems hidden in these steps are problems which can be understood from the robotics and the machine learning point of view including perceptuo-action coupling, imitation learning, movement decomposition, probabilistic planning problems, motor primitive learning, reinforcement learning, model learning and motor control.

Format

The goal of this workshop is to bring together people that are interested in robotics as a source and inspiration for new Machine Learning challenges, or which work on Machine Learning methods as a new approach to robotics challenges. In the robotics context, among the questions which we intend to tackle are

Reinforcement Learning, Imitation, and Active Learning:

What methods from reinforcement learning scale into the domain of robotics?
How can we improve our policies acquired through imitation by trial and error?
Can we turn many simple learned demonstrations into proper policies?
Does the knowledge of the cost function of the teacher help the student?
Can statistical methods help for generating actions which actively influencing our perception? E.g., Can these be used to plan visuo-motor sequences that will minimize our uncertainty about the scene?
How can image understanding methods be extended to provide probabilistic scene descriptions suitable for motor planning?

Motor Representations and Control:

Can we decompose human demonstations into elemental movements, e.g., motor primitives, and learn these efficiently?
Is it possible to build libraries of basic movements from demonstration? How to create higher-level structured representations and abstractions based on elemental movements?
Can structured (e.g., hierarchical) temporal stochastic models be used to plan the sequencing and superposition of movement primitives?
Is probabilistic inference the road towards composing complex action sequences from simple demonstrations? Are superpositions of motor primitives and the coupling in timing between these learnable?
How to generate compliant controls for executing complex movement plans which include both superposition and hierachies of elemental movements? Can we find learned versions of priortized hierachical control?
Can we learn how to control in task-space of redundant robots in the presence of underactuation and complex constraints? Can we learn force or hybrid control in task-space?
Is real-time model learning the way to cope with executing tasks on robots with unmodeled nonlinearities and manipulating uncertain objects in unpredictable environmental interactions?
What new regression techniques can help real-time model learning to improve the execution of tasks on robots with unmodeled nonlinearities and manipulating uncertain objects in unpredictable environmental interactions?

Learning structured models and representations:

What kind of probabilistic models provide a compact and suitable description of real-world environments composed of manipulable objects?
How can abstractions or compact representations be learnt from sensori-motor data?
How can we extract features of the sensori-motor data that are relevant for motor control or decision making? E.g., can we extract visual features of objects directly related to their manipulability or 'affordance'?

Preliminary Program

Morning session: 7:30am�10:30am
7:30am	Welcome, Jan Peters, Max Planck Institute, Marc Toussaint, Technical University of Berlin
7:35am	Learning Nonparametric Policies by Imitation, David Grimes and Rajesh Rao, University of Washington
8:05am	Machine learning for developmental robotics, Manuel Lopes, Luis Montesano, Francisco Melo, Instituto Superior Tecnico
8:15am	Machine Learning Application to Robotics and Human-Robot Interaction, Aude Billard, EPFL

8:45am	coffee break

9:00am	Poster Spotlights

9:20am	Bayesian Reinforcement Learning in Continuous POMDPs with Application to Robot Navigation, Stephane Ross, Joelle Pineau, McGill University
9:50am	Self-Supervised Learning from High-Dimensional Data for Autonomous Offroad Driving, Ayse Naz Erkan, Raia Hadsell, Pierre Sermanet, Koray Kavukcuoglu, Marc-Aurelio Ranzato, Urs Muller, Yann LeCun, NYU
10:00am	Task-based motion primitives for the control and analysis of anthropomorphic systems, Luis Sentis, Stanford University

Skiing Break

Afternoon session: 3:30pm�6:30pm
3:30am	STAIR: The STanford Artificial Intelligence Robot project, Andrew Ng, Stanford University
4:00am	Robot Perception Challenges for Machine Learning, Chieh-Chih Wang, National Taiwan University
4:10am	Probabilistic inference methods for nonlinear, non-Gaussian, hybrid control, Nando de Freitas, University of British Columbia

4:40pm	coffee break

5:00am	A new mathematical framework for optimal choice of actions, Emo Todorov, UCSD

5:30pm	Poster Spotlights
5:50pm	Poster Session

Accepted Posters

The conditioning effect of stochastic dynamics in continuous reinforcement learning, Yuval Tassa, Hebrew University, Jerusalem [pdf]
Learned system dynamics for adaptive optimal feedback control, Djordje Mitrovic, Stefan Klanke, and Sethu Vijayakumar (U.Edinburgh) [pdf]
Reinforcement Learning with Multiple Demonstrations, Adam Coates, Pieter Abbeel, Andrew Y. Ng, Stanford University [pdf]
Policy gradient approach to multi-robot learning, Francisco Melo, Instituto Superior T�cnico [pdf]
Improving Gradient Estimation by Incorporating Sensor Data, Gregory Lawrence, U.C. Berkeley [pdf]
Learning Robot Control Policies, Daniel H Grollman, Odest Chadwicke Jenkins [pdf]
A Step Towards Autonomy in Robotics via Reservoir Computing, E. Antonelo, X. Dutoit, B. Schrauwen, D. Stroobandt, H. Van Brussel, M. Nuttin, KU Leuven [pdf]
Towards Active Learning for Socially Assistive Robots, Adriana Tapus, Maja Mataric, USC [pdf]
Relocatable Action Models for Autonomous Navigation, Bethany R. Leffler Michael L. Littman, Rutgers University [pdf]
Learning to Associate with CRF-Matching, Fabio Ramos, Bertrand Douillard, Dieter Fox, U. Washington [pdf]
TORO: Tracking and Observing Robot, Deepak Ramachandran, Rakesh Gupta, Honda Research Institute [pdf]
Maximum Entropy Inverse Reinforcement Learning, Brian D. Ziebart, J. Andrew Bagnell, Anind K. Dey, CMU [pdf]
Learning 3-D Object Orientation from Images, Ashutosh Saxena, Justin Driemeyer, Andrew Y. Ng, Stanford University [pdf]
Bayesian Nonparametric Regression with Local Models, Jo-Anne Ting, Stefan Schaal, USC [pdf]
Active Learning for Robot Control, Philipp Robbel (MIT), Sethu Vijayakumar (University of Edinburgh), Marc Toussaint (TUB) [pdf]
Learning Robot Low Level Control Primitives: A Case Study, Diego Pardo, Cecilio Angulo, Ricardo Tellez,Technical University of Catalunya [pdf]
Efficient Sample Reuse by Covariate Shift Adaptation in Value Function Approximation, Hirotaka Hachiya, Takayuki Akiyama, Masashi Sugiyama [pdf]
Tekkotsu as a Framework for Robot Learning Research,David S. Touretzky, Ethan J. Tira�Thompson, CMU [pdf]

Please note that these posters were selected out of 42 poster submissions to this workshop using a rigorous reviewing process. We apologize that we did not have more space for posters as there were many other great submissions.

Poster Size Recommendation

We recommend a poster size of 3' x 4'.

Participants

This workshop will bring together researchers from both the robotics and machine learning in order to explore how to approach the topic of solving the current statistical learning challenges in robotics in a principled way. Participants of the workshop (inclusive of the audience) are encouraged to actively participate by responding with questions and comments about the talks and give stand-up talks. Please contact the organizers if you would like to reserve apriori some time for expressing your view on a particular topic.

Organizers

The workshop is organized by Marc Toussaint, Technical University of Berlin, Germany, and by Jan Peters, Max-Plank Institute for Biological Cybernetics, Tuebingen, Germany.

Location and More Information

The most up-to-date information about NIPS 2007 can be found on the NIPS 2007 website.