Action Contextualization

The question

The same action may need to be performed differently depending on the task. Handling a full container, for example, introduces contextual requirements that a generic action label does not capture.

The approach

A large language model reasons over an explicit action vocabulary and qualitative and quantitative behaviour parameters. Motion metrics and structured execution checks provide feedback for parameter retuning or replanning. Underlying dynamical-system controllers handle reactive low-level execution.

Context–behaviour interface

Results & evidence

• Context-sensitive action planning and parameter adaptation over a predefined repertoire of robot behaviours.
• Robot arm–hand execution of sequential table-clearing tasks with reactive low-level control.
• Structured feedback that can trigger action retuning or task replanning.

Scope & assumptions

The framework depends on the available action vocabulary, behaviour parameterisation, execution checks and the language model’s interpretation. Controller convergence properties do not constitute a guarantee that every generated plan will succeed.

Related publications

@article{gupta2024contextualization,
  title = {Action Contextualization: Adaptive Task Planning and Action Tuning Using Large Language Models},
  author = {Sthithpragya Gupta and Kunpeng Yao and Loïc Niederhauser and Aude Billard},
  journal = {IEEE Robotics and Automation Letters},
  year = {2024},
  doi = {10.1109/LRA.2024.3460408}
}