Embodied AI systems, particularly humanoid robots deployed in real world scenarios require whole-body control policies that are both task-responsive and physically smooth. However, smoothness is not uniform across the body: lower body must remain sufficiently reactive, while the upper body must be tightly regulated to preserve stability.
Existing reinforcement learning approaches typically impose smoothness through auxiliary terms in the reward function, which compete with task objectives and provide no direct control over the physical quantities responsible for smooth behavior.
We introduce DeCap (Decoupled Constraint-aware policy), a constrained reinforcement learning algorithm that formulates smoothness as explicit constraints on physical motion limits, with separate constraint groups for the upper and lower body. To improve constraint satisfaction near feasibility boundaries, DeCap incorporates a bounded barrier penalty that activates proactively as limits are approached while maintaining finite gradients for stable optimization.
On real-world humanoid locomotion task, DeCap reduces upper-body action rate by 2.50x and acceleration by 2.18x relative to reward-based smoothness policies, while also improving lower-body smoothness and reducing transient motion. We demonstrate that a fixed set of smoothness constraints transfers across four diverse terrains, alleviating the need of extensive reward tuning.