Lifestyle Support Robot
  Human pose estimation in unusual postures

Human pose is one of the important information for estimating human states. If the robot knows the pose, it can, for example, examine head and face regions for measuring vital signs like breathing. We have developed depth data-based pose estimation methods for usual and unusual poses. One of the method uses a deep neural network and a huge dataset of synthetically-generated human depth data with body part labels annotation.

[References]

  • K. Nishi and J. Miura, "A Head Position Estimation Method for a Variety of Recumbent Positions for a Care Robot", Proc. 2015 Int. Conf. on Advanced Mechatronics (ICAM-2015), Tokyo, Japan, Dec. 2015.
  • K. Nishi, M. Demura, J. Miura and S. Oishi, "Use of Thermal Point Cloud for Thermal Comfort Measurement and Human Pose Estimation in Robotic Monitoring", Proc. 5th Int. Workshop on Assistive Computer Vision and Robotics (ACVR 2017), Venice, Italy, Oct. 2017.
  • K. Nishi and J. Miura, "Generation of Human Depth Images with Body Part Labels for Complex Human Pose Recognition", Pattern Recognition, Vol. 71, pp. 402-413, 2017.

  Physical state monitoring of residence for lifestyle support

Keeping the physical states such as temperature, humidity, and lighting of a residence comfortable is very important for lifestyle support. We have been developing a robot system that automatically moving around to monitor the physical states. Based on the monitored states and the state of the resident, the robot will take appropriate actions like controlling appliances and/or issuing an alert.

[References]

  • S. Kani and J. Miura, "Mobile Monitoring of Physical States of Indoor Environments for Personal Support", Proc. 2015 IEEE/SICE Int. Symp. on System Integration (SII-2015), pp. 393-398, Nagoya, Japan, Dec. 2015.

  Thermal comfort estimation by a mobile robot

Estimating thermal comfort of a residence is important. A thermal comfort measure, PMV (predicted mean vote), can be calculated by six major factors, four of which are about the physical state and the rest are about the human state. We have developed a method of estimating one of the human states, that is, the clothing insulation (the amount of clothing in terms of thermal insulation), or clo-value, using thermal images and image-based human recognition techniques. We developed an experimental system to estimate PMV of indoor environments.

[References]

  • K. Nishi, M. Demura, J. Miura and S. Oishi, "Use of Thermal Point Cloud for Thermal Comfort Measurement and Human Pose Estimation in Robotic Monitoring", Proc. 5th Int. Workshop on Assistive Computer Vision and Robotics (ACVR 2017), Venice, Italy, Oct. 2017.