Top / Research

Lucia: A healthcare robot that performs mobile patrols, wandering detection notification, telepresence for the disabled, physiological measurements, and gait measurement. Spina: A modular robot with an engineered spinal structure. Each module cooperates with each other via CAN communication to perform cooperative and compensatory actions. Mani: A robot that brings food and drinks to the mouth and hand. Mani uses cutlery to deliver food and drinks, and determines the shape and volume of the food to be served. Cibo: A system that recognizes eating behavior and dining conditions. Detects mastication and prevents aspiration by voice prompting. Cibo provides voice guidance to visually impaired people. Camina: A system that measures the gait test of physiotherapy and evaluates walking movements. It visualizes, judges, and predicts gait by categorizing elemental movements. Memoria: The system maintains memories by answering quizzes generated from photos and impressions of daily events. It enhances the user's vitality by foreshadowing future events. Occhi: A gazing training system that helps to restore visual acuity and prevent depth perception loss due to eye strain. Stretches the eye muscles by moving the gaze. Parla: A system that provides explanations and responses to questions from presentation materials containing explanatory text with guaranteed audiovisual information. It generates explanatory videos. Orale: An interface to operate devices using the tongue. It measures the state of the tongue as an optical image. The tongue can be manipulated delicately by using the pseudo-central fossa. Mano: A concentric chain joint and a push-pull cable transmit power to assist finger flexion and extension. It can be used for mirror therapy.

---

Studies on Human Machine Symbiosis

---

• Care and Medical Support Robotics
• Body Extension Interface
• Machine Learning and Cognitive Development Systems
• Sensory-Motor Integration and Kansei Informatics

---

Nursing Robots and Environmental Sensor Networks

---

We are developing a mobile robot Terapio that supports medical doctors while doing the rounds in hospitals. Terapio has a function to deliver medical tools and information by autonomous tracking and power assisted locomotion. The robot is able to provide access to clinical information and audio-video data stored in previous treatments and recode new information on-line while giving medical cares. We are developing environmental sensor networks that accommodates total nursing with Terapio. The network system connects mobile nursing robots and sensors embedded in the environment. The system then recognizes activities of patients and caregivers to support them with nursing robots. A data server manages sensor information of the position, motion, and identity of people and materials as well as sensory-motor information of nursing robots in order to coordinate between people, robot and environment.
(Contributers: Terashima, K., Tasaki, R., Miura, J., Kitazaki, M.)

---

Developmental action perception and sensory-motor integration

---

We propose a cognitive system that allows an anthropomorphic robot to observe object operation performed by humans and reproduce the similar operation. The robot characterizes a self-generated action based on the action effect for objects. This action coding enables the robot to identify actions that share the common goal but performed by different agents (e.g. the self and other). Moreover, the system allows cross-sensory association that can reconstruct sense of touch from visual observation. The system does not require either body kinematics nor sensory-motor configuration for this learning, since the robot autonomously differentiates the own body from the environment and then learns primitive actions of object operation through stochastic motor exploration.
(Contributers: Saegusa, R., Natale, L., Metta, G., Sandini, G.)

Body schema development based on visuo-motor correlation

This work proposes a plausible approach for a humanoid robot to define its own body based on visuomotor correlation. The high correlation of motion between vision and proprioception informs the robot that a visually moving object is related to the motor function of its own body. When the robot finds a motor-correlated object during motor exploration, visuomotor cues such as body posture and the visual features of the object are stored in visuomotor memory. Then, the robot developmentally defines its own body without prior knowledge on body appearances and kinematics. Body definition is also adaptable for an extended body such as a tool that the robot is grasping. The body movements are generated in the manner of stochastic motor babbling, whereas visuomotor memory biases the babbling to keep the body parts in sight. This ego-attracted bias helps the robot explore the joint space more efficiently. After motor exploration, visuomotor memory allows the robot to anticipate a visual image of its own body from a motor command. The proposed approach was experimentally evaluated with humanoid robot iCub.
(Contributers: Saegusa, R., Metta, G., Sandini, G.)

---

Self-protective whole body movements and motor instruction

---

We developed an intelligence model that allows a humanoid robot to protect the self by whole body movements that the robot learned. The model consolidates local motor reflexes driven by force sensing with global adaptive reactions based on visual and vestibular stimuli. The robot therefore reduces physical load from the environment. We think that these self-protective behaviors are an important factor to realize robot's long-term motor learning. We are extending the model to allow an humanoid robot to learn whole body movements by humans' kinesthetic teaching and its reconstruction. We first teach the robot movements (e.g. walking and sitting-up) with supporting the robot. The robot extracts motor features, and use them in reinforcement learning of the movements without the support. The recycle of encoded motor knowledge enhances the learning speed in the different condition.
(Contributers: Shimizu, T., Saegusa, R., Ikemoto, S., Ishiguro, H., Metta, G.)

---

Rhythm associator to create music with humans and drawing effect

---

We developed a rhythm associating system. We first demonstrate an ensemble of rhythm performances for the system. The system then extracts couples of rhythm sequences and trains the rhythm associator. The similarity of rhythms is computed based on the hierarchical analysis, that provides the natural association in termes of human sense. After the leaning, the system plays rhythm sequences by listening the human rhythm performance. The behavior of the system is original in accepting the human player's suggestion to change the rhythm, and proposing a more suitable rhythm to the human player. This coordination create an music ensemble in which both the man and system can take leadership.
(Contributers: Kasahara, S., Saegusa, R., Hashimoto, S.)

---

Nonlinear data compression and pattern recognition

---

We developed a neuromorphic model of nonlinear principal component analysis that compresses high-dimensional data. Principal component analysis (PCA) is a standard linear method in data analysis. We proposed nonlinearization of the PCA. Non-parametric nonliear functions that play the role of data encoding or decoding are constructed with a hierarchical constrain. The hierarchically structured functions represent the data efficiently with less principal components than the linear PCA. We evaluated efficiency of data compression and decompression with large sets of image data. This approach is applicable for subspace-based pattern recognition. The set of functions forms manifolds (nonlinear principal axes) in the data space, and the manifolds totally represent subspace of a categorical data.
(Contributers: Saegusa, R., Sakano, H.,　Hashimoto, S.)