Robotics Laboratory - Expertise and Research Topics

The current research topics  in Robotics Laboratory:

A Generalized Approach to Modelling Dynamics of Human and Humanoid Motion

The research in this area suggests a generalized approach to the modeling of human and humanoid motion. Instead of the usual inductive approach that starts from the analysis of different situations of real motion (like bipedal gait and running; playing tennis, soccer, or volleyball; gymnastics on the floor or by using some gymnastic apparatus; etc.) and tries to make a generalization, we suggest a deductive approach: one starts with considering a completely general problem. Once the general model is formulated, one may derive different real situations as being special cases. Such approach needs a serious effort in formulating the general model. The general methodology is explained and the feasibility is supported by applying the general model of some illustrative examples: first, a well-known situation – the single-support phase of a biped motion, and second, a completely different problem – gymnastic exercise on a horizontal bar.

Intelligent Control Techniques in Robotics

Primary aim of the research in this area is the synthesis of intelligent robotic controllers for non-contact and contact tasks, as well as the application in the domain of locomotion robotic mechanisms. The synthesis of new control algorithms based on adaptive learning systems - neural networks is performed. The various forms of neural network training and different topologies of neural networks for efficient realization of robotic contact and non-contact tasks are proposed. Good results are achieved in the application of neural networks for robotic contact tasks where the problem of uncertainties of the robot model and parameters, as well as uncertainties of the dynamic environment is very important. Very interesting and applicable results are obtained using hybrid control method based on combination of conventional control techniques and soft computing paradigms (neural networks, uzzy logics, genetic algorithms). This complementary approach is successfully applied to manipulation robots and locomotion robots. The proposed intelligent techniques which are analyzed for a broader range of contact robotic tasks, show an important advantage in comparison with conventional techniques of robot compliance control.

Wavelet network classifier for  advanced  learning control of robotic contact tasks

Connectionist Reactive Control by Robot Hand with soft grippers

Reinforcement Learning in Humanoid Robotics

Classical humanoid robotics still rely heavily on teleoperation or fixed a priori determined behavior based control  with very little autonomous ability to react to the environment. Key  missing elements is the ability to create control systems that can deal with a  large movement repertoire, variable speeds, constraints and most importantly, uncertainty in the real-world environment in a fast, reactive manner. The acquisition and improvement of motor skills and  control policies for robotics from trial and error is of essential importance if robots should ever leave precisely pre-structured  environments.

 For physical agents, such as humanoid robots acting in the real world, it is much more difficult to gain experience through the process of learning. Robot learning in realistic environments requires novel algorithms for learning to identify important events in the stream of sensory inputs, and to temporarily memorize them in adaptive, dynamic, internal states until the memories can help to compute proper control actions. While supervised statistical learning techniques have significant applications for model and imitation learning, they do not suffice for all biped learning problems, particularly when no expert teacher or idealized desired behavior is available. Since no exact teaching information is available, this is a typical reinforcement learning problem  and the failure signal serves as the reinforcement signal.  Reinforcement learning  offers one of the most general framework to humanoid robotics towards true autonomy and versatility. Humanoid Robotics is a very challenging domain for reinforcement learning, however, since robots cannot perceive the underlying state of their environment  and because training time is usually quite limited.

This research considers a optimal solutions for application of reinforcement learning in humanoid robotics.  The importance of hybrid approach is emphasized. The hybrid aspect is connected with application of model-based and model free approaches as well as with combination of different paradigms of computational intelligence.

The general goal in synthesis of reinforcement learning control algorithms  is the development of methods which scale into the dimensionality of humanoid robots and can generate actions for biped with many degrees of freedom.  In this research, we will consider specially that control of walking of active and passive dynamic walkers by using of reinforcement learning can be efficiently solved .

Dynamic bipedal walking is difficult to learn because  combinatorial explosion in order  to optimize performance in every possible configuration of the robot., uncertainties of the robot dynamics that must be only experimentally validated, and because coping  with dynamic discontinuities caused by collisions with the ground and with the problem of delayed reward -torques applied at one time may have an effect on the performance many steps into the future. Hence, for a physical robot, it is essential to learn from few trials in order to have some time left for exploitation.  It is thus necessary to speed the learning up by using different methods (hierarchical learning, subtask decomposition, imitation,…), that will be presented  

Various straightforward  and hybrid  intelligent control algorithms  based  RL for active and passive biped locomotion is presented.  The proposed reinforcement learning algorithms is based on two different learning structures: actor-critic architecture and  Q-learning structures.  Also, RL algorithms can use numerical and fuzzy evaluative feedback information for external reinforcement. The proposed RL algorithms use the learning elements that consists of various types of neural networks, fuzzy logic nets or fuzzy-neuro networks with focus on fast convergence properties and small number of learning trials.The structure of controller involves two feedback loops: model-based dynamic controller and
fuzzy reinforcement learning feedback around Zero-Moment Point. The reinforcement learning architecture as external
reinforcement use fuzzy evaluative feedback.

Hybrid Control Algorithm for Humanoid Robots Based on Reinforcement Structure

Reinforcement during process of walking

Multi-Arm Cooperating Robots Dynamics and Control

 Several consistent solutions for cooperative system control have recently been identified by the authors of the current monograph. This was achieved by solving three separate tasks that are essential for solving the problem of cooperative manipulation as a whole. The first task is related to the understanding of the physical nature of cooperative manipulation and finding a way for a sufficiently exact characterization of cooperative system statics, kinematics and dynamics. After successfully completing this task, in the frame of the second task, the problem of coordinated motion of the cooperative system is solved. Finally, as a solution to the third task, the control laws of cooperative manipulation are synthesized.

The starting point in dealing with the above three tasks of cooperative manipulation was the assumption that the problem of force uncertainty in cooperative manipulation can be resolved by introducing elastic properties into the cooperative system, at least in the part where force uncertainty appears. In static and dynamic analysis of the elastic structure of cooperative systems the finite element method is applied. In contrast to the procedure used in the major part of the available literature where deformation work is expressed by deviations from the unloaded state of fixed elastic structure, in this monograph the deformation work is expressed by internal forces as a function of the absolute coordinates of contacts of mobile elastic structure. Coordinated motion and control in cooperative manipulation are solved as the problem of coordinated motion and control of a mobile elastic structure, taking into account the specific features of cooperative manipulation. Coordinated motion and control laws in cooperative manipulation are synthesized on the basis of a non-linear model where the problem of uncertainty is solved, which is not the case in the available literature. Simple examples demonstrate the consistent procedure of  mathematical modeling and synthesis of nominal coordinated motion, as well as control of the cooperative system.

Cooperative Robot Dynamics 

Reference ZMP^o  and real ZMP [m]

  Identification of  Anthropomorphic Parameters - Kinematical and dynamic body-segment characteristics

 Quantitative biomechanical analyses of human movement typically require estimation of the body segment parameters (mass, position of the center of mass, principal radii of gyration, moments of inertia). Specifying details of biometry measurements as well as applying methodology of Zatsiorsky-Seluyanov (based on statistics obtained measuring cadavers) improved tabular approach of parameter determination proposed by Paolo De Leva, a specialized software for identification of anthropomorphic parameters was developed. For the known mass and stature of the body and with certain photometry measurements the anthropomorphic parameters with satisfactory high accuracy can be obtained.

Kinematic parameters and mass of links of human body of m=84.07 kg, H=1.90 m  

 Dynamic parameters – inertia tensor and radii of gyration of human body of m=84.07 kg, H=1.90 m  

 A graphic description of the main adjustments to the relative CM positions according Paolo De Leva

used in the parameter identification

Quantitative Biomechanical Analysis of Biped Locomotion – Capture of human motion

 Accurate quantification of kinematics (locomotion) represents a significant requirement for the purpose of physical rehabilitation in medicine, designing of prosthesis in orthopedics, analysis and optimization of sporting disciplines, researches in bio-mechanics, humanoid robotics, etc. Bio-inspired behavior of biped locomotion robots (humanoids) is based on the deep analysis of a human motion. For this purpose a variety of experimental measurements were done in the capture motion studio.  In this research project specialized software for processing of the experimental data of measurements was designed. In the laboratory conditions a set of the extensive experiments were done, measuring capture motion characteristics (using corresponding fluorescent markers and 6-infra red cameras) and ground reaction forces and torques (by  the 6-axis force platform).   

Experimental measurements in the laboratory conditions: (i) fluorescent markers attached to the human body, (ii) Vicon infra-red camera, (iii) 6-sensors force platform

Identified joint angles of human legs during locomotion

 Experimentally measured ground reaction forces and torques at the contact foot

 Bio-Inspired Synthesis of the Artificial Humanoid Motion - Knowledge-based generator of the joint trajectories

  The goal of this project is to generate/determine a desired biped motion, applying learning algorithms (artificial neural network structure), based on the imposed trajectories of the feet and hip joint centers in the reference, Descartes coordinate system of robot motion. Multi-layer artificial neural network structure was used to learn the inverse kinematics of the lower extremities (legs). Simulation results of the open-loop, biped model (used for the rough/initial training) as well as the data of experimental measurements (for the fine net training) were used as the training sets.  Designed knowledge-based algorithm enables real-time generation of the nominal joint trajectories. As a network input set, considering Descartes trajectories of the feet-sole contour points as well as the hip center coordinates determined at the higher strategic control level were used. Developed algorithm for trajectory generation is suitable for the trajectory planning and for planning of the obstacle avoidance at the strategic control level. Designed knowledge-based algorithm ensures bio-inspired synthesis of the artificial motion since it was based on the experimental measurements.      

Coordinates of the feet contour points and hip joint centers (left and right) imposed as the input variables to the knowledge-based generator of  joint trajectories

 A characteristic example of the slalom trajectory of a biped robot with fixed obstacles in its environment 

Advanced Modeling  of Humanoid Robots – Kinematics and dynamics of biped locomotion

 Humanoids, being the future of robotic science, are becoming more and more human-like in all aspects of their functioning. It is expected that they will replace humans in a variety of tasks. Thus, it is generally accepted that their shape and motion should be based on biomechanical principles – to be bio-inspired. Because of the complexity and high requirements imposed on such robots, their control system has to utilize the dynamic model. So, the control, the design, and the simulation, strongly require general dynamic models that will make humanoid robots capable of handling the increasing diversity of expected tasks. As the result of this research/developing project, a user-friendly simulation software of humanoid kinematics and dynamics was developed. Modeling software was realized as a MATLAB/SIMULINK toolbox using all advances of the Matlab engineering interface.  Developed spatial, non-linear model in the present state considers structure of the biped mechanism with 32 d.o.f. It includes full kinematical and dynamic models as well as model of foot impact dynamics in all phases of biped gait: single support, double support and flyer phase (no contact with environment).

 Kinematic scheme of the 32 d.o.f. biped mechanism/system considered in the project  

 Human body for its complex motion uses synergy of more than 600 muscles. It has more then 300 degrees of freedom. Some of these particular motions are essential for the human activities (gait, work, sport, dancing, etc.) while the others give it a full mobility. A biped locomotion mechanism of the anthropomorphic structure was considered as a mechanical representative of a human body. In that sense, an articulated system, consisting of the basic kinematic chain and four side branch chains was considered as a biped mechanism. In a mechanical sense, it represents a multi-body, large-scale dynamic system with a variable structure

   Branched mechanism of a biped locomotion robot, b) Decomposition of the complex mechanism structure into the set of single chains

Developed modeling software calculates: Jacobians, Descartes coordinates of the joint centers in the inertial coordinate system attached to the ground support, corresponding velocities and accelerations of robot joint centers, feet positions/orientations relative to the support, inertia matrix, vector of centrifugal, Coriolis and gravitational forces/torques, contact/impact forces and ground reaction forces/torques, driving torques in robot joints and Zero Moment Point location with respect to the supporting polygon of biped system.

Mathematical  Modeling of Non-Smooth Impact Dynamics of Human Gait – Theory & experiments

 The impact phase starts when the biped link reaches the surrounding object(s). Strictly speaking, the restricted coordinates at the robot foot reach zero one by one. So a complex contact is established as a series of simpler contact effects. The impact begins when the biped foot (feet) reaches the constraint surface. Constraint in a general case can be an ordinary curve, prismatic or flat surface. Accurate modeling of non-smooth, frictional impact dynamics was subject of the comprehensive researches conducted in this project.

Modeling of feet impact dynamics – mechanical set of the problem

 Impact model in this project was considered as a Linear Complementarity Problem (LCP) formulation. Stiction and friction phenomena were included into the model, too.

Functional coordinates of the left and right foot determining the relative positions of the feet contour points with respect to the support in the normal direction

Model-based, ground reaction forces at the right and the left foot during step phases

 Integrated Control  of Humanoid Robots 

Control of humanoid robots has to satisfy the following requirements: (i) accuracy of tracking the desired trajectories of the mechanism joints (ii) maintenance of dynamic balance of the mechanism during the motion, (iii) minimization of the impact arising at the moment of contact of the free foot and the ground during the gait, (iv) minimization of dynamic loads at the robot joints, and (v) realization of anthropomorphic characteristics of the gait. Fulfillment of requirement (i) enables the realization of a desired mode of motion, walk repeatability and avoidance of potential obstacles. To satisfy requirement (ii) it means to maintain balance of the dynamic reactions acting upon the robotic system during the motion. Fulfillment of requirement (iii) decreases the impact effects on the overall system at the moment when the unconstrained leg foot strikes the ground. Fulfillment of requirement (iv) is needed for the purpose of minimizing dynamic loads at the robotic joints, which is especially important for the joints bearing the highest load during the walk, e.g. the hip. Requirement (v) is related to the quality of artificial (human-like) gait. Walk of a physically healthy human represents a balanced and harmonious sequence of movements, with minimal displacements of the position of the mass center about an imaginary central position corresponding to the human’s posture at rest. Bearing in mind the control requirements, it is necessary to control the following quantitites: positions and velocities of the robot joints, ZMP position, contact force at the instant of foot striking the ground, and dynamic load forces at particular mechanism joints. Control system designed in this project involves four feedback loops: (i) position-velocity feedback, (ii) dynamic reaction feedback at the ZMP, (iii) impact-force feedback at the foot of the free (unconstrained) leg, and (iv) load feedback at the mechanism joints. The dynamic controller was realized at two control levels: tactical and executive. 

Block-diagram of the integrated dynamic control of a biped locomotion mechanism with four feedback loops and with executive control block at the lower level

 Advanced Modeling and Simulation of Automotive Systems – Model of chassis, suspensions, tires, steering, power - drivetrain & braking systems

 The specialized automotive engineering software toolbox designed for modeling and simulation of spatial, non-linear, 22 d.o.f. (degrees of freedom) road vehicle dynamics was developed. In mechanical sense, road vehicle represents a complex, distinctly non-linear, dynamic system with a relatively large number of d.o.f., belonging to the class of so called large-scale dynamic systems.  These d.o.f. (independent motion directions), especially those that essentially influence the system's stability and quality of its dynamic behavior, have to be controlled.  The vehicle stability, quality of its dynamic behavior and maneuvering capabilities depend to a great extent on the system structure as well as on performances of its power-steering and power-drivetrain subsystems.  Synthesis of the best control strategy and design of the corresponding driver-assisted system represent delicate problems whose solving requires knowledge of dynamic behavior of entire vehicle dynamics under different conditions of motion (uneven inclined surface, slippery road, wind gust, etc.).  In that sense, mathematical modeling of vehicle dynamics and its tire-road interaction represents starting point in design process, control synthesis, safety analysis, etc.  

Block-scheme of the model structure with particular functional modules

  Intelligent  Control of Vehicle Lateral and Yaw Dynamics– Smart control system to improve vehicle stability

 Advanced hybrid, knowledge-based – model-based control system of road vehicle interactive dynamics was designed in the project. The proposed driver-assisted control system was designed to improve vehicle stability, active safety and handling performances using four wheel steering, active driving/braking and active suspension system. The control scheme was designed based on a centralized, dynamic control approach with a supplementary knowledge-based compensator (connectionist structure) in the feedback loop. The distributed hierarchy control strategy, with two control levels - tactical and operative, was implemented.

Figure 14: Block-scheme of the driver-vehicle system with driver assisted control system in the loop 

Structure of the training process during system identification

Closed Loop Model of Driver-Vehicle System – Fuzzy sets in modeling of driver behavior with automotive systems   

With conventional road vehicles the direction and the speed of motion are adjusted by the human operator using driver-vehicle interface (steering wheel and acceleration/braking pedals). Vehicle follows the desired path depending on driver’s psycho-physical abilities to response to the driving demands.  

 Structure of the hybrid, man-machine - driver-vehicle system 

 Block-diagram of the driver-model implementing fuzzy sets

 Hazard Prevention Control System – Intelligent, active safety, driver-assisted control of automotive systems

 The main role of the Hazard Prevention System (HPC) designed with automotive systems is to predict and to actively control a hazard motion of road vehicles: e.g. to decrease vehicle speed, to stop the car, to improve the steering wheel angle, etc. This is a delicate control task that demands knowledge of vehicle dynamics as well as driver-model. The HPC system consists of two functional modules: (i) the path observer and, (ii) the fuzzy controller. Together, they make a smart control structure, i.e. an active safety control system based on the fuzzy model of a human operator - driver. 

Hazard prevention system within a closed-loop control structure 

Path observer identifies the road geometry along which the vehicle moves. Fuzzy controller represents a fuzzy model of an experienced vehicle driver. Parameters of the fuzzy controller are tuned off-line on the basis of experimental and simulation data. The best fit of the model parameters obtains using simulation results of the different running tests. Main task of the fuzzy controller in the loop is the same as the role of the driver instructor assisting to an inexperienced vehicle operator. The control signals from the fuzzy controller are led to the vehicle actuators. They serve to change the driver manual commands in a direct way using corresponding servo-actuators of steering, traction or braking system. Beside the identification of desired road geometry and control of path tracking the proposed HPC system was designed to have certain additional functions in hazard prevention, cruising control, platooning, automatic parking, etc. With a radar system installed in the car, the HPC is able to control vehicle motion in the presence of obstacles on the road.

Back to Robotics Laboratory Main Page