Query：
学者姓名：左国玉
Refining：
Year
Type
Indexed by
Source
Complex
Co-Author
Language
Clean All
Abstract ：
The positional information of objects is crucial to enable robots to perform grasping and pushing manipulations in clutter. To effectively perform grasping and pushing manipulations, robots need to perceive the position information of objects, including the coordinates and spatial relationship between objects (e.g., proximity, adjacency). The authors propose an end-to-end position-aware deep Q-learning framework to achieve efficient collaborative pushing and grasping in clutter. Specifically, a pair of conjugate pushing and grasping attention modules are proposed to capture the position information of objects and generate high-quality affordance maps of operating positions with features of pushing and grasping operations. In addition, the authors propose an object isolation metric and clutter metric based on instance segmentation to measure the spatial relationships between objects in cluttered environments. To further enhance the perception capacity of position information of the objects, the authors associate the change in the object isolation metric and clutter metric in cluttered environment before and after performing the action with reward function. A series of experiments are carried out in simulation and real-world which indicate that the method improves sample efficiency, task completion rate, grasping success rate and action efficiency compared to state-of-the-art end-to-end methods. Noted that the authors' system can be robustly applied to real-world use and extended to novel objects. Supplementary material is available at .
Keyword ：
deep neural networks deep neural networks deep learning deep learning intelligent robots intelligent robots
Cite：
Copy from the list or Export to your reference management。
| GB/T 7714 | Zhao, Min , Zuo, Guoyu , Yu, Shuangyue et al. Position-aware pushing and grasping synergy with deep reinforcement learning in clutter [J]. | CAAI TRANSACTIONS ON INTELLIGENCE TECHNOLOGY , 2023 . |
| MLA | Zhao, Min et al. "Position-aware pushing and grasping synergy with deep reinforcement learning in clutter" . | CAAI TRANSACTIONS ON INTELLIGENCE TECHNOLOGY (2023) . |
| APA | Zhao, Min , Zuo, Guoyu , Yu, Shuangyue , Gong, Daoxiong , Wang, Zihao , Sie, Ouattara . Position-aware pushing and grasping synergy with deep reinforcement learning in clutter . | CAAI TRANSACTIONS ON INTELLIGENCE TECHNOLOGY , 2023 . |
| Export to | NoteExpress RIS BibTex |
Abstract ：
Learning from visual observations is a significant yet challenging problem in Reinforcement Learning (RL). Two respective problems, representation learning and task learning, need to solve to infer an optimal policy. Some methods have been proposed to utilize data augmentation in reinforcement learning to directly learn from images. Although these methods can improve generation in RL, they are often found to make the task learning unsteady and can even lead to divergence. We investigate the causes of instability and find it is usually rooted in high-variance of Q-functions. In this paper, we propose an easy-to-implement and unified method to solve above-mentioned problems, Data-augmented Reinforcement Learning with Ensemble Exploration and Exploitation (DAR-EEE). Bootstrap ensembles are incorporated into data augmented reinforcement learning and provide uncertainty estimation of both original and augmented states, which can be utilized to stabilize and accelerate the task learning. Specially, a novel strategy called uncertainty-weighted exploitation is designed for rational utilization of transition tuples. Moreover, an efficient exploration method using the highest upper confidence is used to balance exploration and exploitation. Our experimental evaluation demonstrates the improved sample efficiency and final performance of our method on a range of difficult image-based control tasks. Especially, our method has achieved the new state-of-the-art performance on Reacher-easy and Cheetah-run tasks.
Keyword ：
Bootstrap ensembles Bootstrap ensembles Reinforcement learning from images Reinforcement learning from images Robot learning Robot learning Data augmentation Data augmentation
Cite：
Copy from the list or Export to your reference management。
| GB/T 7714 | Zuo, Guoyu , Tian, Zhipeng , Huang, Gao . A stable data-augmented reinforcement learning method with ensemble exploration and exploitation [J]. | APPLIED INTELLIGENCE , 2023 , 53 (21) : 24792-24803 . |
| MLA | Zuo, Guoyu et al. "A stable data-augmented reinforcement learning method with ensemble exploration and exploitation" . | APPLIED INTELLIGENCE 53 . 21 (2023) : 24792-24803 . |
| APA | Zuo, Guoyu , Tian, Zhipeng , Huang, Gao . A stable data-augmented reinforcement learning method with ensemble exploration and exploitation . | APPLIED INTELLIGENCE , 2023 , 53 (21) , 24792-24803 . |
| Export to | NoteExpress RIS BibTex |
Abstract ：
Efficient and collision-free coordination of two robot arms is increasingly needed in various service-oriented robotic applications. This paper proposes a dual arm coordination algorithm to improve the efficiency of coordination by considering both robot's actions and operating sequences of the tasks that need to use two arms to complete complex operations. Teleoperation demonstration is first performed to obtain the robot's human-like motion trajectories, so as to reduce the probability of the collisions between the two arms. The coordination diagram in time domain is then designed to more clearly represent the situations of trajectory collisions and find the collision-free coordination action law. A Coordination Pair Generator (CPG) is designed to reorganize the operating sequences according to the characteristics of input trajectories and the action coordination. The effectiveness and efficiency of our method are verified on the simulation and physical experiments which execute the drug sorting task in nursing homes, respectively, on the ABB YuMi robot model and self-developed robot system. According to the experiment results, the operation time has been reduced by 9% and the collision area has been reduced by 7.5%.
Keyword ：
Dual arm coordination Dual arm coordination teleoperation demonstration teleoperation demonstration coordination diagram coordination diagram
Cite：
Copy from the list or Export to your reference management。
| GB/T 7714 | Zuo, Guoyu , Xu, Zichen , Huang, Gao . Dual Arm Coordination with Coordination Diagram Based on Teleoperation Demonstration [J]. | INTERNATIONAL JOURNAL OF HUMANOID ROBOTICS , 2022 , 19 (04) . |
| MLA | Zuo, Guoyu et al. "Dual Arm Coordination with Coordination Diagram Based on Teleoperation Demonstration" . | INTERNATIONAL JOURNAL OF HUMANOID ROBOTICS 19 . 04 (2022) . |
| APA | Zuo, Guoyu , Xu, Zichen , Huang, Gao . Dual Arm Coordination with Coordination Diagram Based on Teleoperation Demonstration . | INTERNATIONAL JOURNAL OF HUMANOID ROBOTICS , 2022 , 19 (04) . |
| Export to | NoteExpress RIS BibTex |
Abstract ：
Grasping in cluttered scenes is an important issue in robotic manipulation. The cooperation of grasping and pushing actions based on reinforcement learning is an effective means to obtain the target object when it is completely blocked or there is no suitable grasping position around it. When exploring invisible objects, many existing methods depend excessively on model design and redundant grasping actions. We propose a graph-based deep reinforcement learning model to efficiently explore invisible objects and improve the performance for cooperative grasping and pushing tasks. Our model first extracts the state features and then estimates the Q value with different graph Q-Nets according to whether the target object is found. The graph-based Q-learning model contains an encoder, a graph reasoning module and a decoder. The encoder is used to integrate the state features such that the features of one region include those of other regions. The graph reasoning module captures the internal relationships of features between different regions through graph convolution networks. The decoder maps the features transformed by reasoning to the original state features. Our method achieves a 100% success rate in the task of exploring the target object and a success rate of more than 90% in the task of grasping and pushing cooperatively in simulation experiment, which performs better than many existing state-of-the-art methods. Our method is an effective means to help robots obtain completely occluded objects by grasping and pushing cooperation in the cluttered scenes. The verification experiment on the real robot further shows the generalization and practicability of our proposed model.
Keyword ：
Reinforcement learning Reinforcement learning Graph convolution network Graph convolution network Robotic manipulation Robotic manipulation Fully occluded object Fully occluded object Grasping and pushing Grasping and pushing
Cite：
Copy from the list or Export to your reference management。
| GB/T 7714 | Zuo, Guoyu , Tong, Jiayuan , Wang, Zihao et al. A Graph-Based Deep Reinforcement Learning Approach to Grasping Fully Occluded Objects [J]. | COGNITIVE COMPUTATION , 2022 , 15 (1) : 36-49 . |
| MLA | Zuo, Guoyu et al. "A Graph-Based Deep Reinforcement Learning Approach to Grasping Fully Occluded Objects" . | COGNITIVE COMPUTATION 15 . 1 (2022) : 36-49 . |
| APA | Zuo, Guoyu , Tong, Jiayuan , Wang, Zihao , Gong, Daoxiong . A Graph-Based Deep Reinforcement Learning Approach to Grasping Fully Occluded Objects . | COGNITIVE COMPUTATION , 2022 , 15 (1) , 36-49 . |
| Export to | NoteExpress RIS BibTex |
Abstract ：
The ability of robots to perceive position information of objects is important for pushing and grasping tasks in clutter. We propose a Coordinate Attention Grasping and Pushing Network (Ca-GPNet) to learn synergistic grasping and pushing strategies in cluttered environments. We use both fully convolutional networks to predict actions: One predicts the grasping direction and position of the gripper, and the other predicts the initial push position and direction of the gripper. We propose to use the coordinate attention module to capture position information of objects along the horizontal and vertical directions of space and aggregate the features. The attention module extracts long-range interdependencies in one dimension while keeping the information about the location of objects in the another dimension. Then a pair of attention maps that perceive the location and orientation features of objects are generated to reinforce the perception of objects' locations by the network. The grasping success rate of the system is 73.7% and 71.4% of the action efficiency in the simulation. Our approach can be applied to real-world pushing and grasping tasks. Ca-GPNet grasping success rate is 79.2% and action efficiency is 75.3 % in the real world. Compared with baseline methods, our system can rapidly learn the pushing and grasping cooperative policy in a cluttered environment, with higher sample efficiency, grasping success rate and action efficiency.
Keyword ：
coordinate attention coordinate attention clutter clutter robots robots
Cite：
Copy from the list or Export to your reference management。
| GB/T 7714 | Zhao, Min , Zuo, Guoyu , Huang, Gao . Collaborative Learning of Deep Reinforcement Pushing and Grasping based on Coordinate Attention in Clutter [J]. | 2022 INTERNATIONAL CONFERENCE ON VIRTUAL REALITY, HUMAN-COMPUTER INTERACTION AND ARTIFICIAL INTELLIGENCE, VRHCIAI , 2022 : 156-161 . |
| MLA | Zhao, Min et al. "Collaborative Learning of Deep Reinforcement Pushing and Grasping based on Coordinate Attention in Clutter" . | 2022 INTERNATIONAL CONFERENCE ON VIRTUAL REALITY, HUMAN-COMPUTER INTERACTION AND ARTIFICIAL INTELLIGENCE, VRHCIAI (2022) : 156-161 . |
| APA | Zhao, Min , Zuo, Guoyu , Huang, Gao . Collaborative Learning of Deep Reinforcement Pushing and Grasping based on Coordinate Attention in Clutter . | 2022 INTERNATIONAL CONFERENCE ON VIRTUAL REALITY, HUMAN-COMPUTER INTERACTION AND ARTIFICIAL INTELLIGENCE, VRHCIAI , 2022 , 156-161 . |
| Export to | NoteExpress RIS BibTex |
Abstract ：
新工科背景下面向创新能力培养的微机原理与应用课程改革
Keyword ：
课程改革 课程改革 高等教育 高等教育 微机原理与应用 微机原理与应用 创新人才培养 创新人才培养 新工科 新工科
Cite：
Copy from the list or Export to your reference management。
| GB/T 7714 | 左国玉 , 雷飞 , 乔俊飞 et al. 新工科背景下面向创新能力培养的微机原理与应用课程改革 [J]. | 左国玉 , 2021 , (2) : 108-112 . |
| MLA | 左国玉 et al. "新工科背景下面向创新能力培养的微机原理与应用课程改革" . | 左国玉 2 (2021) : 108-112 . |
| APA | 左国玉 , 雷飞 , 乔俊飞 , 计算机教育 . 新工科背景下面向创新能力培养的微机原理与应用课程改革 . | 左国玉 , 2021 , (2) , 108-112 . |
| Export to | NoteExpress RIS BibTex |
Abstract ：
新工科背景下基于机器人竞赛的创新人才培养模式
Keyword ：
机器人竞赛 机器人竞赛 创新人才 创新人才 培养模式 培养模式 新工科 新工科
Cite：
Copy from the list or Export to your reference management。
| GB/T 7714 | 左国玉 , 雷飞 , 乔俊飞 et al. 新工科背景下基于机器人竞赛的创新人才培养模式 [J]. | 左国玉 , 2021 , (6) : 44-47 . |
| MLA | 左国玉 et al. "新工科背景下基于机器人竞赛的创新人才培养模式" . | 左国玉 6 (2021) : 44-47 . |
| APA | 左国玉 , 雷飞 , 乔俊飞 , 高教学刊 . 新工科背景下基于机器人竞赛的创新人才培养模式 . | 左国玉 , 2021 , (6) , 44-47 . |
| Export to | NoteExpress RIS BibTex |
Abstract ：
自适应轨迹任务模仿的模仿学习方法研究
Keyword ：
机器人 机器人 性能评价 性能评价 动态系统 动态系统 非线性函数 非线性函数 模仿学习 模仿学习
Cite：
Copy from the list or Export to your reference management。
| GB/T 7714 | 于建均 , 姚红柯 , 左国玉 et al. 自适应轨迹任务模仿的模仿学习方法研究 [J]. | 于建均 , 2021 , 28 (2) : 266-274 . |
| MLA | 于建均 et al. "自适应轨迹任务模仿的模仿学习方法研究" . | 于建均 28 . 2 (2021) : 266-274 . |
| APA | 于建均 , 姚红柯 , 左国玉 , 阮晓钢 , 控制工程 . 自适应轨迹任务模仿的模仿学习方法研究 . | 于建均 , 2021 , 28 (2) , 266-274 . |
| Export to | NoteExpress RIS BibTex |
Abstract ：
The aim of generative adversarial imitation learning (GAIL) is to allow an agent to learn an optimal policy from demonstrations via an adversarial training process. However, previous works have not considered a realistic setting for complex continuous control tasks such as robot manipulation, in which the available demonstrations are imperfect and possibly originate from different policies. Such a setting poses significant challenges for the application of the GAIL-related methods. This paper proposes a novel imitation learning (IL) algorithm, MD2-GAIL, to enable an agent to learn effectively from imperfect demonstrations by multiple demonstrators. Instead of training the policy from scratch, unsupervised pretraining is used to speed up the adversarial learning process. Confidence scores representing the quality of the demonstrations are utilized to reconstruct the objective function for off-policy adversarial training, making the policy match the optimal occupancy measure. Based on the Soft Actor Critic (SAC) algorithm, MD2-GAIL incorporates the idea of maximum entropy into the process of optimizing the objective function. Meanwhile, a reshaped reward function is adopted to update the agent policy to avoid falling into local optima.Experiments were conducted based on robotic simulation tasks, and the results show that our method can efficiently learn from the available demonstrations and achieves better performance than other state-of-the-art methods. (c) 2021 Elsevier B.V. All rights reserved.
Keyword ：
Adversarial imitation learning Adversarial imitation learning Robot learning Robot learning Multiple demonstrators Multiple demonstrators Imperfect demonstrations Imperfect demonstrations
Cite：
Copy from the list or Export to your reference management。
| GB/T 7714 | Zuo, Guoyu , Zhao, Qishen , Huang, Shuai et al. Adversarial imitation learning with mixed demonstrations from multiple demonstrators [J]. | NEUROCOMPUTING , 2021 , 457 : 365-376 . |
| MLA | Zuo, Guoyu et al. "Adversarial imitation learning with mixed demonstrations from multiple demonstrators" . | NEUROCOMPUTING 457 (2021) : 365-376 . |
| APA | Zuo, Guoyu , Zhao, Qishen , Huang, Shuai , Li, Jiangeng , Gong, Daoxiong . Adversarial imitation learning with mixed demonstrations from multiple demonstrators . | NEUROCOMPUTING , 2021 , 457 , 365-376 . |
| Export to | NoteExpress RIS BibTex |
Abstract ：
Reproducing the spatial cognition of animals using computational models that make agents navigate autonomously has attracted much attention. Many biologically inspired models for spatial cognition focus mainly on the simulation of the hippocampus and only consider the effect of external environmental information (i.e., exogenous information) on the hippocampal coding. However, neurophysiological studies have shown that the striatum, which is closely related to the hippocampus, also plays an important role in spatial cognition and that information inside animals (i.e., endogenous information) also affects the encoding of the hippocampus. Inspired by the progress made in neurophysiological studies, we propose a new spatial cognitive model that consists of analogies between the hippocampus and striatum. This model takes into consideration how both exogenous and endogenous information affects coding by the environment. We carried out a series of navigation experiments that simulated a water maze and compared our model with other models. Our model is self-adaptable and robust and has better performance in navigation path length. We also discuss the possible reasons for the results and how our findings may help us understand real mechanisms in the spatial cognition of animals.
Keyword ：
spatial cognition spatial cognition brain-inspired computation brain-inspired computation striatum striatum hippocampus hippocampus Exogenous and endogenous information Exogenous and endogenous information
Cite：
Copy from the list or Export to your reference management。
| GB/T 7714 | Huang, Jing , Yang, He-Yuan , Ruan, Xiao-Gang et al. A Spatial Cognitive Model that Integrates the Effects of Endogenous and Exogenous Information on the Hippocampus and Striatum [J]. | INTERNATIONAL JOURNAL OF AUTOMATION AND COMPUTING , 2021 , 18 (4) : 632-644 . |
| MLA | Huang, Jing et al. "A Spatial Cognitive Model that Integrates the Effects of Endogenous and Exogenous Information on the Hippocampus and Striatum" . | INTERNATIONAL JOURNAL OF AUTOMATION AND COMPUTING 18 . 4 (2021) : 632-644 . |
| APA | Huang, Jing , Yang, He-Yuan , Ruan, Xiao-Gang , Yu, Nai-Gong , Zuo, Guo-Yu , Liu, Hao-Meng . A Spatial Cognitive Model that Integrates the Effects of Endogenous and Exogenous Information on the Hippocampus and Striatum . | INTERNATIONAL JOURNAL OF AUTOMATION AND COMPUTING , 2021 , 18 (4) , 632-644 . |
| Export to | NoteExpress RIS BibTex |
Export
| Results： |
Selected to |
| Format： |
