Tony Fang

Transparent objects are common in our daily life and frequently handled in the automated production line. Robust vision-based robotic grasping and manipulation for these objects would be beneficial for automation. However, the majority of current grasping algorithms would fail in this case since they heavily rely on the depth image, while ordinary depth sensors usually fail to produce accurate depth information for transparent objects owing to the reflection and refraction of light. In this work, we address this issue by contributing a large-scale real-world dataset for transparent object depth completion, which contains 57,715 RGB-D images from 130 different scenes. Our dataset is the first large-scale, real-world dataset that provides ground truth depth, surface normals, transparent masks in diverse and cluttered scenes. Cross-domain experiments show that our dataset is more general and can enable better generalization ability for models. Moreover, we propose an end-to-end depth completion network, which takes the RGB image and the inaccurate depth map as inputs and outputs a refined depth map. Experiments demonstrate superior efficacy, efficiency and robustness of our method over previous works, and it is able to process images of high resolutions under limited hardware resources. Real robot experiments show that our method can also be applied to novel transparent object grasping robustly. The full dataset and our method are publicly available at www.graspnet.net/transcg

Robot Operation Python Engine (ROPE) provides tele-operation function from haptic devices and various types of robots, including UR-5, Franka Panda, Kuka IIWA, and Flexiv Robot. The project is now in final refinement, and the code will be released soon.

Current scholar search engine cannot recognize “jargon”, that is, specialized termilogy associated with a particular field or area of activity. In order to make the search more precisely, we build Oh-My-Papers, a hybrid context-aware citation recommendation system, as well as a scholar search engine. We first point out that we can learn jargons from academic paper citation information. Since specialists usually use jargons such as “ResNet” in the academic paper writings, and the reference to the corresponding paper usually follows the jargon. Moreover, citation information (especially citation context) of academic papers can help us to improve the searching results. In this work, we also create a large dataset recording citation information from papers of computer vision field in recent years. The dataset is ~10x larger than the biggest dataset from the previous works.

In this work, we review the previous protein contact-map prediction models and apply the several attention mechanisms into the classic protein contact-map prediction networks ResPRE. The combination has reached a relatively great results, which improve both short-term and long-term prediction by approximately 6%.

In this work, we use CUDA and CUDNN to implement the famous deep learning network Inception-V3. We use implicit im2col method to optimize the convolution layers and reach satisfactory performance. Our CUDA implementation has an average inference time of 61.096 ms, which only takes 60% of the CUDNN inference time.

In this work, we propose a handy toolbox to construct high dynamic range (HDR) image from multiple low dynamic range (LDR) images. We provide traditional methods using alignment, and several deep-learning-based methods (DeepHDR, NHDRRNet) as well as our proposed DeepNLHDR network. Experiments show that our model outperforms baseline models by 1% ~ 15%.

I gained knowledges and experiences while studying courses in Computer Science during undergraduate studies, and I got great results in many of the courses. Here are some selected course:

• CS158: Data Structure (Honor) (Spring 2019, 100/100) [Project Deque and BTree]
• CS214: Algorithm and Complexity (Spring 2020, 100/100) [Homeworks and Project]
• CS145: Computer Architecture Experiments (Spring 2020, 100/100) [Labs]
• CS385: Machine Learning (Spring 2021, 97.5/100) [Project 1 and Project 2 (VAE and GAN)]
• EE447: Mobile Internet (Spring 2021, 96/100) [Homeworks, Labs, and Project Oh-My-Papers]

Sharing is caring.

I started this project to share the notes written by my friends and me, which may be helpful for other students who take the same courses in the future.

I’m currently research in Machine Vision and Intelligence Group (MVIG-SJTU), and advised by Prof. Cewu Lu. During undergraduates, I have done the following works.

• Adapt AlphaPose to foot keypoints;
• Revise GPD to make it work in GraspNet framework [details], and SPVCNN backbone migration to GraspNet;
• Cooperate author of paper “Graspness Discovery in Clutters for Fast and Accurate Grasp Detection” (ICCV 2021);
• Main contributor of transparent object depth completion project TransCG (RA-L 2022).

Before that, I took part in the 36th Participation in the research Program (PrP) in SJTU. The program aims at providing undergraduate students with an initial understanding of research. During the programs, I’m advised by Prof. Qinxiang Cao and have done the following works.

• Learn to use the Proof Assistant: Coq;
• Use Coq to prove the correctness and complexity of Euclidean algorithm [details];
• Use Coq to prove the correctness of depth-first search (DFS) algorithm [details];
• Use Coq to build the proof framework of Tarjan algorithm.

• Shanghai Outstanding Graduates; 2022; top 1%.

• Shanghai Scholarship (Shanghai Education Bureau); 2020; top 3%; CNY ¥8,000.
• Zhiyuan Scholarship (Zhiyuan College, SJTU); 2018/2019/2020/2021; top 5%; CNY ¥5,000.
• Fuguang Scholarship (Fuguang Foundation); 2018/2019/2020/2021; top 0.05% in College Entrance Examination (Gaokao); CNY ¥40,000.
• Shanghai Jiao Tong University Class B Excellent Scholarship (SJTU); 2018/2019/2020; top 30%; CNY ¥1,000.
• Shanghai Jiao Tong University Class C Excellent Scholarship (SJTU); 2021; top 50%; CNY ¥500.
• Computer Science Department Education Development Fund of ‘85 Alumni and Yang Yuanqing Education Fund Scholarship (Computer Science and Engineering Department, SJTU); 2021; top 1%; CNY ¥25,000.

• UBS Quant Hackerthon 2020, Finalist (8/371);
• 2020 Interdisciplinary Contest in Modeling, Honorable Mention;
• 2017 National Olympiads in Informatics (NOI), Bronze Medal;
• 2017 Asia-Pacific Informatics Olympiad (APIO), Bronze Medal;
• 2017 China Team Selection Competition (CTSC) in Informatics Olympiad, Bronze Medal;
• 2017 Winter Camp (WC) in Informatics Olympiad, Bronze Medal;
• 2016 National Olympiads in Informatics in Provinces (NOIP), First Prize;
• 2015 National Olympiads in Informatics in Provinces (NOIP), First Prize.

• Spring 2019, Data Structure (Honor) (taught by Prof. Yong Yu);
• Fall 2020, C++ Programming Language (Honor) (taught by Prof. Huiyu Weng);
• Fall 2020, Mathematical Analysis (Honor) I (taught by Prof. Keying Chen);
• Fall 2021, Mathematical Analysis (Honor) I (taught by Prof. Keying Chen);
• Fall 2021, Linear Algebra (Honor) (taught by Prof. Hao Shen);
• Spring 2022, Algorithm and Complexity (taught by Prof. Xiaofeng Gao)
• Fall 2022, Linear Algebra (Honor) (taught by Prof. Hao Shen) [Course Materials];

Research services

• Oct. 2022, reviewer of ICRA’23.
• Mar. 2023, reviewer of IROS’23.

Social Services

• Apr. 2019, volunteer, Shanghai International Half Marathon;
• Jul. 2019, member, “The Choice of Youth Project” (a project aims to let the students understand the current situation of different industries);
• Aug. 2019, member, “Zhufei Project” (a project aims to help the students of SJTU who have financial difficulties);
• Nov. 2019, volunteer, Shanghai International Marathon.