Lingzhi Zhang

I am a PhD student in CS at at the University of Pennslvnia, advised by Professor Jianbo Shi. I am interested in problems in computer vision, robotics and artificial intelligence. Recently, I have been working on image editings and boosting visual recognition with generative models.

profile photo
Research
Learning Diverse Object Placement by Inpainting for Compositional Data Augmentation
Lingzhi Zhang, Tarmily Wen, Jie Min, David Han, Jianbo Shi
ECCV 2020

We study the problem of common sense placement of visual objects in an image. We first propose a self-learning framework that automatically generates necessary training data without any manual labeling by detecting, cutting, and inpainting objects from an image. We learn a generative model that predicts a set of diverse common sense locations when given a foreground object and a background scene. We show experimentally our object placement newtork can be used to augment training data to boost instance segmentation.

Nested Scale-Editing for Conditional Image Synthesis
Lingzhi Zhang*, Jiancong Wang*, Yinshuang Xu, Jie Min, Tarmily Wen, James C. Gee, Jianbo Shi (* indicates equal contribution)
CVPR 2020

We proposed an image synthesis approach that provides stratified navigation in the latent code space. We achieve this through scale-independent editing while expanding scale-specific diveristy. Scale-independent is achieved with a nested scale disentanglement loss. Scale-specific diversity is created by incorporating a progressive diversification constraint.

Deep Image Blending
Lingzhi Zhang, Tarmily Wen, Jianbo Shi
WACV 2020

We propose a Poisson blending loss that achieves the same purpose of Poisson Image Editing. We jointly optimize the proposed Poisson blending loss with style and content loss computed from a deep network, and reconstruct the blending region by iteratively updating the pixels using the L-BFGS solver. In the blending image, we not only smooth out gradient domain of the blending boundary but also add consistent texture into the blending region.

Multimodal Image Outpainting with Regularized Normalized Diversification
Lingzhi Zhang, Jiancong Wang, Jianbo Shi
WACV 2020

We study the problem of generating a set of realistic and diverse backgrounds when given only a small foreground region, which we formulate as image outpainting task. We propose a generative model by improving the normalized diversification framework to encourage diverse sampling in this conditional synthesis task. The results show that our proposed approach can produce more diverse images with similar or better quality compare to the state-of-the-arts methods.

Neural Embedding for Physical Manipulations
Lingzhi Zhang*, Andong Cao*, Rui Li, Jianbo Shi (* indicates equal contribution)
Machine Learning for Physical Science Workshop, NeurIPS, 2019

Inspired by the properties of grid cells in mammalian brains, we build a generative model that enforces a normalized pairwise distance constraint between the latent space and output space to achieve data-efficient discovery of output spaces. We leverage this approach to learn the full topology of action and state spaces when given only few and sparse observations.

Teaching
upenn Teaching Assistant, CIS581 Computer Vision and Computational Photography (Professor Jianbo Shi), Fall 2018, Fall 2019

Teaching Assistant, CIS680 Vision and Learning (Professor Jianbo Shi), Fall 2019

Teaching Assistant, CIS519 Applied Machine Learning (Professor Dan Roth), Spring 2018

Industrial Experiences
ibm Adobe Research Intern, May 2020 - Present

ibm Machine Learning Intern, June - August 2015

zhenfund Investment Analyst Intern, August 2015 - February 2016