3D Vision and Modeling Challenges in eCommerce

ICCV 2023 Workshop

October 2, 2023 @ Paris, France
Room E06, Paris Convention Center


📢 Live Q&A on Slido: https://tinyurl.com/3DVeComm-slido (use it to ask questions for presentations and panel discussion).
📢 Remote presentations on Zoom: https://tinyurl.com/3DVeComm-zoom.

Introduction

This workshop aims to bring together researchers working on 3D computer vision and graphics for eCommerce, with a focus on the three topics: (1) 3D object/scene modeling and understanding in 3D eCommerce such as semantic segmentation, affordance and motion, multi-view reconstruction; (2) human modeling and fashion in 3D eCommerce such as virtual try-ons and personalized fashion recommendation, and (3) language-assisted reasoning such as shape/scene synthesis from texts and language grounding in 3D models. We invited 6 keynote speakers from academia and 3 talks from industry experts. We will also host a challenge on 3D part labeling for 3D models from real products sold online.


Schedule

All times in Paris Time (UTC+02:00)

8:50am - 9:00am Welcome and Introduction (Richard Zhang)
9:00am - 9:35am Invited Talk 1 (Leonidas Guibas)
Title: Compositional Object Modeling: Parts, Language, and Functionality
9:35am - 10:10am Invited Talk 2 (Rana Hanocka)
Title: Data-Driven Shape Editing without 3D Data
10:10am - 10:20am Coffee break
10:20am - 10:55am Invited Talk 3 (Michael Black)
Title: Implicit, Explicit, Real, and Synthetic: Spinning the Virtual Fashion Flywheel
10:55am - 11:30am Invited Talk 4 (Ming Lin)
Title: Dynamics-Inspired Learning-based Virtual Try-On
11:30am - 12:00pm Invited student paper presentations
HAL3D: Hierarchical Active Learning for Fine-Grained 3D Part Labeling (Fenggen Yu)
TriCoLo: Trimodal Contrastive Loss for fine-grained Text to Shape Retrieval (Angel Chang)
Improving Unsupervised Visual Program Inference with Code Rewriting Families (Aditya Ganeshan)
Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior (Junshu Tang)
D-IF: Uncertainty-aware Human Digitization via Implicit Distribution Field (Xueting Yang)
12:00pm - 1:30pm Lunch break
1:30pm - 2:00pm Winner presentations of the challenge
2:00pm - 2:35pm Invited Talk 5 (Roozbeh Mottaghi)
Title: Habitat 3.0: A Co-Habitat for Humans, Avatars, and Robots
2:35pm - 3:10pm Invited Talk 6 (Katerina Fragkiadaki)
Title: 3D Part Segmentation and Reconstruction with Little or No Training
3:10pm - 3:20pm Coffee break
3:20pm - 3:45pm Industry Talk 1 (Eric Bennett from Amazon)
Title: Scaling 3D eCommerce Innovations
A 10,000 Meter View on Building Durable 3D Content Creation Algorithms for Growth
3:45pm - 4:10pm Industry Talk 2 (Itamar Berger from Snap)
Title: Snap to Fit: AR Try-on in Snapchat
4:10pm - 4:35pm Industry Talk 3 (Chengfei Lv from Alibaba)
Title: Introduction to 3D and XR Technology in Taobao
4:35pm - 5:00pm Panel discussion and community building
Panelists: Ming Lin, Rana Hanocka, Michael Black
Moderator: Daniel Ritchie

Invited Speakers


Leonidas Guibas Paul Pigott Professor of Computer Science and Electrical Engineering at Stanford University. He heads the Geometric Computation group in the Computer Science Department. He works on algorithms for sensing, modeling, reasoning, rendering, and acting on the physical world. More recently, he has focused on shape analysis and computer vision using deep neural networks.


Rana Hanocka Assistant Professor of Computer Science at the University of Chicago. She directs 3DL, a group of enthusiastic researchers passionate about 3D, machine learning, and visual computing. She obtained her Ph.D. in 2021 from Tel Aviv University under the supervision of Daniel Cohen-Or and Raja Giryes. Her research is focused on building artificial intelligence for 3D data, spanning the fields of computer graphics, machine learning, and computer vision.


Michael Black Honorary Professor at the University of Tuebingen and one of the founding directors at the Max Planck Institute for Intelligent Systems in Tübingen, Germany, where he leads the Perceiving Systems department. He was also a Distinguished Amazon Scholar (VP, 2017-2021). Black's research interests in computer vision include optical flow estimation, 3D shape models, human shape and motion analysis, robust statistical methods, and probabilistic models of the visual world. Black co-founded he co-founded Body Labs Inc., which commercialized 3D body model technology, and was acquired by Amazon.com in 2017.


Ming Lin Distinguished University Professor and former Elizabeth Stevinson Iribe Chair of Computer Science at the University of Maryland College Park. Her research interests include computational robotics, haptics, physically-based modeling, virtual reality, sound rendering, and geometric computing. She has (co-)authored more than 300 refereed publications in these areas and co-edited/authored four books. Many of Lin’s research findings have been patented and licensed by more than 50 companies world wide. Lin is an Amazon Scholar with Amazon Fashion.


Roozbeh Mottaghi Research Scientist Manager at Meta and Affiliate Associate Prof. at the University of Washington working on Vision-and-Language and Embodied AI. Prior to joining FAIR, he was the Research Manager of the PRIOR team at the Allen Institute for AI. Before that, I was a Postdoctoral Researcher in the Computer Science Department at Stanford University. He was a post-doctoral researcher at the Computer Science Department at Stanford University. He obtained his PhD in Computer Science from University of California, Los Angeles.


Katerina Fragkiadaki Assistant Professor in the Machine Learning Department at Carnegie Mellon University. She received her Ph.D. from the University of Pennsylvania and was a postdoctoral fellow at UC Berkeley and Google research after that. Her work is on learning visual representations with little supervision and combining spatial reasoning in deep visual learning. Her group develops algorithms for mobile computer vision, learning of physics, and common sense for agents that move around and interact with the world.


Eric Bennett Director of Science at Amazon Imaging. Bennett leads the research and development of cutting-edge solutions for the creation of 3D models using ML, CV, photogrammetry, and more to bring new immersive experiences to Amazon's customers. He obtained his Ph.D. in Computer Science from University of North Carolina at Chapel Hill.


Itamar Berger Computer Vision Engineering Manager at Snap. His team develops products for improving the Augmented Reality experiences using Deep Learning and Generative AI for SnapAR. Before Snap, he co-founded a statup in the field of real-time motion capture and was a R&D manager at Autodesk.


Chengfei Lv Head of Alibaba's 3D/XR Technology Department. His team works on 3D/XR technologies specific to eCommerce, such as 3D reconstruction of commodities, high-performing volumetric video for MR, and 3D modeling and rendering engines, responsible for exploring innovative consumer applications for immersive experiences.


Challenge

Call for particpations: The workshop will host a competition focused on fine-grained semantic segmentation of 3D shapes. The competition will use 3D models from five categories (chair, table, cabinet, lamp, and bed) from the Amazon-Berkeley-Object (ABO) dataset, which has a total of 3400 models. The ABO dataset, recently published, features high quality, uniformly standard 3D models of real products sold online, created by artists. The models are made up of build-aware connected components, which form the basis of various shape properties such as texture, motion, function, interaction, and construction. The workshop challenge focuses on assigning fine-grained semantic labels (e.g., leg and arm, defined based on PartNet) to these connected components in the ABO dataset.

Submission site: https://eval.ai/web/challenges/challenge-page/2027/overview

Important Dates

Release of train and validation sets June 01, 2023
Release of test set Aug 15, 2023
Submission deadline Sep 10, 2023
Workshop date Oct 02, 2023

Organizers

Yiming Qian
Amazon
(Primary Contact)
Fenggen Yu
Simon Fraser University
Xu Zhang
Amazon
Jasmine Collins
UC Berkeley
Angel Chang
Simon Fraser University
Daniel Ritchie
Brown University
Huan Fu
Alibaba

Senior Organizers

Hao (Richard) Zhang
Simon Fraser University & Amazon



Acknowledgments

We thank Matthieu Guillaumin for his assistance in data uploading and for driving the ABO dataset release! We thank visualdialog.org for the webpage format.