3D Vision and Modeling Challenges in eCommerce

ICCV 2023 Workshop

October 2, 2023 @ Paris, France


Introduction

This workshop aims to bring together researchers working on 3D computer vision and graphics for eCommerce, with a focus on the three topics: (1) 3D object/scene modeling and understanding in 3D eCommerce such as semantic segmentation, affordance and motion, multi-view reconstruction; (2) human modeling and fashion in 3D eCommerce such as virtual try-ons and personalized fashion recommendation, and (3) language-assisted reasoning such as shape/scene synthesis from texts and language grounding in 3D models. We invited 6 keynote speakers from academia and 3 talks from industry experts. We will also host a challenge on 3D part labeling for 3D models from real products sold online.


Schedule

All times in Paris Time (UTC+02:00)

8:50am - 9:00am Welcome and Introduction
9:00am - 9:35am Invited Talk 1 (Leonidas Guibas)
Title: TBD
9:35am - 10:10am Invited Talk 2 (Rana Hanocka)
Title: TBD
10:10am - 10:20am Coffee break
10:20am - 10:55am Invited Talk 3 (Michael Black)
Title: Implicit, Explicit, Real, and Synthetic: Spinning the Virtual Fashion Flywheel
10:55am - 11:30am Invited Talk 4 (Ming Lin)
Title: TBD
11:30am - 11:40am Coffee break
11:40am - 12:00pm Invited student paper presentations
12:00pm - 1:30pm Lunch break and invited poster presentations
1:40pm - 2:00pm Winner presentations of the challenge
2:00pm - 2:35pm Invited Talk 5 (Roozbeh Mottaghi)
Title: TBD
2:35pm - 3:10pm Invited Talk 6 (Katerina Fragkiadaki)
Title: TBD
3:10pm - 3:20pm Coffee break
3:20pm - 3:45pm Industry Talk 1 (Eric Bennett from Amazon)
Title: TBD
3:45pm - 4:10pm Industry Talk 2 (Itamar Berger from Snap)
Title: TBD
4:10pm - 4:35pm Industry Talk 3 (Chengfei Lv from Alibaba)
Title: TBD
4:35pm - 5:00pm Panel discussion and community building

Invited Speakers


Leonidas Guibas Paul Pigott Professor of Computer Science and Electrical Engineering at Stanford University. He heads the Geometric Computation group in the Computer Science Department. He works on algorithms for sensing, modeling, reasoning, rendering, and acting on the physical world. More recently, he has focused on shape analysis and computer vision using deep neural networks.


Rana Hanocka Assistant Professor of Computer Science at the University of Chicago. She directs 3DL, a group of enthusiastic researchers passionate about 3D, machine learning, and visual computing. She obtained her Ph.D. in 2021 from Tel Aviv University under the supervision of Daniel Cohen-Or and Raja Giryes. Her research is focused on building artificial intelligence for 3D data, spanning the fields of computer graphics, machine learning, and computer vision.


Michael Black Honorary Professor at the University of Tuebingen and one of the founding directors at the Max Planck Institute for Intelligent Systems in Tübingen, Germany, where he leads the Perceiving Systems department. He was also a Distinguished Amazon Scholar (VP, 2017-2021). Black's research interests in computer vision include optical flow estimation, 3D shape models, human shape and motion analysis, robust statistical methods, and probabilistic models of the visual world. Black co-founded he co-founded Body Labs Inc., which commercialized 3D body model technology, and was acquired by Amazon.com in 2017.


Ming Lin Distinguished University Professor and former Elizabeth Stevinson Iribe Chair of Computer Science at the University of Maryland College Park. Her research interests include computational robotics, haptics, physically-based modeling, virtual reality, sound rendering, and geometric computing. She has (co-)authored more than 300 refereed publications in these areas and co-edited/authored four books. Many of Lin’s research findings have been patented and licensed by more than 50 companies world wide. Lin is an Amazon Scholar with Amazon Fashion.


Roozbeh Mottaghi Research Scientist Manager at Meta and Affiliate Associate Prof. at the University of Washington working on Vision-and-Language and Embodied AI. Prior to joining FAIR, he was the Research Manager of the PRIOR team at the Allen Institute for AI. Before that, I was a Postdoctoral Researcher in the Computer Science Department at Stanford University. He was a post-doctoral researcher at the Computer Science Department at Stanford University. He obtained his PhD in Computer Science from University of California, Los Angeles.


Katerina Fragkiadaki Assistant Professor in the Machine Learning Department at Carnegie Mellon University. She received her Ph.D. from the University of Pennsylvania and was a postdoctoral fellow at UC Berkeley and Google research after that. Her work is on learning visual representations with little supervision and combining spatial reasoning in deep visual learning. Her group develops algorithms for mobile computer vision, learning of physics, and common sense for agents that move around and interact with the world.


Eric Bennett Director of Science at Amazon Imaging. Bennett leads the research and development of cutting-edge solutions for the creation of 3D models using ML, CV, photogrammetry, and more to bring new immersive experiences to Amazon's customers. He obtained his Ph.D. in Computer Science from University of North Carolina at Chapel Hill.


Itamar Berger Computer Vision Engineering Manager at Snap. His team develops products for improving the Augmented Reality experiences using Deep Learning and Generative AI for SnapAR. Before Snap, he co-founded a statup in the field of real-time motion capture and was a R&D manager at Autodesk.


Chengfei Lv Head of Alibaba's 3D/XR Technology Department. His team works on 3D/XR technologies specific to eCommerce, such as 3D reconstruction of commodities, high-performing volumetric video for MR, and 3D modeling and rendering engines, responsible for exploring innovative consumer applications for immersive experiences.


Challenge

Call for particpations: The workshop will host a competition focused on fine-grained semantic segmentation of 3D shapes. The competition will use 3D models from five categories (chair, table, cabinet, lamp, and bed) from the Amazon-Berkeley-Object (ABO) dataset, which has a total of 3400 models. The ABO dataset, recently published, features high quality, uniformly standard 3D models of real products sold online, created by artists. The models are made up of build-aware connected components, which form the basis of various shape properties such as texture, motion, function, interaction, and construction. The workshop challenge focuses on assigning fine-grained semantic labels (e.g., leg and arm, defined based on PartNet) to these connected components in the ABO dataset.

Submission site: TBD

Important Dates

Release of train and validation sets May 15 2023
Release of test set July 15 2023
Submission deadline Sep 1 2023
Workshop date Oct 2 2023

Organizers

Angel Chang
Simon Fraser University
Jasmine Collins
UC Berkeley
Huan Fu
Alibaba
Yiming Qian
Amazon
Daniel Ritchie
Brown University
Fenggen Yu
Simon Fraser University
Xu Zhang
Amazon

Advisory Board

Hao (Richard) Zhang
Simon Fraser University & Amazon



Acknowledgments

We thank Matthieu Guillaumin for his assistance in data uploading and for driving the ABO dataset release! We thank visualdialog.org for the webpage format.