Shan Yang

Shan Yang

Staff Applied Scientist, Adobe Foundry | Ex-Tech Lead Amazon | Xoogler

I am a Staff Applied Scientist at Adobe Foundry, where I post-train Adobe's family of generative AI foundation models for customer-facing products powering Firefly and creative tooling. My research centers on four threads: post-training (SFT, RLHF/DPO, distillation, test-time training) to align foundation models with creator intent; multi-modal understanding across video, image, and language; world modeling — building models with internal representations of how the physical world behaves; and reasoning — RL and test-time compute for physics and scientific problem solving.

Previously I was a Tech Lead at Amazon, where I drove GenAI Live Action Studio and Amazon Video Search, and a Senior Research SDE at Google Research working on multi-modal generative models. I completed my PhD at UNC-CH under Prof. Ming C. Lin.

Publications

TTC-Net

Beyond Test-Time Training: Learning to Reason via Hardware-Efficient Optimal Control

Peihao Wang, Shan Yang, Xijun Wang, Tesi Xiao, Xin Liu, Changlong Yu, Yu Lou, Pan Li, Zhangyang Wang, Ming Lin, René Vidal

ICML 2026

VLAP

Schema Perception for Robust Video Question Answering

Xijun Wang, Shan Yang

Internal 2025

VLAP

VLAP: Efficient Video-Language Alignment via Frame Prompting and Distilling for Video Question Answering

Xijun Wang, Junbang Liang, Chun-Kai Wang, Kenan Deng, Yu Lou, Ming Lin, Shan Yang

ECCV 2024

ICAR

ICAR: Image-based Complementary Auto Reasoning

Xijun Wang, Anqi Liang, Junbang Liang, Ming Lin, Yu Lou, Shan Yang

AAAI 2024

MeSa

MeSa: Masked, Geometric, and Supervised Pre-training for Monocular Depth Estimation

Muhammad Osama Khan, Junbang Liang, Chun-Kai Wang, Shan Yang, Yu Lou

NIPS 2023 Workshop SSLTheoryPractice

RoSI

RoSI: Recovering 3D Shape Interiors from Few Articulation Images

Akshay Gadi Patil, Yiming Qian, Shan Yang, Brian Jackson, Eric Bennett, Hao Zhang

In submission

MBT

Attention Bottlenecks for Multimodal Fusion

Arsha Nagrani, Shan Yang, Anurag Arnab, Aren Jansen, Cordelia Schmid, Chen Sun

NeurIPS 2021

Optical Mouse

Optical Mouse: 3D Mouse Pose From Single-View Video

Shan Yang*, Bo Hu*, David A. Ross, Avneesh Sud, Yi Liu, Graham Ruby, Bryan Seybold

CVPR 2021 (CV4Animal Workshop)

AI Choreographer

AI Choreographer: Music Conditioned 3D Dance Generation with AIST++

Shan Yang*, Ruilong Li*, David A. Ross, Angjoo Kanazawa

ICCV 2021

Cloth Material Recovery

Learning-based Cloth Material Recovery from Video

Shan Yang, Junbang Liang, Ming C. Lin

ICCV 2017

Garment Recovery

Physics-Inspired Garment Recovery from a Single-View Image

Shan Yang, Zherong Pan, Tanya Amert, Ke Wang, Licheng Yu, Tamara Berg, Ming C. Lin

ACM TOG 2018

Referring Expressions

Modeling Context in Referring Expressions

Licheng Yu, Patric Poirson, Shan Yang, Alex Berg, Tamara Berg

ECCV 2016

Prostate Cancer Classification

Classification of Prostate Cancer Grades and T-Stages based on Tissue Elasticity Using Medical Image Analysis

Shan Yang, Vladimir Jojic, Jun Lian, Ronald Chen, Hongtu Zhu, Ming C. Lin

MICCAI 2016

Bayesian Estimation

Bayesian Estimation of Non-Rigid Mechanical Parameters Using Temporal Sequences of Deformation Samples

Shan Yang, Ming C. Lin

ICRA 2016

MaterialCloning

MaterialCloning: Acquiring Elasticity Parameters from Images for Medical Applications

Shan Yang, Ming C. Lin

IEEE TVCG 2016

Simultaneous Estimation

Simultaneous Estimation of Elasticity for Multiple Deformable Bodies

Shan Yang, Ming C. Lin

Computer Animation and Virtual Worlds, 2015

Buried Suture

Real-time simulation for buried suture

Shan Yang, Wenlong Lu, Lixu Gu

CARS 2012

Projects

Lumi Research Manager

AI-powered research project manager with pixel-art agent teams (Scout, Theorist, Architect, Coder, and more)

Multimodal Content Creation Training Infrastructure (MINT)

Supporting multimodal dance generation

AIST++ Dataset API

Developer tools and loaders for the AIST++ dataset

RL Learning Log

Personal log of learning Reinforcement Learning — notes, experiments, and insights