SAM2Act: Integrating Visual Foundation Model with A Memory Architecture for Robotic Manipulation
ArXiv, 2025
Welcome to MAGIC Lab
The MAGIC Lab at the National University of Singapore, School of Computing , was established in 2026 and is led by Assistant Professor Jiafei Duan . At MAGIC Lab, we study how to develop robotic intelligence that can perceive, reason, and act in complex physical environments, with the goal of creating real-world economic and societal impact through our research. We focus on a fundamental question: what are the essential building blocks needed to bring AI into the physical world and make it ready for deployment?
Our research interests span MLLM reasoning, 3D vision, robot learning, simulation, dexterous manipulation, cross-embodiment learning and more.
Join Us
MAGIC Lab is recruiting PhD students, postdocs, research assistants, masters students, and undergraduates. PhD, postdoc, and RA positions come with full funding. If you are excited about bringing AI into the physical world through robotics, we would love to hear from you. Sign up here.
Robotics experience is welcome but not required. Reach out by email after filling out the interest form linked from Jiafei Duan's personal website.
Why Singapore and Me?
Why choose Singapore for your research journey?
- Industry: Many industry and tech companies in AI are setting up shops in Singapore for research, for example Google, Meta, NVIDIA, and more.
- Startups: Many startups in robotics are also opening HQs or offices here: Unitree, Sharpa, Cortex AI, and many more.
- Academia: We have many globally recognized researchers in AI and Robotics based here.
- Environment: Singapore is one of the nicest, safest, and most promising countries to live in. It's a major global hub, with strong government support for embodied AI and physical AI through the NRP (National Robotics Programme).
But why me?
Join me to find out!
News
- Now MAGIC Lab is actively recruiting across levels!
- 2026 MolmoAct2 released for real-world deployment research.
- 2025 MolmoSpace accepted to RSS 2026.
Publications
Full publication list adapted from Jiafei Duan's publications page .
AHA: A Vision-Language-Model for Detecting and Reasoning over Failures in Robotic Manipulation
ICLR, 2025
SAT: Spatial Aptitude Training for Multimodal Language Models
ArXiv, 2024
Manipulate-Anything: Automating Real-World Robots using Vision-Language Models
CoRL, 2024
RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotic
CoRL, 2024
EVE: Enabling Anyone to Train Robot using Augmented Reality
UIST, 2024
Octopi: Object Property Reasoning with Large Tactile-Language Models
RSS, 2024
THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation
RSS, 2024
Selective Visual Representations Improve Convergence and Generalization for Embodied-AI
ICLR, 2024
NEWTON: Are Language models Capable of Physical Reasoning
EMNLP Findings, 2023
AR2-D2:Training a Robot Without a Robot
CoRL, 2023
BOSS: A Benchmark for Human Belief Prediction in Object-context Scenarios
ArXiv Preprint
Good Time to Ask: A Learning Framework for Asking for Help in Embodied Visual Navigation
Ubiquitous Robots, 2023
A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categories
CogSci, 2023
ABCDE: An Agent-Based Cognitive Development Environment
CVPR Workshop, 2022
A Survey on Machine Learning Approaches for Modelling Intuitive Physics
IJCAI-ECCAI, 2022
PIP: Physical Interaction Prediction via Mental Simulation with Span Selection
ECCV, 2022
AVoE: A Synthetic 3D Dataset on Understanding Violation of Expectation for Artificial Cognition
NeurIPS Workshop, 2021
SPACE: A Simulator for Physical Interactions and Causal Learning in 3D Environments
ICCV Workshop, 2021
A Survey of Embodied AI: From Simulators to Research Tasks
IEEE Transactions on Emerging Topics in Computational Intelligence
ActioNet: An Interactive End-to-End Platform for Tasked-Based Data Collection and Augmentation in 3D Environment
ICIP, 2020
Team
Director
Jiafei Duan
Jiafei Duan is an assistant professer under the Presidential Young Professorship scheme at the National University of Singapore School of Computing. He did his PhD in Robotics and AI at the Paul G. Allen School of Computer Science & Engineering at the University of Washington, under co-advised by Ranjay Krishna and Dieter Fox.
Contact: duanj1 [at] cs.washington.edu
Current
- Haoquan Fang (UW BS → Incoming Stanford PhD)
- Shuo Liu (UW MS)
- Shanli Xing (UW BS → Incoming CMU PhD)
- William Tsai (UW BS/MS)
- Sam Wang (UPenn CS)
- Yalcin Tur (Stanford BS)
Alumni
- Jun Wang (2023-2024) SWE Google
- Ye Jie (2021-2022) SWE at Shopee
- Wilbert Pumacay (2023-2025) AI2 Predoctoral Investigator
- Arijit Dasgupta (2021-2022) MIT PhD
Group Photo
Global Visitors
Where MAGIC Lab visitors are coming from.