SAM2Act: Integrating Visual Foundation Model with A Memory Architecture for Robotic Manipulation
ArXiv, 2025
Welcome to MAGIC Lab
The MAGIC Lab at the National University of Singapore, School of Computing , was established in 2026 and is led by Assistant Professor Jiafei Duan . At MAGIC Lab, we study how to develop robotic intelligence that can perceive, reason, and act in complex physical environments, with the goal of creating real-world economic and societal impact through our research. We focus on a fundamental question: what are the essential building blocks needed to bring AI into the physical world and make it ready for deployment?
Our research interests span dexterous manipulation, multimodal large language model reasoning, vision-language-action models, robot learning, and understanding data for robotic manipulation.
News
- 2026 MolmoAct2 released for real-world deployment research.
- 2025 AHA accepted to ICLR 2025.
- 2024 Manipulate-Anything accepted to CoRL 2024.
- Now MAGIC Lab is actively recruiting across levels.
Publications
Full publication list adapted from Jiafei Duan's publications page .
AHA: A Vision-Language-Model for Detecting and Reasoning over Failures in Robotic Manipulation
ICLR, 2025
SAT: Spatial Aptitude Training for Multimodal Language Models
ArXiv, 2024
Manipulate-Anything: Automating Real-World Robots using Vision-Language Models
CoRL, 2024
RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotic
CoRL, 2024
EVE: Enabling Anyone to Train Robot using Augmented Reality
UIST, 2024
Octopi: Object Property Reasoning with Large Tactile-Language Models
RSS, 2024
THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation
RSS, 2024
Selective Visual Representations Improve Convergence and Generalization for Embodied-AI
ICLR, 2024
NEWTON: Are Language models Capable of Physical Reasoning
EMNLP Findings, 2023
AR2-D2:Training a Robot Without a Robot
CoRL, 2023
BOSS: A Benchmark for Human Belief Prediction in Object-context Scenarios
ArXiv Preprint
Good Time to Ask: A Learning Framework for Asking for Help in Embodied Visual Navigation
Ubiquitous Robots, 2023
A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categories
CogSci, 2023
ABCDE: An Agent-Based Cognitive Development Environment
CVPR Workshop, 2022
A Survey on Machine Learning Approaches for Modelling Intuitive Physics
IJCAI-ECCAI, 2022
PIP: Physical Interaction Prediction via Mental Simulation with Span Selection
ECCV, 2022
AVoE: A Synthetic 3D Dataset on Understanding Violation of Expectation for Artificial Cognition
NeurIPS Workshop, 2021
SPACE: A Simulator for Physical Interactions and Causal Learning in 3D Environments
ICCV Workshop, 2021
A Survey of Embodied AI: From Simulators to Research Tasks
IEEE Transactions on Emerging Topics in Computational Intelligence
ActioNet: An Interactive End-to-End Platform for Tasked-Based Data Collection and Augmentation in 3D Environment
ICIP, 2020
Team
Principal Investigator
Jiafei Duan
Jiafei Duan is the director of MAGIC Lab and an incoming Presidential Young Professor at the National University of Singapore School of Computing. He is currently a PhD student in Robotics and AI at the Paul G. Allen School of Computer Science & Engineering at the University of Washington, co-advised by Ranjay Krishna and Dieter Fox.
His research centers on robot learning, embodied AI, and large-scale robotics foundation models, with work spanning robot manipulation, failure reasoning, generalist control, and scalable evaluation.
Contact: duanj1 [at] cs.washington.edu
Join Us
MAGIC Lab is recruiting PhD students, postdocs, research assistants, masters students, and undergraduates. PhD, postdoc, and RA positions come with full funding. If you are excited about bringing AI into the physical world through robotics, we would love to hear from you.
Robotics experience is welcome but not required. Reach out by email after filling out the interest form linked from Jiafei Duan's personal website.
Global Visitors
Where MAGIC Lab visitors are coming from.