Sean Hardesty Lewis

I am a predoctoral researcher at MIT studying how intelligent agents learn transferable representations of complex environments and the people acting within them. My work often uses large-scale physical and virtual worlds to study how representations of environments and behavior generalize across settings.

Previously, I completed my M.S. in Information Science at Cornell Tech, where I was advised by Wendy Ju and Anthony Townsend. There, I created Searchable.City, a semantic representation built from over 50 million street-view images that enables natural-language search across the physical world.

I also created Human Operator, a vision-language system that translates observations into real-time motor actions executed through a human body, which reached over 30 million views online.

Before Cornell, I received my B.S. in Mathematics from UT Austin, where I worked with Junfeng Jiao on AI systems for city-scale sensing and simulation.

Sean Hardesty Lewis profile picture

email:

end@mit.edu

selected work in progress

Learning Models of People from Virtual Worlds

Sean Hardesty Lewis

In Progress

selected publications

Searchable.City: An Open-Vocabulary Semantic Atlas

Sean Hardesty Lewis

In Proceedings of SIGGRAPH 2026
SIGGRAPH 2026 Emerging Technologies Award

DOIPDFCode

A Decentralized Digital Twin via Crowdsourced Sensing and Browser-Based Edge Computation

Sean Hardesty Lewis, Junfeng Jiao, Yiming Xu, Jihyung Park, Connor Phillips

Presented at AAAI 2026 Deployable AI Workshop

PDFCode

news

  • Sep 2026

    Transferable visual representations was accepted to the SIGSPATIAL 2026 UMFM Workshop. See you in Riverside! 🇺🇸

  • July 2026

    Presented Searchable.City at SIGGRAPH 2026 in Los Angeles and received the DCAJ award. See you in Tokyo! 🇯🇵

  • May 2026

    Demonstrated Daunt, a real-time runtime for embodied AI agents in virtual worlds, to 600+ people at Cornell Tech.

  • Mar 2026

    Won the MIT Media Lab HARD MODE hackathon with Human Operator, using VLMs and EMS to control human bodies.

  • Oct 2025

    Happy that StreamTwin was accepted to the Deployable AI Workshop at AAAI 2026. See you in Singapore! 🇸🇬

  • Sep 2025

    Thrilled to share that OpenCityCorpus was accepted to the NeurIPS AI4Science workshop.

  • Sep 2025

    Proud of my identical twin brother Ryan on starting his PhD at MIT!

  • Aug 2025
    Presented Minecraft to 3D at SIGGRAPH 2025 in Vancouver. 🇨🇦
  • Aug 2024
    Started my M.S. at Cornell Tech in New York City.
  • May 2024
    Graduated from UT Austin with a B.S. in Mathematics.
  • Feb 2024
    Our work on emergency AI, SafeMate, was funded by the City of Austin.