Claude-built worlds: Union Square recreated in browser-native 3D
Fable 5.1 World Modeling
PhiloLabs' fable51-worlds repo shows how autonomous Claude Fable 5.1 agent swarms can research, model, and quality-check entire real-world locations, shipping them as plain Three.js apps. The first world, Union Square, San Francisco, includes 453 building footprints, 129 named storefronts, working traffic lights and cable cars, and explorable interiors of Apple and Nintendo stores. Everything is generated from open data and public imagery, with code in the repo for re-running the pipeline.
Every building, storefront, sign, tree and traffic light is generated from open data and public reference imagery by code that lives in this repo.
- kodefreeze
I have experimented with this type of modeling for a RTS game I'm building: https://playwarpact.com
Few things:
1. Opus 5 is just as good for this tbh and cheaper.
2. They don't generate optimized 3d models. They have high poly count for simple geometries.
3. A better approach I've utilized for game ready assets is to use the model to create low poly silhouettes of the 3d model and then bake textures that include a lot more details like windows, doors etc with tools like Meshy. Can post a tutorial if there's interest.
The models need some more RL to be able to do this autonomously.
- WordNotSee
Fundamentally it’s an image model predicting out frames of a sequence.
It’s not enough to call it just an image model though. Not quite video either because it’s so specific to rendering images of the world.
Thus, “world model”?
The problem I have with “world” is it should imply so much more depth. To me this is actually a “POV image model” or “First-person perspective model” (maybe we call this an FPM).
There could also be TPMs (third-person models).
This maps better to modern game and 3D dev phrasing, where “world model” might imply 3D object collections spatially organized in a cohesive file format - complete with interactivity, audio, and all the baseline constituents of what one might call a “world”.
FPMs and TPMs, think about it. Reserve POVMs for photographic realism.
- luciana1u
the 'most ai games want me to sign in' bit is the real world model. the game world is fine, it's the account creation funnel that's fully realized.
- hadlock
This is the kind of demo that would really benefit from a youtube demo of 2-3 min
Fake edit: there is a longer video here 1 min long: https://github.com/PhiloLabs/fable51-worlds/blob/main/union-...
I would be especially curious to see the NPC person/car logic and if they're on rails or what, that's a pretty good NPC density for a demo.
- julien_dev
These things always look impressive but I fail to see how they are usable beyond demos or simple games. In my experience the topology is often a mess and texturing is difficult as well