Decades in the Making: How Stephen Robertson Spent 18 Years Perfecting BM25

People that worked on the same idea for decades

Decades in the Making: How Stephen Robertson Spent 18 Years Perfecting BM25

This collection highlights individuals who pursued the same core idea for decades before achieving their magnum opus. One example is Stephen Robertson, whose 1976 probabilistic relevance theory evolved through multiple papers into BM25, the search algorithm still used by Lucene and Elasticsearch. Another is Katsushika Hokusai, who refined his wave imagery for 40 years, culminating in The Great Wave off Kanagawa. The post reflects on the patience and persistence behind breakthrough work.

At 100, I may reach a level of divine understanding, and if I live decades beyond that, everything I paint — dot and line — will be alive.
  1. srl

    "Few paintings have lasted the test of time as the great wave of Kanagawa. It took Kanagawa 40 years before he drew the wave he is most known for."

    The painting is by Hokusai. Kanagawa is a place.

  2. CM30

    Surprisingly common in fangames and mods. Mushroom Kingdom Fusion started in 2007 at the latest, and it's not going to be done for a year or so at the earliest. So, 20 years, maybe 30 if you count predecessor projects.

    Brutal Mario started in 2006 at the latest, and ended in the 2020s as far as I recall, so about 14-15 years. Of course, the 2006 demo was technically demo 6, so we have no idea when the first 5 versions were created since they seem to be gone forever.

    Super Mario Star World is a remake/sequel to the Mario 64 hack Super Mario Star Road, which was made for Super Mario 64 DS instead. It seemed to start out as a sequel in the Super Mario 64 engine in 2013, then ended up as a Super Mario 3D Land mod a few years later, got retooled as a Super Mario 64 DS hack a few years after that and is currently in development by a new creator after original creator SKELUX got cancer and moved onto other projects.

    In other mediums, I guess there's the Thief & the Cobbler. Started in 1964, got an unfinished/messed up release in 1993, and a better recuts in 2013 and 2023.

  3. interroboink

    I'm reminded of an old talk by Rich Hickey [1] (he created Closure, initially in semi-isolation over some years).

    > When was the last time that you thought about something for an

    > entire hour? ... Howabout for a whole day? ... or over the course of a

    > month? ... or a year? These are tremendously valuable moments, if you

    > get to have them at all.

    [1] https://www.youtube.com/watch?v=f84n5oFoZBc

  4. jll29

    Software

    Richard Stallman: GCC and Emacs - both massive pro?ects and still going strong.

    Linus Torvalds: Linux OS kernel

    Science

    Geoffrey Hinton: scaling neural networks

    Books

    Donald E. Knuth: Structure and Interpretation of Computer Programs (vol 4c of 12)

    Buildings

    misc. architects and builders: (most) cathedrals (Cologne, Barcelona, ...)

  5. chubot

    I think Charles Babbage is a special case -- IIRC his son also picked up his work, after his death. So he actually worked on it for "generations", by having a child who also worked on his ideas :-)

More from this day

2026-09-04