Rex - Parallel functional language for scientific workflows
Show HN: Rex, a parallel functional language for scientific workflows
Rex is a statically typed, pure functional workflow language designed for scientific computing and data processing. It combines a real functional language with content-addressable storage, typed tool APIs, and isolated Docker execution to make workflows concise, inspectable, and parallelizable. Rex treats data as immutable values, uses BLAKE3 hashes for artifact identity, and provides typed interfaces to tools like FFmpeg and ImageMagick, ensuring safe and reproducible pipelines. It also offers a clean boundary for LLM-generated workflows, with static types providing fast feedback and a closed tool boundary limiting execution risks.
Rex starts with a small general-purpose language instead of a DAG, making control flow, data flow, reuse, and error handling part of one expressive language rather than a mixture of YAML, shell, and application-specific configuration.
- troelsSteegin
I am curious to see where async would fit. I recognize distributed async is beyond scope for Rex. Given the treatment of content addressable storage (CAS), one could specify workflow states declaratively, poll artifacts for next, and fire tasks to a delegate and then move forward. It would be nice to have a subsystem do that. A question is where parallelism is useful in your scientific workflows, and where it blocks. Quantum chemistry use cases (QDX is behind Rex) are presently beyond me, so I have no idea.
I am thinking that Rex programs are inteneded as an output from a higher level worklow specification language. It would be interesting to see that.
- lubitelpospat
Great work!
Will be curious to see how it competes with the existing workflow languages such as Nextflow or Snakemake - do you by any chance have a "Why migrate from Nextflow/Snakemake to Rex" pitch I could read somewhere?
- cl3misch
I expected a reference to Dex, but they seem to be unrelated?