Programmers Love map and filter, but Can't Stand reduce

For me it's the name[0]. map puts out an array that has been mapped from another array. filter puts out an array that is a filter of the input array. both of those are always true. reduce, on the other hand, may put out a reduction of the input array (probably most of the time), but the fact that it may not means that what is happening is not actually a reduction. In languages like js/ts, you don't even have to return anything of the same type as the input array's elements. You could literally "reduce" and array of integers to a cancellation token, or a state object, or anything else.

I realize it's not the most efficient way to work, but I like my code to read like instructions. There's nothing reduce will do that a for loop won't accomplish and the for loop (+ an accumulator, of course) is more clearly "readable" than reduce. If I read map, I know what's going on. If I read filter, I know what's going on. If I read reduce, I have to figure out what's going on, even if I'm pretty sure what is going on. If I could rely on reduce to always give me back an element of the input array, I would use it more. But since it can give back anything, I prefer the simplicity of a for loop.

[0] I don't have any suggestions for "better" names because the whole operation is hard to sum up in a word? "dispatch" makes sense, as a function dispatching a function over each element in an array, but it masks the concept of accumulation from return values. "transform" is accurate, but hardly descri […]

Related to the point about worse performance, I'm pretty sure I was there when reduce was "banished" from Python 3 -- demoted to functools.reduce(), instead of the builtin reduce() in Python 2

The story is that sometime in 2006 or 2007, Guido van Rossum was debugging why a web page in Google's internal code review tool (which he wrote) was taking 30+ seconds to render.

This is basically a "production" incident, since thousands of Google engineers relied on the tool. Requests like this were probably tying up threads and exhausting thread pools, perhaps

Eventually it was tracked down to a line wrapping algorithm written with reduce(). I don't think he wrote it -- it may have come in through a dependency. As many know, reduce() is basically:

s1 + s2

s1 + s2 + s3

s1 + s2 + s3 + s4

...

And that's O(n^2) when s_i are strings. And I think it showed up if you viewed a 5000+ line diff, or a 5000+ line file. (Newer programs like Github also suffer here)

I believe, in Python at that time, += was already optimized to avoid this (just like essentially all JS VMs are). Or you can use the idiom of append() to list and join() after.

But reduce() basically forces the inefficient implementation, and I'm sure this is still true in Python 3.

---

So basically Guido spent a long time debugging a performance problem related to reduce(), and made the decision to eject it, to help users avoid "footguns". I was his officemate at the time, so I recall this, but I wasn't i […]

Ive only used reduce at work half a dozen times and it does raise an eyebrow each time.

But for unioning a bunch of spark dataframes together i think

df = reduce(DataFrame.union, list_of_dfs)

is much nicer than

df, *rest = list_of_dfs

for other in rest:

df = df.union(other)

People just get a bit funny, especially now you have to import it from functools

Map and Filter are nice because they let you reason locally about a single element in isolation. Reduce(Fold) forces you to reason globally about intermediate results. Reduce also forces you to conjure up a "zero" value of the relevant type, which isn't usually difficult but it does constitute some extra mental overhead.

(JS/TS is my main language)

I love reduce()! It's a hammer/nail method for me. Everything looks like a problem solvable by reduce. (I'm often wrong on that, but I quite enjoy learning why by trying).

I really like taking the implementation away from the call site, so that the call site reads

const myNewValue = data.reduce(doSomethingMagic);

(and then `doSomethingMagic` is defined somewhere else). So simple.

I failed a job interview once by using reduce() in a coding test. The reviewer didn't understand why I hadn't used a loop. Loops are easier, for sure, but they sprawl and are open to hacking. They can bring in state from outside the loop. They make the call site long (you always have to read the implementation to learn that you don't need to read it). The same interviewer actively liked to have loop bodies modify the loop conditions (e.g. by taking items out of the source array and decrementing the end condition, so the loop would end earlier). That's the kind of "clever" I find unpredictable and hard to think about. Probably a good thing he rejected me.

Speaking as someone who often tries to reduce my use of reduce by replacing it with map and filter where possible, for me, falling back to reduce is analogous to falling back to a while loop or a for loop: I avoid it if I can.

The problem with reduce is that it can do so much, and therefore it is less clear when reading it quickly what it might be doing.

Even in the world of functional programming, there's an argument to be made that `fold` is a bit of a code smell, in a similar vein as `while` being slightly smelly in an imperative code base. There's good reasons for each to be used, but they are such low level iteration primitives that you might be better off with a higher one (e.g. for loops or iterators in imperative programs; in FP you might reach for monoidic reduces (as opposed to folds where the accumulator is a different type from the list element), monadic traverses, or recursion schemes). Even though you can implement iterators or for loops in terms of while loops, you probably shouldn't, and similar for functional traversals.

In languages like python or Java though, you don't really have access to many of the higher power functional traversals however. So that puts you into a similar kind of bind as working in a language with only while loops