3 Reasons why You Shouldn’t Replace Your for-loops by Stream.forEach()

Awesome! We’re migrating our code base to Java 8. We’ll replace everything by functions. Throw out design patterns. Remove object orientation. Right! Let’s go!

Wait a minute

Java 8 has been out for over a year now, and the thrill has gone back to day-to-day business.

A non-representative study executed by baeldung.com from May 2015 finds that 38% of their readers have adopted Java 8. Prior to that, a late 2014 study by Typsafe had claimed 27% Java 8 adoption among their users.

What does it mean for your code-base?

Some Java 7 -> Java 8 migration refactorings are no-brainers. For instance, when passing a Callable to an ExecutorService:

ExecutorService s = ...

// Java 7 - meh...
Future<String> f = s.submit(
    new Callable<String>() {
        @Override
        public String call() {
            return "Hello World";
        }
    }
);

// Java 8 - of course!
Future<String> f = s.submit(() -> "Hello World");

The anonymous class style really doesn’t add any value here.

Apart from these no-brainers, there are other, less obvious topics. E.g. whether to use an external vs. an internal iterator. See also this interesting read from 2007 by Neil Gafter on the timeless topic:
http://gafter.blogspot.ch/2007/07/internal-versus-external-iterators.html

The result of the following two pieces of logic is the same

List<Integer> list = Arrays.asList(1, 2, 3);

// Old school
for (Integer i : list)
    System.out.println(i);

// "Modern"
list.forEach(System.out::println);

I claim that the “modern” approach should be used with extreme care, i.e. only if you truly benefit from the internal, functional iteration (e.g. when chaining a set of operations via Stream’s map(), flatMap() and other operations).

Here’s a short list of cons of the “modern” approach compared to the classic one:

1. Performance – you will lose on it

Angelika Langer has wrapped up this topic well enough in her article and the related talk that she’s giving at conferences:

Java performance tutorial – How fast are the Java 8 streams?

In many cases, performance is not critical, and you shouldn’t do any premature optimisation – so you may claim that this argument is not really an argument per se. But I will counter this attitude in this case, saying that the overhead of Stream.forEach() compared to an ordinary for loop is so significant in general that using it by default will just pile up a lot of useless CPU cycles across all of your application. If we’re talking about 10%-20% more CPU consumption just based on the choice of loop style, then we did something fundamentally wrong. Yes – individual loops don’t matter, but the load on the overall system could have been avoided.

Here’s Angelika’s benchmark result on an ordinary loop, finding the max value in a list of boxed ints:

ArrayList, for-loop : 6.55 ms
ArrayList, seq. stream: 8.33 ms

In other cases, when we’re performing relatively easy calculations on primitive data types, we absolutely SHOULD fall back to the classic for loop (and preferably to arrays, rather than collections).

Here’s Angelika’s benchmark result on an ordinary loop, finding the max value in an array of primitive ints:

int-array, for-loop : 0.36 ms
int-array, seq. stream: 5.35 ms

Premature optimisation is not good, but cargo-culting the avoidance of premature optimisation is even worse. It’s important to reflect on what context we’re in, and to make the right decisions in such a context. We’ve blogged about performance before, see our article Top 10 Easy Performance Optimisations in Java

2. Readability – for most people, at least

We’re software engineers. We’ll always discuss style of our code as if it really mattered. For instance, whitespace, or curly braces.

The reason why we do so is because maintenance of software is hard. Especially of code written by someone else. A long time ago. Who probably wrote only C code before switching to Java.

Sure, in the example we’ve had so far, we don’t really have a readability issue, the two versions are probably equivalent:

List<Integer> list = Arrays.asList(1, 2, 3);

// Old school
for (Integer i : list)
    System.out.println(i);

// "Modern"
list.forEach(System.out::println);

But what happens here:

List<Integer> list = Arrays.asList(1, 2, 3);

// Old school
for (Integer i : list)
    for (int j = 0; j < i; j++)
        System.out.println(i * j);

// "Modern"
list.forEach(i -> {
    IntStream.range(0, i).forEach(j -> {
        System.out.println(i * j);
    });
});

Things start getting a bit more interesting and unusual. I’m not saying “worse”. It’s a matter of practice and of habit. And there isn’t a black/white answer to the problem. But if the rest of the code base is imperative (and it probably is), then nesting range declarations and forEach() calls, and lambdas is certainly unusual, generating cognitive friction in the team.

You can construct examples where an imperative approach really feels more awkward than the equivalent functional one, as exposed here:

But in many situations, that’s not true, and writing the functional equivalent of something relatively easy imperative is rather hard (and again, inefficient). An example could be seen on this blog in a previous post:
https://blog.jooq.org/2015/09/09/how-to-use-java-8-functional-programming-to-generate-an-alphabetic-sequence/

In that post, we generated a sequence of characters:

A, B, ..., Z, AA, AB, ..., ZZ, AAA

… similar to the columns in MS Excel:

MS Excel column names

The imperative approach (originally by an unnamed user on Stack Overflow):

import static java.lang.Math.*;
 
private static String getString(int n) {
    char[] buf = new char[(int) floor(log(25 * (n + 1)) / log(26))];
    for (int i = buf.length - 1; i >= 0; i--) {
        n--;
        buf[i] = (char) ('A' + n % 26);
        n /= 26;
    }
    return new String(buf);
}

… probably outshines the funcitonal one on a conciseness level:

import java.util.List;
 
import org.jooq.lambda.Seq;
 
public class Test {
    public static void main(String[] args) {
        int max = 3;
 
        List<String> alphabet = Seq
            .rangeClosed('A', 'Z')
            .map(Object::toString)
            .toList();
 
        Seq.rangeClosed(1, max)
           .flatMap(length ->
               Seq.rangeClosed(1, length - 1)
                  .foldLeft(Seq.seq(alphabet), (s, i) -> 
                      s.crossJoin(Seq.seq(alphabet))
                       .map(t -> t.v1 + t.v2)))
           .forEach(System.out::println);
    }
}

And this is already using jOOλ, to simplify writing functional Java.

3. Maintainability

Let’s think again of our previous example. Instead of multiplying values, we divide them now.

List<Integer> list = Arrays.asList(1, 2, 3);

// Old school
for (Integer i : list)
    for (int j = 0; j < i; j++)
        System.out.println(i / j);

// "Modern"
list.forEach(i -> {
    IntStream.range(0, i).forEach(j -> {
        System.out.println(i / j);
    });
});

Obviously, this is asking for trouble, and we can immediately see the trouble in an exception stack trace.

Old school

Exception in thread "main" java.lang.ArithmeticException: / by zero
	at Test.main(Test.java:13)

Modern

Exception in thread "main" java.lang.ArithmeticException: / by zero
	at Test.lambda$1(Test.java:18)
	at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
	at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557)
	at Test.lambda$0(Test.java:17)
	at java.util.Arrays$ArrayList.forEach(Arrays.java:3880)
	at Test.main(Test.java:16)

Wow. Were we just…? Yes. These are the same reasons why we’ve had performance issues in item #1 in the first place. Internal iteration is just a lot more work for the JVM and the libraries. And this is an extremely easy use-case, we could’ve displayed the same thing with the generation of AA, AB, .., ZZ series.

From a maintenance perspective, a functional programming style can be much harder than imperative programming – especially when you blindly mix the two styles in legacy code.

Conclusion

This is usually a pro-functional programming, pro-declarative programming blog. We love lambdas. We love SQL. And combined, they can produce miracles.

But when you migrate to Java 8 and contemplate using a more functional style in your code, beware that FP is not always better – for various reasons. In fact, it is never “better”, it is just different and allows us to reason about problems differently.

We Java developers will need to practice, and come up with an intuitive understanding of when to use FP, and when to stick with OO/imperative. With the right amount of practice, combining both will help us improve our software.

Or, to put it in Uncle Bob’s terms:

The bottom, bottom line here is simply this. OO programming is good, when you know what it is. Functional programming is good when you know what it is. And functional OO programming is also good once you know what it is.

http://blog.cleancoder.com/uncle-bob/2014/11/24/FPvsOO.html

41 thoughts on “3 Reasons why You Shouldn’t Replace Your for-loops by Stream.forEach()

    • You’re right, but that is on purpose (somewhat). The thing here is that developers will stop caring about where that forEach() method is declared / implemented. Often, this doesn’t matter, but sometimes it does, from a performance perspective.

  1. I’m going to go ahead and refute every point you’ve made here.

    1. Performance: First, if performance is your main concern, streams are obviously not the route you are going to choose. Even the “enhanced” for-loop is slower, in comparison, to a straight

    for(int i =0; i {
        IntStream.range(0, i).forEach(j -> {
            System.out.println(i / j);
        });
    });
    

    Did you miss flatmap() in the new Stream API? Ignoring the trivial example, let’s go with something more reasonable.

    “Let’s say I have a List where each TimeSeries is essentially a Map. I want to get a list of all dates for which at least one of the time series has a value. flatMap to the rescue:

    list.stream().parallel()
        .flatMap(ts -> ts.dates().stream()) // for each TS, stream dates and flatmap
        .distinct()                         // remove duplicates
        .sorted()                           // sort ascending
        .collect(toList());
    

    Not only is it readable, but if you suddenly need to process 100k elements, simply adding parallel() will improve performance without you writing any concurrent code.” (thanks to assylias from Stackoverflow for this)

    3. Maintainability: I would almost concede this point if I hadn’t worked with modern Java frameworks like Spring and Hibernate. Stacktraces like the one you show in this article are much worse and more commonplace that even I (as someone who works with Java everyday) would like to admit. There’s eventually a point where you (as an engineer) just have to “deal with it” and move on.

    • I’m going to go ahead and refute every point you’ve made here.

      You’ll be on a roll 🙂

      Did you miss flatmap() in the new Stream API?

      flatMap() clearly underlines my point #2. By using flatmap in such a case, in order to retain a (i, j) tuple, I’d have to introduce a third party library to add support for tuples in streams. It would further add cognitive friciton to an otherwise trivial algorithm – and short of value types, perhaps even excessive pressure on the GC, for absolutely no added value. At least in this example.

      Ignoring the trivial example, let’s go with something more reasonable.

      Your example is also trivial 🙂 The imperative solution would probably loop over the list, maintain a temporary HashSet for the dates produced by ts.dates() and outperform even your parallel solution. Parallel is really overrated, but that’s an entirely different discussion.

      There’s eventually a point where you (as an engineer) just have to “deal with it” and move on.

      Heh 🙂 We’ve reached that point in our industry, haven’t we… Well, the winners are the tool vendors. Eventually, they’ll “refactor” the display of stack traces to more meaningful things.

  2. Keep calm and use Javaslang.

    This is equivalent to your example (“the” functional one):

    Stream Σ =
        Stream.rangeClosed('A', 'Z').map(String::valueOf);
    
    Σ.appendSelf(s -> s.crossProduct(Σ).map(t -> t._1 + t._2))
        .take(max).stdout();
    

    I use the style, the language, the pattern, … that fits best regarding simplicitry, readability, correctness, …

    Beware of premature optimization. First write correct, readable, concise code. Then identify bottlenecks. Often more readable but slower variants don’t affect the overall time/mem performance at all.

    It depends, as always.

    • I’m sure that naming variables Σ will help you make tons of friends in your team 🙂

      The important aspect here is that by changing the general style from imperative to functional, we increase the overall load of the system. There’s not much you can do against that, once the style ship has sailed. We’re then not talking about bottlenecks in terms of performance, but about load.

  3. I’m sure that naming variables Σ will help you make tons of friends in your team 🙂

    Good example for optimization. It is ugly but now the code lines fit into 70 columns 🙂

    We’re then not talking about bottlenecks in terms of performance, but about load.

    And here comes scalability into play. Functional code is scalable, imperative in general not.

        • The medium-size systems I used to “scale” so far only used very classic legacy code (granted, pretty stateless stuff), and SQL. I’m not sure what kind of blog post you’re expecting…?

          • I would start with an example, where the increased load you mentioned leads to significant problems. In a second step there could be an example how to balance the load to keep the system stable/responsive.

            But while writing this I think I’ve yet not fully understood which kind of _overall system load_ you mean. Maybe database centric – managed via statistics, execution plans et al.

            • Aha, I see what you mean. Well, that particular blog post is in the pipeline, don’t worry 🙂 It’s going to be a big rant about this premature optimisation cargo cult, which is often just an excuse for people to stop using their brains in early project phases when it really matters. It won’t be generally comparing FP / imperative though, or mutable / immutable. That’s not the real issue here.

              But I like your comment. It goes in the same direction. You’re suspecting that it might have to do with execution plans, which is again targeting performance of individual queries, rather than the overall load.

              Will be a fun post, attracting lots of haters from reddit 🙂

          • “Anger, fear, aggression; the dark side of the Force are they. Once you start down the dark path, forever will it dominate your destiny.”

            – Master Yoda

  4. Hi Lukas,

    I just wanted to tell you that I like your post 🙂

    Lambdas are great but as programmers we still have to use our brain and ask ourselves “Am I using the right tool for the job?”.

  5. I found another problem with IntStream foreach. I noticed this with threading code.

    private int count = 0;
    	
    	public synchronized void increment() {
    		count++;
    	}
    	
    	public int getCount() {
    		return count;
    	}
    	
    	public static void main(String[] args) {
    		App app = new App();
    		app.doWork();
    		
    		try {
    			TimeUnit.SECONDS.sleep(2);
    		} catch (InterruptedException e) {
    			e.printStackTrace();
    		}
    		
    		System.out.println("Count is " + app.getCount());
    	}
    	
    	public void doWork() {
    		Thread t1 = new Thread(() -> {
    			IntStream.range(0, 10_000)  //  increment());
    			System.out.println("Count is " + count);
    		});
    		Thread t2 = new Thread(() -> {
    			IntStream.range(0, 10_000) //  increment());
    			System.out.println("Count is " + count);
    		});
    //		Thread t3 = new Thread(() -> {
    //			for (int i = 0; i < 10_000; i++)
    //				increment();
    //			System.out.println("Count is " + count);
    //		});
    		t1.start();
    		t2.start();
    		
    	}
    

    when I start, I get count 10 and 20k and in main it's 0. Because the streams are lazy, none of it makes effect. So it can not be used general loop. If anyone has a solution for this, couse I like this loop style more than for loop, answer 🙂
    I'm still learning and performance is not the issue for me, I've been learning streams for 2-3 weeks and I love it! But I'm sad that this isn't working :/

    • I’m not sure what you’re trying to say. Your IntStream.range(0, 10_000) stream is never “consumed”, i.e. it never runs a terminal operation. How should it work?

      Also, I’d personally use an AtomicInteger which uses volatile and sun.misc.Unsafe internally, rather than synchronizing on a monitor…

  6. wow, this article is very inaccurate. You don’t lose performance with stream, in any case you may gain performance if your stream is not in memory. Also streams support parallel operations right out of the box if you have multiple cores in your system.

    And the code is much more readable with streams. Explain to me how this

    List list = getSomeList();
    List = resultList = new ArrayList();
    
    for (Something x : list) {
        if (someCondition(x) && someOtherCondition(x)) {
            doSomething(x);
            resultList.add(convert(x));
        }
    }
    

    is better than this

    List list = getSomeList();
    List = list.stream()
        .filter(this::someCondition)
        .filter(this::someOtherCondition)
        .forEach(this::doSomething)
        .map(this::convert)
        .collec(Collections.toList());
    
  7. Think it’s not forbidden to use functions in functional programs. The “cell headers problem” is a kind of counting. Thus, we can use Horner’s method in form of a function to map (column) numbers to their string representation. Here’s my solution:

      public char charval(int i){
        if (i==0) return '0';
        return (char)( (int) 'A' + i - 1);    
      } 
      
      private String hornerhelp(int n, int base, String s){
        if (n==0) return s;
        int rest = n % base;
        return hornerhelp(n/base, base, charval(rest)+s);
      }
      
      public String horner(int n, int base){
        return hornerhelp(n, base, "");
      }
      
      public static void main(String[] args){
        StreamTest test = new StreamTest();
        IntStream.range(1, 2000)
          .boxed()                                  // convert to Stream
          .map(k->test.horner(k, 27))    
          .filter(k->!k.contains("0"))
          .forEach(System.out::println);
     ...
    

    I find it a quite readable (ad hoc) version including good things of both worlds. Did not check performance.
    K

    • Yeah, interesting. How about taking it a step further and make horner a higher order function?

      public IntFunction<String> horner(int base) {
        return n -> hornerhelp(n, base, "");
      }
      

      The stream would then look like this:

      IntStream.range(1, 2000)
               .mapToObj(test.horner(27))
               .filter(k -> !k.contains("0"))
               .forEach(System.out::println)
      
      • Very nice. Here a version of the horner function with all the helpers hidden as inner functions (interesting point: recursive lambda definition):

        public IntFunction horner(int base) {

        Function charval =
        i -> i == 0 ? ‘0’ : (char) ((int) ‘A’ + i – 1);

        Function3 hornerHelper =
        (h, m, s) ->
        {
        if (m == 0) {
        return s;
        } else {
        int rest = m % base;
        return (String) h.apply(h, m / base, charval.apply(rest) + s);
        }
        };
        return n -> hornerHelper.apply(hornerHelper, n, “”);
        }

        To make this work, a functional interface for functions with 3 arguments is needed:
        @FunctionalInterface
        interface Function3 {
        public R apply(U u, V v, W w);
        }

  8. Started to dislike my previous solution because you do not have control about how many elements you get. I started with 2000, but many of them (all containing 0) are filtered out. How many are these?
    Here a more combinatorial solution. I prefer this one because in my opinion it shows nicely the power of the functional approach and recursion smelling not more like Java than absolutely necessary. No forEach (except test) and no extra libraries are used.

    seed: produces an initial sequence A,B, C… (n characters)

      public List seed(int n){
        List res = IntStream.range(0, n)
          .boxed()
          .map(i->(char)('A'+i))
          .map(c->Character.toString(c))
          .collect(Collectors.toList());
        return res;
      }
      
    // combines a string with the seed list, i.e. produces a stream which has all the seed elements appended to s.
    
      public Stream combine(String s, List seed){
        return seed.stream().map(t->s+t);
      }
      
     // constructs the sequence recursively. To each element of the current sequence each seed element is appended (by combine method). The resulted stream of streams is flattened and appended to the current list. 
      public List construct(List current, List seed, int depth){
        if (depth==0){
          return current;
        } else {      
          current.addAll(construct(current
            .stream()
            .map(c->combine(c, seed))
            .flatMap(s->s)
            .collect(Collectors.toList()),seed, depth-1));
          return current;
        }
      }
    
     // test
      public static void main(String[] args){
        StreamTest test = new StreamTest();
        List seed = test.seed(5); // A-E
        List res = test.construct(seed, seed, 2); // up to 3(!) places
        res.forEach(System.out::println);
      }
    </pre
  9. I agree with this point “forEach in bad in stream”. But not in the way you are claiming but because it encouraging mutation.

    If you use foreach in a stream. I bet you are mutating/or producing side effect. Mutating/side effects are evil when you want to run your program concurrent/parallel.

    Since you quoted uncle bob. Please watch his speech on Functional programming https://www.youtube.com/watch?v=7Zlp9rKHGD4

    In a way you are defending/arguing you are not able to mutate but my counter is Functional programming is all about not changing the state/mutation.

    Never whine that you are not able to cut the tree with a chainsaw like you did with an axe. chainsaw should be used differently.

    • Regarding chainsaws and axes, of course, you’re aware of Lukas’s Law of Analogies, right? 🙂

      The article wasn’t about functional programming per se, but about replacing for loops (external) by forEach loops (internal), a practice that is currently quite widely used. Both loops have side-effects and are thus not strictly functional. Nor is parallelism essential to this argument.

      So, in fact, I’m not sure what your argument really is. Care to discuss chainsaws, trees, and axes instead?

      • I afraid that, your article may mislead the new java 8 users to move away from streams.

        Performance: The compiler/runtime can change and find a better algorithm which can run faster. Its not programmers job anymore to control the performance in code level. As a programmer all we need to worry about was the clean code and less headache while debugging. Declarative style does that. stream is all about declarative style. And stream can change its underlying algorithm any time to improve the performance.

        Readability : Familiar != readable. Imperative is not readable it is familiar to you because you coded in java for years. Readable means that even a layman is able to read and understand what is the code is all about. Stream is readable with the method name such as map, filter, reduce, collect etc

        Maintainability : Well! I am able to read and understand the stack trace in both the cases. Moreover Java 9 improves to understand the stack better way http://www.javaworld.com/article/3188289/core-java/java-9s-other-new-enhancements-part-5-stack-walking-api.html . As I said the language can improve itself from its current behaviour.

        Having said that, I still stand with this point “the peek and forEach methods in stream is bad because it encourages immutability, not because of the performance issue”. Programmers have to move towards functional in other words immutable world. As quoted by uncle bob “The state is failed” and immutability is the rescuer.

        • I afraid that, your article may mislead the new java 8 users to move away from streams.

          I’m flattered to hear that you found my article so powerful, but I sincerely doubt that this will happen. People will think for themselves. They will meh at this article and move on. Cheers!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s