Java 8 parallel stream of list of method param

Question

I have a method:

invokList(List<Object> list);

This method is inside a jar and I have no access to the source code of it. So for that, I need to execute the invokList in a parallel way, can someone help for this?

The idea is to split the list to many lists and execute invokList in parallel.

I have made this example:

            import java.util.Arrays;
            import java.util.Collections;
            import java.util.List;

            public class Test {

                public static void main(String[] args) {
                    List<Integer> list = Arrays.asList(1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20);
                    list.parallelStream()
                            .map(Collections::singletonList)
                            .forEach(Test::invokList);
                }

                public static void invokList(List<Integer> list) {
                    try {
                        Thread.sleep(100);
                        System.out.println("The Thread :" + Thread.currentThread().getName() + " is processing this list" + list);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                }
            }

if you execute the test above the result is processing one by one element i want something like batch processing — Gogo
– Gogo, Commented Mar 11, 2019 at 16:32
Don’t write this in a comment, add it into your question. And note that telling about your wishes still is a factual statement, not a question. See How to Ask — Holger
– Holger, Commented Mar 11, 2019 at 16:34

MikeFHay · Accepted Answer · 2019-03-11 16:13:02Z

6

Guava has methods Lists.partition and Iterables.partition that do something like what you're asking. Say you have a large List and want to process it in chunks of 5, you could do:

int batchSize = 5;
Lists.partition(list, batchSize)
   .parallelStream()
   .forEach(batch -> invokeList(batch));

answered Mar 11, 2019 at 16:13

MikeFHay

9,1704 gold badges36 silver badges55 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Adrian · Accepted Answer · 2019-03-12 07:50:30Z

if you don't want to bring additional dependencies like Guava, then you can write a collector which divide your list in chunks:

static <T> Collector<T, List<List<T>>, List<List<T>>> toChunks(int size) {
    return Collector.of(ArrayList::new, (list, value) -> {
        List<T> chunk = list.isEmpty() ? null : list.get(list.size() - 1);
        if (chunk == null || chunk.size() == size) {
            chunk = new ArrayList<>(size);
            list.add(chunk);
        }
        chunk.add(value);
    }, (list1, list2) -> {
        throw new UnsupportedOperationException();
    });
}

and then call it as follows:

 List<Integer> list = Arrays.asList(1,26,17,18,19,20);
 list.stream().collect(toChunks(5))
              .parallelStream()
              .forEach(System.out::println);

Kartik · Accepted Answer · 2019-03-12 05:53:28Z

1

Looks very verbose, but you can try the following. The runAsync() method will make the list chunks run in parallel.

private void test(List<Object> list, int chunkSize) throws ExecutionException, InterruptedException {
    AtomicInteger prev = new AtomicInteger(0);
    List<CompletableFuture> futures = new ArrayList<>();
    IntStream.range(1, (int) (chunkSize * (Math.ceil(Math.abs(list.size() / (double) chunkSize)))))
            .filter(i -> i % chunkSize == 0 || i == list.size())
            .forEach(i -> {
                List<Object> chunk = list.subList(prev.get(), i);
                futures.add(CompletableFuture.runAsync(() -> invokeList(chunk)));
                prev.set(i);
            });
    CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).get();
}

private void invokeList(List<Object> list) {
    System.out.println("Invoked for: " + list);
}

I ran it for a list of 30 integers, with a chunk size of 5 like this:

public static void main(String[] args) throws ExecutionException, InterruptedException {
    List<Object> list = IntStream.range(0, 30).mapToObj(i1 -> (Object) String.valueOf(i1)).collect(Collectors.toList());
    int chunkSize = 5;
    new Test().test(list, chunkSize);
}

Output:

Invoked for: [15, 16, 17, 18, 19]
Invoked for: [0, 1, 2, 3, 4]
Invoked for: [5, 6, 7, 8, 9]
Invoked for: [10, 11, 12, 13, 14]
Invoked for: [20, 21, 22, 23, 24]

answered Mar 12, 2019 at 5:53

Kartik

7,9456 gold badges33 silver badges55 bronze badges

1 Comment

Kartik Over a year ago

@MoradAngel You can use handle() method, like I showed in this answer

Collectives™ on Stack Overflow

Java 8 parallel stream of list of method param

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related