I would like to implement an Iterator in Java that behaves somewhat like the following generator function in Python:
def iterator(array):
for x in array:
if x!= None:
for y in x:
if y!= None:
for z in y:
if z!= None:
yield z
x on the java side can be multi-dimensional array or some form of nested collection. I am not sure how this would work. Ideas?
Had the same need so wrote a little class for it. Here are some examples:
Generator<Integer> simpleGenerator = new Generator<Integer>() {
public void run() throws InterruptedException {
yield(1);
// Some logic here...
yield(2);
}
};
for (Integer element : simpleGenerator)
System.out.println(element);
// Prints "1", then "2".
Infinite generators are also possible:
Generator<Integer> infiniteGenerator = new Generator<Integer>() {
public void run() throws InterruptedException {
while (true)
yield(1);
}
};
The Generator class internally works with a Thread to produce the items. By overriding finalize(), it ensures that no Threads stay around if the corresponding Generator is no longer used.
The performance is obviously not great but not too shabby either. On my machine with a dual core i5 CPU @ 2.67 GHz, 1000 items can be produced in < 0.03s.
The code is on GitHub. There, you'll also find instructions on how to include it as a Maven/Gradle dependency.
Indeed Java has no yield, but you can now use Java 8 streams. IMO it's really a complicated iterator since it's backed by an array, not a function. Given it's a loop in a loop in a loop can be expressed as a Stream using filter (to skip the nulls) and flatMap to stream the inner collection. It's also about the size of the Python code. I've converted it to an iterator to use at your leisure and printed to demonstrate, but if all you were doing was printing, you could end the stream sequence with forEach(System.out::println) instead of iterator().
public class ArrayIterate
{
public static void main(String args[])
{
Integer[][][] a = new Integer[][][] { { { 1, 2, null, 3 },
null,
{ 4 }
},
null,
{ { 5 } } };
Iterator<Object> iterator = Arrays.stream(a)
.filter(ax -> ax != null)
.flatMap(ax -> Arrays.stream(ax)
.filter(ay -> ay != null)
.flatMap(ay -> Arrays.stream(ay)
.filter(az -> az != null)))
.iterator();
while (iterator.hasNext())
{
System.out.println(iterator.next());
}
}
}
I'm writing about implementation of generators as part of my blog on Java 8 Functional Programming and Lambda Expressions at http://thecannycoder.wordpress.com/ which might give you some more ideas for converting Python generator functions into Java equivalents.
filter(Objects::nonNull) instead of writing a new lambda for that.
I wish Java had generator/yield, but since it doesn't using Iterators is probably your best bet.
In this example I stuck with arrays, but in general I would advise using Iterable Collection instead, eg. List. In the example I show how it's pretty easy to get iterators for arrays though:
package example.stackoverflow;
import com.sun.xml.internal.xsom.impl.scd.Iterators;
import java.util.Arrays;
import java.util.Iterator;
public class ArrayGenerator<T> implements Iterable<T> {
private final T[][][] input;
public ArrayGenerator(T[][][] input) {
this.input = input;
}
@Override
public Iterator<T> iterator() {
return new Iter();
}
private class Iter implements Iterator<T> {
private Iterator<T[][]> x;
private Iterator<T[]> y;
private Iterator<T> z;
{
x = Arrays.asList(input).iterator();
y = Iterators.empty();
z = Iterators.empty();
}
@Override
public boolean hasNext() {
return z.hasNext() || y.hasNext() || x.hasNext();
}
@Override
public T next() {
while(! z.hasNext()) {
while(! y.hasNext()) {
y = Arrays.asList(x.next()).iterator();
}
z = Arrays.asList(y.next()).iterator();
}
return z.next();
}
@Override
public void remove() {
throw new UnsupportedOperationException("remove not supported");
}
}
public static void main(String[] args) {
for(Integer i :
new ArrayGenerator<Integer>(
new Integer[][][]{
{
{1, 2, 3},
{4, 5}
},
{
{},
{6}
},
{
},
{
{7, 8, 9, 10, 11}
}
}
)) {
System.out.print(i + ", ");
}
}
}
There is no yield in Java, so you have to do all these things for yourself, ending up with ridiculous code as this one:
for(Integer z : new Iterable<Integer>() {
@Override
public Iterator<Integer> iterator() {
return new Iterator<Integer>() {
final Integer[][][] d3 =
{ { { 1, 2, 3 }, { 4, 5, 6 }, { 7, 8, 9 } },
{ { 10, 11, 12 }, { 13, 14, 15 }, { 16, 17, 18 } },
{ { 19, 20, 21 }, { 22, 23, 24 }, { 25, 26, 27 } } };
int x = 0;
int y = 0;
int z = 0;
@Override
public boolean hasNext() {
return !(x==3 && y == 3 && z == 3);
}
@Override
public Integer next() {
Integer result = d3[z][y][x];
if (++x == 3) {
x = 0;
if (++y == 3) {
y = 0;
++z;
}
}
return result;
}
@Override
public void remove() {
throw new UnsupportedOperationException();
}
};
}
}) {
System.out.println(z);
}
But if your sample would have more than one single yield it would end up even worse.
The translation from Python-style generators to Java-style iterators can be automated. If you're willing to accept code generation in your build process, you might be interested in this prototype tool that does the translation for you:
Assuming the Python data structure you describe in your question can be described using the following Java type:
List<List<List<T>>>;
and you want to use it in an operation like this:
for (T z : iterator(array)) {
// do something with z
}
If so, then one can implement your Python iterator() pretty trivially using Java 8 streams:
public <T> Iterable<T> iterator(List<List<List<T>>> array) {
return array.stream()
.filter(Objects::nonNull) // -> emits stream of non-null `x`s
.flatMap(x -> x.stream()).filter(Objects::nonNull) // -> emits […] `y`s
.flatMap(y -> y.stream()).filter(Objects::nonNull) // -> emits […] `z`s
.collect(Collectors.toList()); // get list of non-null `z`s to iterate on
}
Of course, you can not collect the results and output a stream for further streamed processing (people tell me that it is a good idea):
public <T> Stream<T> streamContent(List<List<List<T>>> array) {
return array.stream()
.filter(Objects::nonNull) // -> emits stream of non-null `x`s
.flatMap(x -> x.stream()).filter(Objects::nonNull) // -> emits […] `y`s
.flatMap(y -> y.stream()).filter(Objects::nonNull); // -> emits […] `z`s
}
// ...
streamContent(array).forEach(z -> {
// do something with z
});
No, Java does not have "generators" or "yield" per-se, but the same functionality is available by using the Observer Pattern. This is enhanced when using a modern implementation like RxJava. Your code would subscribe to the Obserable and whenever it tries to read the next value from the Observable it would cause it "generate" it's next value. The Observable can maintain it's own state just like a generator for Python or JavaScript. When there are no new values to be read, the "next()" method will block waiting on new data to be available. A good example of this can be found HERE.
Very late to the game but I wanted to offer my solution as a reference. https://github.com/tsi-software/JavaGenerator A Java class that allows you to write "Generator" code as similarly as possible to Python and C#.
Using Seq from a new library, which has implemented generator in Java, you can write your own generator functions just like in Python
public Seq<Integer> generate(List<List<List<Integer>>> array) {
return c -> {
for (List<List<Integer>> x : array) {
if (x != null) {
for (List<Integer> y : x) {
if (y != null) {
for (Integer z : y) {
if (z != null) {
c.accept(z);
}
}
}
}
}
}
};
}
Then you can manipulate/collect the returned seq just like a normal Java stream.
You can use the iterator of a stream to accomplish this.
// Save the iterator of a stream that generates fib sequence
Iterator<Integer> myGenerator = Stream
.iterate(new Integer[]{ 1, 1 }, x -> new Integer[] { x[1], x[0] + x[1] })
.map(x -> x[0]).iterator();
// Print the first 5 elements
for (int i = 0; i < 5; i++) {
System.out.println(myGenerator.next());
}
System.out.println("done with first iteration");
// Print the next 5 elements
for (int i = 0; i < 5; i++) {
System.out.println(myGenerator.next());
}
Output:
1
1
2
3
5
done with first iteration
8
13
21
34
55
(z for x in array if x is not None for y in x if y is not None for z in y if z is not None)