3

I'm looking for a way to implement aggregation/fold function on a window that also have a state. I understand how to aggregate on a window, and how to use key/global state - but not both.

Just to be clear, when I say a window with state - I mean that the state should be initialized (nullified) every time the window is changed/moved.

For example: I want to count the number of events keyed by event type every 5 minutes. But in addition to event type (which is the window key) the event has some id field - and I would like to count each id only once - so I need to save a state of all the ids I've already counted in that window.

Is there a simple way to do this in Flink?

Improve this question

1 Answer 1

Reset to default
2

Flink has a RichReduceFunction, which will give you access to state that is global across all windows for a given key. If you need per-window state, see [Flink-5929] which will be part of Flink 1.3.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

thank you for your answer. I'm not sure I fully understand, but does Flink-5929 will implement a state reset whenever one window period is finished and another started?
Yes, the window state available via Flink-5929 is scoped to a specific window instance and a specific key.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.