From tensorflow offical doc, it says
For example: N = 2, source[0] is 'hello world' and source[1] is 'a b c', then the output will be st.indices = [0, 0; 0, 1; 1, 0; 1, 1; 1, 2] st.shape = [2, 3] st.values = ['hello', 'world', 'a', 'b', 'c']
What if I want something like [['hello', 'world'], ['a','b','c']], how can I get this?
Thanks.
Use tf.map_fn to map your batch onto the function tf.string_split.
https://www.tensorflow.org/api_docs/python/tf/map_fn
The map function will split your batch along the first dimension (your batch size, N as referenced by the documentation in your question), then it will pass each of the samples to tf.string_split individually, each of which will return ['hello', 'world'] and ['a', 'b', 'c'] respectively. Then the map function will recombine the individual results into an array which will result in [['hello', 'world'], ['a', 'b', 'c']] as desired.
tf.string_split() for a tensor having shape [None, None]?
map_fn seems to expect the input and output of the mapper to have the same shape....