How to create RDD(Sub-Array[String]) from RDD(Array[String])?

Question

My initial RDD; Array[Array[String]], looks something like:

    a b h c ....... d e x q
    d r d x ....... e q g m
    f e u t ....... q t g y
    a b h c ....... d e x q
    d r d x ....... e q g m
    f e u t ....... q t g y
    ....
    ....

New RDD; Array[Array[String]], that contains:

u t ...... q t
h c ...... d e

Any idea on how to work on it? Size of the array is large, so one could want the sub-array of any size. Thanks in advance.

If you have function bigToSmallArray then you just have to do rdd ofSmallArrays = rddOfBigArrays.map(bigToSmallArray) — abalcerek
– abalcerek, Commented Dec 25, 2015 at 23:10

zero323 · Accepted Answer · 2015-12-26 05:44:04Z

1

It looks like all you need here is just drop*:

rdd.map(_.drop(2).dropRight(2))

And if you're in doubt it is usually useful to check Scala collections docs.

answered Dec 26, 2015 at 5:44

community wiki

zero323

Sign up to request clarification or add additional context in comments.

1 Comment

user3180835 Over a year ago

This works for the column. What if i want to select the rows, for example row(2) to row(8)?

Collectives™ on Stack Overflow

How to create RDD(Sub-Array[String]) from RDD(Array[String])?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related