Scala String parser

You can use xml reader in spark as

val df = sqlContext.read
  .format("com.databricks.spark.xml")
  .option("rootTag", "DATA")
  .option("rowTag", "COLUMNS")
  .load("path to the xml file")

And apply some inbuilt functions such as explode and select as

import org.apache.spark.sql.functions._
df.select(explode(col("COLUMN")))
  .select(col("col.*"))
  .filter(col("_DATA_TYPE").isNotNull || col("_ID").isNotNull)
  .select("_DATA_TYPE", "_ID")

which should give you

+----------+--------------------------+
|_DATA_TYPE|_ID                       |
+----------+--------------------------+
|NUMBER    |APPLICATION_ID            |
|VARCHAR2  |DESCRIPTIVE_FLEXFIELD_NAME|
|VARCHAR2  |LANGUAGE                  |
|NUMBER    |CREATED_BY                |
|NUMBER    |LAST_UPDATE_LOGIN         |
+----------+--------------------------+

Now you can convert it to rdd and parse as you want or leave it as dataframe and apply other operations on it.

While this can be done with spark-xml in an easier and more robust way. You can also use the scala.xml API to parse:

import scala.xml._

val rdd = sc.textFile("file.xml").filter( x => x.contains("DATA_TYPE") || x.contains("ID"))

rdd.map(XML.loadString(_) )
   .map(node => ( (node \\ "@ID").text , (node \\ "@DATA_TYPE").text  ) )
   .collect.foreach(println)

// (APPLICATION_ID,NUMBER)
// (DESCRIPTIVE_FLEXFIELD_NAME,VARCHAR2)
// (LANGUAGE,VARCHAR2)
// (CREATED_BY,NUMBER)
// (LAST_UPDATE_LOGIN,NUMBER)