You can do it as follows, first get all columns that should be included in the matrix:
import org.apache.spark.sql.functions._
val matrixColumns = df.columns.filter(_.startsWith("A")).map(col(_))
Then convert the dataframe to an RDD[Vector]. Since the vector need to contain doubles this conversion need to be done here too.
import org.apache.spark.mllib.linalg.Vectors
import org.apache.spark.mllib.linalg.distributed.{IndexedRow, IndexedRowMatrix}
val rdd = df.select(array(matrixColumns:_*).as("arr")).as[Array[Int]].rdd
.zipWithIndex()
.map{ case(arr, index) => IndexedRow(index, Vectors.dense(arr.map(_.toDouble)))}
Then convert the rdd to an IndexedRowMatrix which can be converted, if required, to a local Matrix:
val dm = new IndexedRowMatrix(rdd).toBlockMatrix().toLocalMatrix()
For smaller matrices that can be collected to the driver there is an easier alternative:
val matrixColumns = df.columns.filter(_.startsWith("A")).map(col(_))
val arr = df.select(array(matrixColumns:_*).as("arr")).as[Array[Int]]
.collect()
.flatten
.map(_.toDouble)
val rows = df.count().toInt
val cols = matrixColumns.length
// It's necessary to reverse cols and rows here and then transpose
val dm = Matrices.dense(cols, rows, arr).transpose()
