XML to data.frame with multiple files in R

Question

I have a file containing multiple XML declarations which I was able to detect and individually read them from this post: Parseing XML by R always return XML declaration error . The data comes from: https://www.google.com/googlebooks/uspto-patents-applications-text.html.

### read xml document with more than one <?xml declaration in R

lines   <- readLines("pa020829.xml")
start   <- grep('<?xml version="1.0" encoding="UTF-8"?>',lines,fixed=T)
end     <- c(start[-1]-1,length(lines))

get.xml <- function(i) {
  txt <- paste(lines[start[i]:end[i]],collapse="\n")
  # print(i)
  xmlTreeParse(txt,asText=T)
  # return(i)
}
docs <- lapply(1:10,get.xml)

> class(docs)
[1] "list"
> class(docs[1])
[1] "list"
> class(docs[[1]])
[1] "XMLDocument"         "XMLAbstractDocument"

The file docs contains 10 similar documents called docs[[1]], docs[[2]], ... . I managed to extract the root of a single doc and to insert it into a matrix:

root <- xmlRoot(docs[[1]])

d <- rbind(unlist(xmlSApply(root[[1]], function(x) xmlSApply(x, xmlValue))))

However, I need to write code that would automatically retrieve the data of all 10 documents and attach them to a single data frame. I tried the code below but it only retrieves the data of the first document's root and attaches it multiple times to the matrix.

d <- lapply(docs, function(x) rbind(unlist(xmlSApply(root, function(x) xmlSApply(x, xmlValue)))))

I guess I need to change the way I call the root in the function.

Any idea on how to create a matrix with the data from all the documents?

makeyourownmaker · Accepted Answer · 2018-10-12 21:42:43Z

1

The following code will return a matrix containing the data from all the documents:

getXmlInternal <- function(x) {
  rbind(unlist(xmlSApply(xmlRoot(x), function(y) xmlSApply(y, xmlValue))))
}

d <- rbind(lapply(docs, function(x) getXmlInternal(x)))

This fixes the xmlRoot issue you mention by running that command on each of the documents supplied by the lapply command. The lapply command is wrapped in a call to rbind to ensure the output is in a matrix as requested.

The getXmlInternal function is included to make the answer a little more readable.

edited Oct 12, 2018 at 21:42

answered Oct 12, 2018 at 20:43

makeyourownmaker

1,8052 gold badges19 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

hrbrmstr Over a year ago

perhaps explain a bit about what it's doing for the OP? perhaps also add some spacing (there are no extra points on SO for one-liners)

makeyourownmaker Over a year ago

I have added some explanation and reformatted the code a little to improve readability.

Amleto Over a year ago

Thanks you @makeyourownmaker for this. Unfortunately it returns a 1 row x 10 columns matrix with lists inside each row. I need to unlist the characters inside the columns into multiple rows. Can you help me with that?

Collectives™ on Stack Overflow

XML to data.frame with multiple files in R

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related