Parsing Large XML File Using Sax

Question

i am trying to parse an xml document, after searching i found out that sax is the best choice, but the document is very large (1.5 GB) waited like 7 minutes but its still running my question is, is that normal ? or i can do better ?

public static void main(String argv[]) {

    try {

        SAXParserFactory factory = SAXParserFactory.newInstance();
        SAXParser saxParser = factory.newSAXParser();

        DefaultHandler handler = new DefaultHandler() {

            int c = 0;
            boolean id = false;
            boolean value = false;
            boolean orgin = false;
            boolean note = false;

            @Override
            public void startElement(String uri, String localName, String eName,
                    Attributes attributes) throws SAXException {

                if (eName.equalsIgnoreCase("ID")) {
                    id = true;
                }

                if (eName.equalsIgnoreCase("VALUE")) {
                    value = true;
                }

                if (eName.equalsIgnoreCase("ORGIN")) {
                    orgin = true;
                }

                if (eName.equalsIgnoreCase("NOTE")) {
                    note = true;
                }

            }

            @Override
            public void endElement(String uri, String localName,
                    String eName) throws SAXException {

            }

            @Override
            public void characters(char ch[], int start, int length) throws SAXException {

                if (id) {
                    System.out.println(new String(ch, start, length));
                    id = false;
                    System.out.println("record num : "+c++);
                }

                if (value) {
                    System.out.println(new String(ch, start, length));
                    value = false;
                }

                if (orgin) {
                    System.out.println(new String(ch, start, length));
                    orgin = false;
                }

                if (note) {
                    System.out.println(new String(ch, start, length));
                    note = false;
                }

            }

        };

        saxParser.parse("./transactions.xml", handler);

    } catch (Exception e) {
        e.printStackTrace();
    }

}

Take a look at this : stackoverflow.com/questions/3411773/… — Karbos 538
– Karbos 538, Commented Sep 1, 2015 at 12:46
@NathanHughes i am using this for the first time, any suggestions ? — Abdullah Asendar
– Abdullah Asendar, Commented Sep 1, 2015 at 13:10
my suggestions: stop getting bad code from mkyong, read the Oracle documentation, and follow sharonbn's advice. — Nathan Hughes
– Nathan Hughes, Commented Sep 1, 2015 at 13:12

Sharon Ben Asher · Accepted Answer · 2015-09-01 12:44:26Z

2

You can save some time by changing equalsIgnoreCase to equals (unless you really encounter "ValuE" and "valUE" and "VaLuE" ...)
The printing is probably taking most of the time. IO operations are usually the bottleneck

answered Sep 1, 2015 at 12:44

Sharon Ben Asher

14.5k6 gold badges35 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Abdullah Asendar Over a year ago

i was printing to test, i am going to save the data to database, do you have any suggestion for the database ? time is important

Sharon Ben Asher Over a year ago

most database engines support batch insert/update (inserting of multiple rows in one statement) - use it

Sharon Ben Asher Over a year ago

the last tip I can give you from my personal experience is that you need to be aware that the parser may call characters() method multiple times for the same xml element and you need to concatanate the result to get the full text. you can then do the processing (DB call etc) in endElement() see details here stackoverflow.com/questions/4567636/…

Sharon Ben Asher Over a year ago

if you mean instance variable then yes

Community · Accepted Answer · 2017-05-23 12:01:37Z

1

If you parse such a huge file you should use Stax instead of Sax. With Stax you can skip parts of your file which makes it faster and it's faster.

StAX is a "pull" type of API. As discussed, there are Cursor and Event Iterator APIs. There are both reading and writing sides of the API. It is more developer friendly than SAX. StAX, like SAX, does not require an entire document to be held in memory. However, unlike SAX, an entire document need not be read. Portions can be skipped. This may result in even improved performance over SAX.

(DOM vs SAX XML parsing for large files)

edited May 23, 2017 at 12:01

CommunityBot

11 silver badge

answered Sep 1, 2015 at 12:53

Christian

22.4k11 gold badges84 silver badges107 bronze badges

Collectives™ on Stack Overflow

Parsing Large XML File Using Sax

2 Answers 2

4 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related