how to faster read text file to ArrayList on android

Question

I have a problem with fast read txt file to ArrayList. If I want read a file size 0,9MB, I must wait 5min. If files size is 34MB (partialy, because android does not accept larger than 1MB files), it's completely not working. I think that process should be max few second.

This is a code:

String word; 
public ArrayList<String> dictionary = new ArrayList<String>();

public void setup() 
{

  try {
      AssetManager assetManager = getAssets();
      InputStream inputf;
      inputf = assetManager.open("dict_1.txt");
      reader = new BufferedReader(new InputStreamReader(inputf));

      word = " ";      
      while(word != null)
      {
        word =  reader.readLine();

        if (word != null)
          dictionary.add(word);
      }
      if(reader.equals("null")) println("No file found");

    } catch (NullPointerException e) {
    e.printStackTrace();
    println("No file found");
    } catch (IOException e) {
    e.printStackTrace();
    }
}

I'm sorry for my english. I hope that all is understadable.

Start by using Traceview to determine where your time is being spent. — CommonsWare
– CommonsWare, Commented Feb 9, 2014 at 22:18
'reader' cannot possibly equal "null", and a NullPointerException doesn't inherently mean 'no file found'. I suggest you fix your strange error- and exception-handling. — user207421
– user207421, Commented Feb 9, 2014 at 23:14
I fix the exception-handling, but it is not important if file is readed completely, though too slow. — TheDealer105
– TheDealer105, Commented Feb 10, 2014 at 0:14

Tony the Pony · Accepted Answer · 2014-02-09 22:23:16Z

3

Your ArrayList keeps getting reallocated as you're adding items. This can consume a non-trivial amount of CPU time, especially as the list grows, since a bunch of pointers need to copied around in memory. A better approach would be to store the number of entries as the first item of the dictionary file, and then pre-allocate the ArrayList:

 dictionary = new ArrayList<String>(numberOfEntries);

A more advanced optimization would be a data structure that doesn't rely on a Java collection class at all. Depending on your needs, this could be a huge UTF-8 byte array that is read into memory in one swoop (or even accessed via a memory-mapped file).

answered Feb 9, 2014 at 22:23

Tony the Pony

41.7k72 gold badges195 silver badges285 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

TheDealer105 Over a year ago

"could be a huge UTF-8 byte array that is read into memory in one swoop (or even accessed via a memory-mapped file)" - how can I use it?

Melih Altıntaş · Accepted Answer · 2014-02-09 22:40:09Z

1

Your problem is adding word to arraylist In arraylist read operations are constant time - O(1) but write operations have the potential to run out of space in the backing array, re-allocation, and a copy - so that runs in O(n) time.so you should use linkedlist instead of arraylist.It provide efficiency o(1)(for adding)

LinkedList<String> dictionary = new LinkedList<String>();

edited Feb 9, 2014 at 22:40

answered Feb 9, 2014 at 22:31

Melih Altıntaş

2,5451 gold badge24 silver badges36 bronze badges

2 Comments

Tony the Pony Over a year ago

+1 However, I'm assuming the OP is going to want some kind of random access to the list items. Those read operations will be very slow -- O(n) slow.

TheDealer105 Over a year ago

Unfortunately, both method give the same result. I change to Pre-allocate ArrayList and LinkedList, and nothin changes. Read time is still a long.

Collectives™ on Stack Overflow

how to faster read text file to ArrayList on android

2 Answers 2

1 Comment

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related