Generating a very long random string of letters

Question

I was trying to fill a file with about ten million random characters (a-z,A-Z,' '). To my surprise the code is extremely slow. I inserted this print i to observe the progress. It was so slow that I just left the program running and went out. After about more than three hours it is still at five millions.

How can I make it fast? Most likely I am doing something wrong. Shouldn't this task be done in less than a couple of minutes?

My system is probably quite powerful one: Dell Precision with Xeone processor having twelve logical cores and 48GBs of RAM.

public class RandChars{
    public RandChars(){

    }
    public static void main(String[] args) throws FileNotFoundException {
        final String alphabet = "AaBbCcDdEeFfGgHhIiJjKkLlMmNnO oPpQqRrSsTtUuVvWwXxYyZz";
        final int N = alphabet.length();
        final int NumChars=10240000;
        String SearchString=null;

        PrintWriter text_writer = new PrintWriter("SearchString.txt");

        Random r = new Random();
        for (int i = 0; i < NumChars; i++) {                
            SearchString +=  alphabet.charAt(r.nextInt(N));
            System.out.println(i); // I know this is making it a bit slow
        }
        text_writer.println(SearchString);
        text_writer.close();            
    }   
}

Refer to this other answer for why it is so slow codereview.stackexchange.com/a/3973/95296 — Tunaki
– Tunaki, Commented May 21, 2016 at 20:12
why so many negative votes? any reason? Just asking to learn. I am not a newbie @SE, I think, I know how to ask. That attached question in comments might be the very same at core but i did not actually know what exactly the problem is. — tod
– tod, Commented May 22, 2016 at 10:11
its better to use UUID.randomUUID().toString(); and concat it sevreal times.. — Tilak Madichetti
– Tilak Madichetti, Commented May 22, 2016 at 15:19
@TilakMadichetti Why? UUIDs aren't uniformly random selections from the alphabet. — 200_success
– 200_success, Commented May 23, 2016 at 0:52
@200_success They are random .Can't you see the function name , functionName.contains("random") == true lol — Tilak Madichetti
– Tilak Madichetti, Commented May 23, 2016 at 8:25

tod · Accepted Answer · 2016-05-25 14:26:30Z

3

Even after six hours it was not near to complete. So tried some hit and trail.

I believe this answer is not a good solution because it's making many IO accesses. But it solved the problem and the whole loop took just under a minute.

Random r = new Random();
for (int i = 0; i < NumChars; i++) {                
    text_writer.print(alphabet.charAt(r.nextInt(N));
    System.out.println(i); 
}
text_writer.close();

The reason for posting this answer here is to just show that there exist solutions which experts can easily provide rather than negatively perceiving it.

Edit: No, further reading shows that this solution is fine. Though it can be improved. It's not making as many IOs as it might look. Since the used PrintWriter uses some buffered mechanism, in addition to, OS buffering. Which handles the overuse of print and hence automatically reduces the IO accesses to a reasonable number.

edited May 25, 2016 at 14:26

answered May 21, 2016 at 22:12

tod

1631 silver badge7 bronze badges

1

\$\begingroup\$ Great! You are on the right track. This solution eliminates the performance problem caused by the poorly done string concatenation. As you point out, there is still room for improvement, since printing one character at a time still incurs a large overhead for each system call. If you're planning to write a very large file, I recommend a compromise between this solution and @Legato's. You don't want to write a character at a time, nor should you need to build the entire file's contents in memory all at once. Rather, write chunks of 8 kB or so. \$\endgroup\$

200_success
– 200_success

2016-05-22 13:08:15 +00:00
Commented May 22, 2016 at 13:08

Add a comment |

200_success · Accepted Answer · 2016-05-22 12:58:01Z

The problem you're having is too many strings being created in memory. You current algorithm creates one for every single character you're writing.

You can utilize a StringBuilder to ensure this isn't the case and still retain the desired results, but in a fraction of a second to run.

Example:

import java.io.Writer;
import java.io.BufferedWriter;
import java.io.OutputStreamWriter;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.Random;


public class Example {
    public static void main(String[] args) {
        Random random = new Random();
        String alphabet = "AaBbCcDdEeFfGgHhIiJjKkLlMmNnO oPpQqRrSsTtUuVvWwXxYyZz";
        int alphabetLength = alphabet.length();
        int charTotal = 10240000;
        StringBuilder writeBuilder = new StringBuilder();
        for (int i = 0; i < charTotal; i++) {
            writeBuilder.append(alphabet.charAt(random.nextInt(alphabetLength)));
        }
        try (Writer writer = new BufferedWriter(new OutputStreamWriter(
        new FileOutputStream("whyDoThis.txt"), "UTF-8"))) {
            writer.write(writeBuilder.toString());
        } catch (IOException ioe) {
            ioe.printStackTrace();
        }
    }
}

GreedyHat · Accepted Answer · 2016-05-23 00:45:44Z

0

You could use a string generator from Apache Commons. Internally they use just an array of chars, extending it step by step with nextInt(), that will be just passed at the end to a string constructor. It seems enough fast: creating and storing take only 330 milliseconds.

String creation:

final long timeX = System.currentTimeMillis();
final String str = RandomStringUtils.random(10000000, "AaBbCcDdEeFfGgHhIiJjKkLlMmNnO oPpQqRrSsTtUuVvWwXxYyZz");
final long timeY = System.currentTimeMillis() - timeX;
System.out.println(timeY);

Output: 255

With writing into file:

final PrintWriter out = new PrintWriter("SearchString.txt");

final long timeX = System.currentTimeMillis();
final String str = RandomStringUtils.random(10000000, "AaBbCcDdEeFfGgHhIiJjKkLlMmNnO oPpQqRrSsTtUuVvWwXxYyZz");
out.print(str);

final long timeY = System.currentTimeMillis() - timeX;
System.out.println(timeY);

Output: 330

PC: MAC 2,2 GHz Intel Core i7, 8 GB 1067 MHz DDR3

edited May 23, 2016 at 0:45

answered May 21, 2016 at 22:26

GreedyHat

887 bronze badges

1

\$\begingroup\$ What is RandomStringUtils? \$\endgroup\$

200_success
– 200_success

2016-05-22 13:09:27 +00:00
Commented May 22, 2016 at 13:09
\$\begingroup\$ It is a part of Apache commons. \$\endgroup\$

GreedyHat
– GreedyHat

2016-05-23 00:27:42 +00:00
Commented May 23, 2016 at 0:27

Add a comment |

Stack Exchange Network

Generating a very long random string of letters

3 Answers 3

You must log in to answer this question.

Linked

Hot Network Questions

Generating a very long random string of letters

3 Answers 3

You must log in to answer this question.

Linked

Related

Hot Network Questions