1

I'm working on an OCaml project which seems to have some problem of computational efficiency. While trying to find the bottleneck, I tried out of curiosity compiling to native code instead of bytecode, wondering how much the difference in running time would be.

It turns out that native code raises stack overflow. I tried executing with OCAMLRUNPARAM=b and I don't get much information about the stacktrace:

Fatal error: exception Stack overflow
Raised at file "map.ml", line 122, characters 10-25
Called from file "str.ml", line 259, characters 6-29

And nothing more. I don't know what's the point in my code which causes it. Furthermore, my program doesn't use large strings, so I don't get why Str should go stack overflow...

Is there some parameter I can tune to investigate the problem?

Improve this question
2
  • Though your program doesn't use large strings, it looks like that it uses lots of strings packed in a list. It is a long shot, but apparently, you have a very big list of strings and you're applying some Str function to each of them using List.map that is not tail recursive in the standard library. This is only a speculation, of course, but the first to come it mind :) Commented Jul 17, 2017 at 18:10
  • The only use of Str was over a small number of elements. I removed it, but the stack overflow still happens when I compile to native code. Commented Jul 17, 2017 at 18:49

3 Answers 3

Reset to default
0

The overflow is caused by the depth of nested function calls, not by the length of the strings involved.

Other than this, you don't give enough information to be helpful. Do you have a file named "map.ml" in your project? What does it look like around line 122?

If you don't have a file named "map.ml" in your project, you are possibly seeing non-tail-recursive behavior of a function in the standard library. In particular, I would guess that the problem is in the compilation of a regular expression in the Str module. Do you have any regular expressions that are exceptionally large?

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

Yes, I didn't mean the string length "per se". No map.ml file, that call is in Str here: github.com/ocaml/ocaml/blob/trunk/otherlibs/str/str.ml#L259 and I assume it's a non-tail-recursive function raising it, as you said. I had a simple regex to check if a string is a valid integer, that I now removed, and a bigger one (not that complex) at the beginning of the program, which is invoked only once.
The reported line numbers indicate a problem with the Map.find that is inside either Str.regexp or Str.regexp_case_fold.
Yes, that was my guess too, but now that I removed every use of Str (and don't require it anymore while compiling), I still get stack overflow. The weird part is that it now states it is raised in "read.mll" at 712, called from "read.mll" at 241... I don't know where those files come from, since I didn't use .mll files in my project! And I still don't get why the exception gets thrown with the native code and not with the bytecode...
This suggests (to me) that other parts of your code are making the stack really deep, then various leaf calls are failing. You might try making the stack bigger to see what happens (for bash: ulimit -s BIGNUMBER).
I removed from my code all the parts which used external packages/libraries and I don't require them anymore at compile time. Then I set ulimit -s 30000 (ulimit -a confirms), and tried executing again. I still get stack overflow, this time without even a stack trace. I assume the problem is within the implementation of List, Set or Map, so I will simply try resolving the problem at code level (must be done anyway). I still don't get why this doesn't happen with bytecode compilation, but whatever...
0

If you build your code using -g (for example by adding true: debug in _tags if you use ocamlbuild), you'll have debug info for your code in the stacktrace (as opposed to just the standard library). That should give you more information about the problem.

Improve this answer

1 Comment

Aw, I assumed I was using the -g, but I forgot I switched environment... Anyway, I re-compiled using ocamlbuild with -tag 'debug', but the stacktrace is still the same (I tried forcing a bug, in that case my functions are listed too...
0

OCAMLRUNPARAM controls the stack size of the OCaml interpreter and has nothing to do with the compiled code, which is run by an operating system loader, thus you need to use your operating system to control the size of the stack. Hint, in UNIX you can use ulimit to set the stack size.

The problem with pinpointing a source of the stack overflow is that the default size of the stack allows such big traces, that a default stack overflow printer is unable to print it up to the source, thus the trick is actually to limit the stack size, see full explanations in my other answer.

Of course, in addition, you need (i) to build your source code with the -g option and (ii) enable a stack tracing (depending on a compiler version, and used libraries, it may or may not be enabled by default). The former is specific to your build system, the latter can be done either by setting the OCAMLRUNPARAM=b environment variable (works both for the bytecode and the native code) or by explicitly calling Printexc.record_backtrace true in your source code, at the start of the application.

These were the general notes. In your case, it is however quite possible that the map call causes the stack overflow. In the OCaml standard library, the List.map is not tail-recursive, so you might consider substitution all List.map xs with List.rev (List.rev_map xs), or just with the List.rev_map if the elements order doesn't matter.

Improve this answer

6 Comments

You're right, I initially forgot using the -g option (OCAMLRUNPARAM=b was indeed set)! Now I've compiled using ocamlbuild with -tag 'debug', but the trace is the same as before. I'll try with ulimit... About List.map: I don't use it, seems to me that Map is invoked by Str. Anyway, I use some Map.Make.filter and List.exists, I'll check if the library implementation is tail-recursive.
hmm... another long shot, do you by any chance use the Janestreet Core library? Also, make sure that you indeed run the recompiled program, do bapbuild -clean before, and make sure (with the -classic-display option), that the -g flag is indeed passed, and passed to all stages, including the linking.
No, I don't use Janestreet's Core. Here's something weird: removing every reference to Str, the program still causes a stack overflow, but this time with file "read.mll" at 712, called from file "read.mll" at 241... And I don't use any .mll file, or custom parser.
OK, my long range cannon is out of ammo :) We definitely need more information, about your project. Is it available somewhere online? What libraries does it use? Etc... The fact that an exception is raised somewhere from read.mll and it is not a file in your repository, means that it is defined in some library, that you're using, and this library is responsible for the SO (that partially explains why you didn't see you code in the SO, as the dependency library is not compiled with the -g option).
I removed from my code all the parts which used external packages/libraries and I don't require them anymore at compile time (still using -tag 'debug') . Then I set ulimit -s 30000 (ulimit -a confirms), and tried executing again (still with OCAMLRUNPARAM=b). I still get stack overflow, this time without even a stack trace. I assume the problem is within the implementation of List, Set or Map, so I will simply try resolving the problem at code level (must be done anyway, running times are unbelievable). I still don't get why this doesn't happen with bytecode compilation, but whatever...
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.