9

I am simply reading the /proc/diskstats file. My code is:

func ReadFromFile(filepath string)(string){
    defer func() {
        if err1 := recover(); err1 != nil {
            fmt.Println("!!!!!!!!!!!!!!!!Panic Occured and Recovered in readFromFile(), Error Info: ", err1)
        }
     }()

    strData := ""

    data, err := ioutil.ReadFile(filepath)
    if err != nil{
        fmt.Println("File read error: ", err)
        return ""
    }

    strData = string(data)
    return strData
}

The error I am getting is:

File read error: open /proc/diskstats: too many open files

Not only for this file, I am also getting the same error for some other files.

I have also run this command:

root@golang:~# lsof|wc -l

785

Please guide me.

Improve this question
8
  • 1
    What's your ulimit? Commented Jun 14, 2016 at 7:18
  • i restarted my service then i didn't get any error,but output of lsof|wc -l output is keep on incresing. Commented Jun 14, 2016 at 7:28
  • 1
    Are you closing after reading it? Commented Jun 14, 2016 at 7:29
  • No i think so ,how to do it. can you please help. Commented Jun 14, 2016 at 7:30
  • 3
    How are you calling the function? Leakage seems to be from somewhere else. Commented Jun 14, 2016 at 7:39

3 Answers 3

Reset to default
17

I ran into the same problem (maybe different circumstances or setup) and fixed it differently:

func some_func(file_name []string) {
    for _, file_name := range file_names {
        f, _ := os.Create(file_name)
        // defer f.Close() // bad idea
        _, _ = f.Write([]byte("some text"))
        f.Close() // better idea
    }
}

The problem is that defer will be executed, when the function will return, which could take a while - depending on the loop size (bad idea). So just do it explicit (better idea).

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

f.Write requires a []byte as an input, i.e. []byte("some text")
8

The OP does not provide a Minimal, Reproducible Example. The error in question is caused by code that was not posted. An easy way to demonstrate this is to simply run the provided code in a minimal example (with no other activity), and see that it does not fail.

The function ioutil.ReadFile, of course, closes the file. It has been implicated, in this case, simply because it is attempting to open a new file when the resource limit has already been reached.

One common gotcha in Go is failure to close streams that were opened implicitly. A specific case of this is that a stream is opened when using the client functions of the http library.

Examples: (1), (2)

The client must close the response body when finished with it:

resp, err := http.Get("http://example.com/")
if err != nil {
    // handle error
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)

Such requests should always include a call to Close, using a form as shown above.

There may be other, similar, cases of implicit streams being opened...

This is a particularly tricky issue because, for trivial programs, you'll never know the difference. You won't know there is a problem until you run this through hundreds, or thousands, of iterations. And, then, the error may often be presented as a failure of some unrelated function call -- just as demonstrated by the OP.

Improve this answer

2 Comments

// Body is the request's body. // // For client requests, a nil body means the request has no // body, such as a GET request. The HTTP Client's Transport // is responsible for calling the Close method. // // For server requests, the Request Body is always non-nil // but will return EOF immediately when no body is present. // The Server will close the request body. The ServeHTTP // Handler does not need to. Body io.ReadCloser
If defer is used in the context of a larger function, it may be a good idea to wrap that portion in a separate function (maybe an IIFE) -- since the execution is deferred "until the surrounding function returns".
6

Basically in UNIX platforms, the OS places a limit to the number of open file descriptors that a process may have at any given time.
The error too many open files is raised since you have reached the limit of file (and or pipe or socket)currently opened and you are trying to open a new file (and or pipe or socket).
To avoid this problem you must close the file when you have finished to use the open file using the Close() function

Improve this answer

4 Comments

To be honest, I would've expected ioutil.ReadFile to close the file (although it may well be that there's other file descriptors being opened elsewhere causing the problem).
If we look on ReadFile we can see that the Close() function is only called when the open file fails (golang.org/src/io/ioutil/ioutil.go?s=1464:1510#L39). There is another option to call automatically the Close() function and is using the return statement (as void so you must return nothing)
No, there's a "defer f.Close()" on line 54, causing the file to be closed during the return from the function. There's an early return if the Open call fails (so as to not call the Close() method on an undefined value).
You are right @Vatine I didn't see the earlier return, my fault

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.