Processing a file byte by byte in go

Question

Similar to this question, what is the idiomatic way to process a file one byte at a time in go?

Put another way, is there a better way to write the following?

file, err := ioutil.ReadFile(filename)
file_string = string(file)
for i, c := range file_string {
    // -snip-
}

This isn't reading the file one byte at a time. This is reading the whole file, converting it to a string, and iterating through it rune by rune (not byte by byte). Do you want to read the file one byte at a time, or process it one byte at a time? — Burak Serdar
– Burak Serdar, Commented Apr 30, 2020 at 16:23
Reading one byte at a time is extremely inefficient, you probably want something like: Reader.ReadByte — Mr_Pink
– Mr_Pink, Commented Apr 30, 2020 at 16:26

Adrian · Accepted Answer · 2022-11-14 14:09:45Z

8

You're reading the entire file as a string (not bytes) at once, then processing it rune-by-rune (not byte-by-byte). To literally read the file one byte at a time (which is very likely not what you want), you must do exactly that:

f, err := os.Open("path")
if err != nil {
    panic(err)
}

b := make([]byte, 1)

for {
    err := f.Read(b)
    if err != nil && !errors.Is(err, io.EOF) {
        fmt.Println(err)
        break
    }

    // process the one byte b[0]

    if err != nil {
        // end of file
        break
    }
}

However, what you more likely want is to read the file efficiently, and process it one byte at a time. For this, you should use a buffered reader:

f, err := os.Open("path")
if err != nil {
    panic(err)
}

br := bufio.NewReader(f)

// infinite loop
for {

    b,err := br.ReadByte()

    if err != nil && !errors.Is(err, io.EOF) {
        fmt.Println(err)
        break
    }

    // process the one byte b

    if err != nil {
        // end of file
        break
    }
}

edited Nov 14, 2022 at 14:09

answered Apr 30, 2020 at 16:31

Adrian

47.2k6 gold badges121 silver badges110 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

h0ch5tr4355 Over a year ago

This for-loop is not working: expected boolean or range expression, found assignment (missing parentheses around composite literal?)

h0ch5tr4355 Over a year ago

for b, err := binFileReader.ReadByte(); err == nil; { fmt.Println(b) } Something like this is needed I think

Collectives™ on Stack Overflow

Processing a file byte by byte in go

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related