http PUT request to upload zip file in golang

Question

I have client code which is basically trying to upload a tar.gz file to the server using a HTTP PUT METHOD. However the server seems to not like it and always seems to send me a 500 ERROR Response. Following is the code. I am not sure what is going wrong.

 func upLoadFileToServer (uploadFileName string) {
    tr := &http.Transport{
          TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
          ExpectContinueTimeout : 30 * time.Second,
        }

    client := &http.Client{ Transport:tr,
                          Timeout: 20 * time.Second}

    timeCurrent = time.Now()
    fileContents, err := ioutil.ReadFile(uploadFileName)
    if err != nil {
     log.Println("Failed to Read the File", uploadFileName, err)
    }

    PutReq, _ := http.NewRequest("PUT", "https://www.example.com/upload", strings.NewReader(string(fileContents)))

    PutReq.Header.Set("Content-Type", "application/zip")
    PutReq.ContentLength = int64(len(string(fileContents)))


    PutReq.Header.Set("Expect", "100-continue")
    PutReq.Header.Set("Accept", "*/*")
    PutReq.Header.Set("Date", timeCurrent.Format(time.RFC1123))
    PutResp, err := client.Do(inventoryPutReq)
    }

Sometimes I notice Connection RESET by PEER error. But most of the times it is 500. I try the exact same request using POSTMAN and it seems to work fine.

Hi and welcome! You might want to read How do I ask a good question, which enhances the probability for getting a useful answer drastically. Also you may find ESR's excellent essay How To Ask Questions The Smart Way helpful, too - despite its rather harsh title... ;) Frankly, the exact output (redacted, if need be) would be more helpful than a description of it. — Markus W Mahlberg
– Markus W Mahlberg, Commented Dec 11, 2018 at 7:35
Apart from a few useless string conversions (use bytes.NewReader(fileContents) and len(fileContents)), and incorrect error handling for ioutil.ReadFile (don't keep going if there is an error) there is nothing obviously wrong here. We can't debug the server by looking at the client code. You should inspect the response body. Hopefully the server gives you a hint about why it's failing. — Peter
– Peter, Commented Dec 11, 2018 at 8:31
One thing which comes to my mind is that we might be uploading to a form handler. In this case, we would talk of a multipart-mime upload - which could lead to said problems. But as Peter pointed out - we need more information on the server. — Markus W Mahlberg
– Markus W Mahlberg, Commented Dec 11, 2018 at 9:13

Markus W Mahlberg · Accepted Answer · 2018-12-11 10:26:12Z

This is a working example.

Most likely, it comes down to the fact that the server is a bit simple and takes the filename literally. And since you are not using filepath.Base on your uploadFileName, it might have path elements on it. Simply use it on your filename for testing purposes.

The resets might be caused by the timeouts.

package main

import (
    "bytes"
    "flag"
    "fmt"
    "io"
    "log"
    "mime/multipart"
    "net"
    "net/http"
    "os"
    "path/filepath"
    "sync"
)

var (
    targetPath string
    filename   string
    port       int
    handlerLog *log.Logger
    mainLog    *log.Logger
)

// This is automatically called after vars have been initialized and before main
func init() {
    flag.StringVar(&targetPath, "target", "./tmp", "target directory for uploads")
    flag.StringVar(&filename, "file", "", "file to upload")
    flag.IntVar(&port, "port", 0, "port to listen on. When 0, a random port is assigned")
    handlerLog = log.New(os.Stdout, "[handler] ", log.LstdFlags)
    mainLog = log.New(os.Stdout, "[main   ] ", log.LstdFlags)
}

// By returning a handler, we have an elegant way of initializing path.
func uploadHandler(path string) http.Handler {

    // We make sure path is an existing directory when the handler takes over
    if s, err := os.Stat(path); err != nil {
        if os.IsNotExist(err) {
            handlerLog.Printf("Target '%s' does not exist. Creating it...", path)
            if cerr := os.MkdirAll(path, 0755); cerr != nil {
                handlerLog.Fatalf("Creating target: %s", err)
            }
        } else {
            handlerLog.Fatalf("Error accessing '%s': %s", path, err)
        }
    } else if !s.IsDir() {
        handlerLog.Fatalf("Target '%s' is not a directory", path)
    }

    // Do NOT use this handler in production!!!
    // It is lacking all sorts of safety measures.
    // However, it is enough to demonstrate.
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        handlerLog.Println("Handling file upload...")

        handlerLog.Println("Parsing form...")
        if err := r.ParseMultipartForm(32 << 20); err != nil {
            handlerLog.Fatalf("Parsing form: %s", err)
        }
        f, h, err := r.FormFile("file")
        if err != nil {
            handlerLog.Printf("Error accessing file: %s", err)
            http.Error(w, http.StatusText(http.StatusBadRequest), http.StatusBadRequest)
            return
        }
        defer f.Close()

        handlerLog.Println("Opening output file...")
        t, err := os.OpenFile(filepath.Join(path, filepath.Base(h.Filename)), os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0644)
        if err != nil {
            handlerLog.Printf("Opening output file: %s", err)
            http.Error(w, http.StatusText(http.StatusInternalServerError)+": "+err.Error(), http.StatusInternalServerError)
            return
        }
        defer t.Close()

        handlerLog.Println("Copying to output file...")
        if _, err = io.Copy(t, f); err != nil {
            handlerLog.Printf("Copying to output file: %s", err)

            http.Error(w, http.StatusText(http.StatusInternalServerError)+": "+err.Error(), http.StatusInternalServerError)
            return
        }
        handlerLog.Println("Finished handler!")
    })
}

func main() {
    flag.Parse()

    // Check input
    if filename == "" {
        mainLog.Fatal("No filename given. Exiting...")
    }

    mainLog.Println("Setting up upload handler...")
    http.Handle("/upload", uploadHandler(targetPath))

    wg := sync.WaitGroup{}
    wg.Add(1)

    // We want to finish the program after upload, as we only want to demonstrate
    go func() {
        mainLog.Println("Setting up listener...")

        listener, err := net.Listen("tcp", fmt.Sprintf("127.0.0.1:%d", port))
        if err != nil {
            mainLog.Fatalf("%s", err)
        }
        defer listener.Close()

        port = listener.Addr().(*net.TCPAddr).Port
        mainLog.Printf("Listening to port %d on localhost", port)

        wg.Done()
        http.Serve(listener, nil)
    }()

    buf := bytes.NewBuffer(nil)
    bodyWriter := multipart.NewWriter(buf)

    // We need to truncate the input filename, as the server might be stupid and take the input
    // filename verbatim. Then, he will have directory parts which do not exist on the server.
    fileWriter, err := bodyWriter.CreateFormFile("file", filepath.Base(filename))
    if err != nil {
        mainLog.Fatalf("Creating fileWriter: %s", err)
    }

    file, err := os.Open(filename)
    if err != nil {
        mainLog.Fatalf("Opening file: %s", err)
    }
    defer file.Close()

    if _, err := io.Copy(fileWriter, file); err != nil {
        mainLog.Fatalf("Buffering file: %s", err)
    }

    // We have all the data written to the bodyWriter.
    // Now we can infer the content type
    contentType := bodyWriter.FormDataContentType()

    // This is mandatory as it flushes the buffer.
    bodyWriter.Close()

    // As we wait for the server to spin up, we need to wait here.
    mainLog.Println("Waiting for the listener to be set up...")
    wg.Wait()

    req, err := http.NewRequest(http.MethodPut, fmt.Sprintf("http://127.0.0.1:%d/upload", port), buf)
    if err != nil {
        mainLog.Fatalf("Creating request: %s", err)
    }
    req.Header.Set("Content-Type", contentType)

    client := http.Client{}

    mainLog.Println("Sending file")
    res, err := client.Do(req)
    if err != nil {
        mainLog.Fatalf("Sending file: %s", err)
    }

    mainLog.Printf("Received %s from server. Exiting...", res.Status)
}

Collectives™ on Stack Overflow

http PUT request to upload zip file in golang

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related