How to test http calls in Go using httptest

Question

I have the following code:

package main

import (
    "encoding/json"
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
    "time"
)

type twitterResult struct {
    Results []struct {
        Text     string `json:"text"`
        Ids      string `json:"id_str"`
        Name     string `json:"from_user_name"`
        Username string `json:"from_user"`
        UserId   string `json:"from_user_id_str"`
    }
}

var (
  twitterUrl = "http://search.twitter.com/search.json?q=%23UCL"
  pauseDuration = 5 * time.Second
)

func retrieveTweets(c chan<- *twitterResult) {
    for {
        resp, err := http.Get(twitterUrl)
        if err != nil {
            log.Fatal(err)
        }

        defer resp.Body.Close()
        body, err := ioutil.ReadAll(resp.Body)
        r := new(twitterResult) //or &twitterResult{} which returns *twitterResult
        err = json.Unmarshal(body, &r)
        if err != nil {
            log.Fatal(err)
        }
        c <- r
        time.Sleep(pauseDuration)
    }

}

func displayTweets(c chan *twitterResult) {
    tweets := <-c
    for _, v := range tweets.Results {
        fmt.Printf("%v:%v\n", v.Username, v.Text)
    }

}

func main() {
    c := make(chan *twitterResult)
    go retrieveTweets(c)
    for {
        displayTweets(c)
    }

}

I'd like to write some tests for it, but I'm not sure how to use the httptest package http://golang.org/pkg/net/http/httptest/ would appreciate some pointers

I came up with this (shamelessly copied from the tests for go OAuth https://code.google.com/p/goauth2/source/browse/oauth/oauth_test.go):

var request = struct {
    path, query       string // request
    contenttype, body string // response
}{
    path:        "/search.json?",
    query:       "q=%23Kenya",
    contenttype: "application/json",
    body:        twitterResponse,
}

var (
    twitterResponse = `{ 'results': [{'text':'hello','id_str':'34455w4','from_user_name':'bob','from_user_id_str':'345424'}]}`
)

func TestRetrieveTweets(t *testing.T) {
    handler := func(w http.ResponseWriter, r *http.Request) {

        w.Header().Set("Content-Type", request.contenttype)
        io.WriteString(w, request.body)
    }

    server := httptest.NewServer(http.HandlerFunc(handler))
    defer server.Close()

    resp, err := http.Get(server.URL)
    if err != nil {
        t.Fatalf("Get: %v", err)
    }
    checkBody(t, resp, twitterResponse)
}

func checkBody(t *testing.T, r *http.Response, body string) {
    b, err := ioutil.ReadAll(r.Body)
    if err != nil {
        t.Error("reading reponse body: %v, want %q", err, body)
    }
    if g, w := string(b), body; g != w {
        t.Errorf("request body mismatch: got %q, want %q", g, w)
    }
}

Adrian · Accepted Answer · 2017-04-05 14:21:43Z

101

httptest does two types of tests: response and server

Response test:

func TestHeader3D(t *testing.T) {
    resp := httptest.NewRecorder()

    uri := "/3D/header/?"
    path := "/home/test"
    unlno := "997225821"

    param := make(url.Values)
    param["param1"] = []string{path}
    param["param2"] = []string{unlno}

    req, err := http.NewRequest("GET", uri+param.Encode(), nil)
    if err != nil {
            t.Fatal(err)
    }

    http.DefaultServeMux.ServeHTTP(resp, req)
    if p, err := ioutil.ReadAll(resp.Body); err != nil {
            t.Fail()
    } else {
            if strings.Contains(string(p), "Error") {
                    t.Errorf("header response shouldn't return error: %s", p)
            } else if !strings.Contains(string(p), `expected result`) {
                    t.Errorf("header response doen't match:\n%s", p)
            }
    }
}

Server test (which is what you need to use):

func TestIt(t *testing.T){
    ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Content-Type", "application/json")
        fmt.Fprintln(w, `{"fake twitter json string"}`)
    }))
    defer ts.Close()

    twitterUrl = ts.URL
    c := make(chan *twitterResult)
    go retrieveTweets(c)

    tweet := <-c
    if tweet != expected1 {
        t.Fail()
    }
    tweet = <-c
    if tweet != expected2 {
        t.Fail()
    }
}

BTW, you don't need to pass in the pointer of r, because it's already a pointer.

err = json.Unmarshal(body, r)

EDIT: for my recorder test, I could use my http handler like this:

handler(resp, req)

But my original code is not using the default mux (but from Gorilla/mux), and I have some wrapping around the mux, e.g. insert server logging, and adding request context (Gorilla/context), so I had to start from mux and call ServeHTTP

edited Apr 5, 2017 at 14:21

Adrian

47.2k6 gold badges121 silver badges110 bronze badges

answered Apr 26, 2013 at 14:36

nemo

12.9k4 gold badges24 silver badges29 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Marçal Juan Over a year ago

Please, add complete code, like importing all packages. Newbie here ;) Thanks!

030 Over a year ago

You could use goimports

030 · Accepted Answer · 2019-09-08 15:50:42Z

9

Originally this code snippet was found on GitHub Gist, but while trying to apply the concept to one of my projects I realized that I had to modify the main code significantly that I decided to test these calls using an integration test by using docker and curl.

edited Sep 8, 2019 at 15:50

answered May 31, 2016 at 15:14

030

11.9k15 gold badges88 silver badges127 bronze badges

Comments

James Henstridge · Accepted Answer · 2013-04-23 04:17:33Z

8

If you want to test your program, it is often best to write it with testing in mind. For instance, if you extracted the inner loop of your retrieveTweets function into something like this:

func downloadTweets(tweetsUrl string) (*twitterResult, error)

You could invoke it with the URL of a test server you've set up using the httptest package without having to worry about the sleeps or repeated requests.

answered Apr 23, 2013 at 4:17

James Henstridge

44.2k9 gold badges139 silver badges119 bronze badges

4 Comments

jwesonga Over a year ago

Can I mock out a server? I think wetting up a test server negates the purpose of the test, if I can mock out a server and the response I'm expecting that would fit in well with the test

James Henstridge Over a year ago

The httptest package is the infrastructure to set up a small HTTP server for the tests. You can implement the request handlers the same way you would normally, and then run your code against that server rather than Twitter.

Rob Kielty Over a year ago

@JamesHenstridge thank you, great answer! You comment here is more informative than the Overview Statement on the official docs for net/http/httptest

Saurabh Over a year ago

In this case, would httptest test response or server?

Collectives™ on Stack Overflow

How to test http calls in Go using httptest

3 Answers 3

2 Comments

Comments

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related