building http based clients in Go

When I came from the very dynamic world of ruby, my first approach to designing HTTP clients in go was to think about mocking similar to what I was used to in Ruby. I quickly learned that it’s not the ideal approach in Go, and over time have learned a few tips and tricks for dealing with HTTP clients.

Hypothetical Client: Runtime

Let’s imagine we have a hypothetical Runtime client with just one public method:

If the implementation is mostly executing an HTTP request against a URL, one can imagine the following simple implementation:

This all makes sense so far, and is straightforward.

Subtle design decision: Always specify your http client

By starting with a specific HTTP client even with our simplistic implementation, we can potentially provide a custom HTTP client with instrumentation, or possibly one which implements some retry semantics.

But most importantly, when going to production, tuning timeout knobs would be the most important thing to not forget.

Testing this implementation

So now we reach that part: do we do mocking? Which line of abstraction do we strike it against?

Because of two important design decisions, we’re able to test this easily without any heavy dependencies, namely:

1. We specify the URL for the client explicitly.
2. We allow the caller to specify an httpClient.

The following is an example test implementation, which simply utilizes the wonderful httptest package:

Conclusion

As we can see, testing http based clients need not resort to any sort of complicated mocking: instead we can just use http.Client as the core interaction point.

Footnotes

[1]: We use ioutil.ReadAll heavily in these examples. Depending on the actual usecase, we should likely use a stream based interface (e.g. json.NewDecoder as an example).

[2]: httptest also has an equivalent NewTLSServer helper function — which creates a listener with TLS. The Client() method then would return an HTTP client which would have the self signed certificate information.