Structuring testable microservices with Go
I have been writing Java microservices for the majority of my career and stumbled into Go as an alternative to the boilerplate logic and structure Java requires. One of the most confusing and debated topics with Go is structuring applications. While there are many approaches to project structure with Go, I have found the following structure easy to maintain, test and (relatively) easy to transition other programmers to the language. To bypass my explanation and jump to the code, see the Github repository here.
Engineers frequently transition between multiple languages on a daily basis. Personally, I migrate between Java, NodeJS and Go depending on the application and use case. I have identified several structural best practices from the languages I work with and incorporated into the following layout where it makes sense.
The Repository Pattern
The first such pattern I would like to discuss is the repository pattern. Abstracting the persistence technology from other layers of the application allows ease of testing and mocking. Let’s start with the common interface.
// UserRepository interface describes repository operations on User types
type UserRepository interface {
GetAll() ([]models.User, error)
GetByID(string) (*models.User, error)
Create(models.User) (string, error)
Delete(models.User) error
}Most Java developers will feel at home with this concept as frameworks like Spring use programming to interfaces extensively. I introduce this abstraction layer for two reasons. First, the abstraction allows us to create mock implementations that do not require external connections to a database. For instance, if we wish to provide a mock implementation for testing our controller we could code a struct that implements this interface and retrieves data from a map data structure.
// MockUserRepository houses logic to retrieve users from a mock repository
type MockUserRepository struct{} // NewMockUserRepository convenience function to create a MockUserRepository
func NewMockUserRepository() repository.UserRepository {
return &MockUserRepository{}
}var users = map[string]models.User{
"1": models.User{
Name: "James Bond",
Gender: "male",
Age: 44,
ID: "1",
},
} // GetAll get all users from the repository
func (r MockUserRepository) GetAll() ([]models.User, error) {
userList := []models.User{}
for _, user := range users {
userList = append(userList, user)
}
return userList, nil
}...
Or, for negative test cases we could implement a type that only returns errors to inject into our controller for teasing out exceptional behavior.
// MockErroringUserRepository returns errors for all operations.
type MockErroringUserRepository struct{} // NewMockErroringUserRepository convenience function to create a MockErroringUserRepository
func NewMockErroringUserRepository() repository.UserRepository {
return &MockErroringUserRepository{}
} // GetAll get all users from the repository
func (r MockErroringUserRepository) GetAll() ([]models.User, error) {
return nil, errors.New("blamo")
}...
Secondly, the repository pattern allows a plug and play extensibility to your application should you wish to change persistence types. For instance, today I may use MySQL, but tomorrow I could be required to switch to another implementation that has quite different usage. I could switch out my persistence layer without impacting other layers of the application by adhering to the interface.
Testing the actual implementation can be tricky depending on the persistence technology but if your in the relational world using the go sql package, DATA-DOG has a great testing library called go-sqlmock that I use extensively. This library creates a database and mock type which can be injected into the type under test. See the MySQL implementation and example tests here.
The Controller Pattern
The next pattern that is highly debated in Go is the controller pattern. I feel controllers allow isolation to handler functions and allow ease of testing. This pattern also allows the injection of mock persistence types to truly isolate the controller and provide interesting negative test cases. I do not introduce an interface at the controller level. Instead I prefer to create a type containing my repository for dependency injection at higher levels. See complete controller and tests here.
// UserController struct containing web related logic to operate on Users
type UserController struct {
userRepository repository.UserRepository
} // NewUserController is a convenience function to create a UserController
func NewUserController(r repository.UserRepository) *UserController {
return &UserController{r}
}// GetUsers retrieve all users
func (u UserController) GetUsers(w http.ResponseWriter, r *http.Request, p httprouter.Params) {
users, err := u.userRepository.GetAll()
if err != nil {
http.Error(w, "service unavailable", http.StatusServiceUnavailable)
return
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(users)
}...
Wiring the Components
Now for the fun part, making it all work together. All wiring of dependencies and database configuration live within my main.go file. I tend to isolate configuration values like connection strings to an environment variable for easy management and flexibility.
func main() {
r := httprouter.New() ur := repository.NewUserRepository(getDatabase())
uc := controllers.NewUserController(ur) r.GET("/users", uc.GetUsers)
r.POST("/users", uc.AddUser)
r.GET("/users/:id", uc.GetUserByID)
r.DELETE("/users/:id", uc.DeleteUser)
http.ListenAndServe(":8080", r)
} func getDatabase() *sql.DB {
db, err := sql.Open("mysql", os.Getenv("MYSQL_HOST"))
if err != nil {
log.Panic(err)
}
return db
}
The repository is created, injected with a database type and provided to the controller for use. I have experimented with injection libraries like inject from Facebook but honestly feel the overhead isn’t worth the benefit for small microservices like the example above. Hand wiring the dependencies is clear and easy to follow.
I hope this example helps you in your journey to learn and use Go. Please share your experiences with structuring Go projects below!
