In the last article, we set up a basic Go application, built a basic Makefile to build it, and set up a basic Docker container that we could use to deploy the application. This time, we will build on this base and add configuration to our application.
Github Repository
Part 2 source code can be found in a branch on this Github Repository
Making your application configurable is essential. You may have database settings which may need to be changed when deploying from your development environment. You may have server settings, behavior settings, and feature flags that you need to configure easily.
The options for configuring an application are various. There's json, yaml, xml, and the old-standby ini files. All of these have their merits, and depending on the scenario, you may choose to read your settings from them. However, I am turning to environment variables these days, especially for web services and applications deployed to a server.
Environment variables are versatile, easy to set, and can be ephemeral. I like to deploy my applications using containers, and it's much easier to set environment variables for the container than to figure out how to copy a file into it or reference it through a volume.
During development, however, it is usually a lot easier just to set variables in a file. This way, you do not have to remember to do a bunch of export commands in your shell. Enter the env file. The .env file, when present, is loaded on application startup and populates the environment based on its contents.
Reading Environment Variables
I like to keep my configuration reading outside of the main application in its own package. We create the following directory:
mkdir -p internal/config
Directory Structure
There are numerous ways to organize your go project's directories. I like to use the internal/ directory to indicate this is internal to my application or library and should not be used or referenced from the outside. While I use lib/ or pkg/ directories to indicate these are available for outside use, and others are free to import them into their library or application.
Inside this directory, we can create a readconfig.go file:
package config
import (
"os"
)
// Primary application configuration structure
type AppConfig struct{
Message string
}
func ReadConfig() (*AppConfig, error) {
message, found := os.LookupEnv("APP_MESSAGE")
if !found {
// Of course, we could return an error here instead.
message = "Hello, World!"
}
return &AppConfig{
Message: message,
}, nil
}
Now we can update our main.go to reference our config package and load and run the config:
package main
import (
"fmt"
"github.com/ymiseddy/golang_sample_project/internal/config"
)
func main() {
appConfig, err := config.ReadConfig()
if err != nil {
panic(err)
}
fmt.Println(appConfig.Message)
}
Now, if we run our application, we still get our familiar message:
❯ go run ./cmd/app/main.go
Hello, World!
However, we can also change the message at run time by setting the APP_MESSAGE variable:
❯ APP_MESSAGE="Hello, Bozo!" go run ./cmd/app/main.go
Hello, Bozo!
Reading a .env file
Of course, we don't want to have to type all the environment variables every time we run our application in development, so it would be helpful if we could read them from the .env file (pronounced dot env).
First, we update our readconfig.go file by adding godotenv to the import section:
package config
import (
"os"
"github.com/joho/godotenv"
)
Now, in our ReadConfig() function, we can read and load the .env file:
func ReadConfig() (*AppConfig, error) {
err := godotenv.Load()
//It's okay if the .env file doesn't exist
if err != nil && !os.IsNotExist(err) {
return nil, err
}
message, found := os.LookupEnv("APP_MESSAGE")
if !found {
// Of course, we could return an error here instead.
message = "Hello, World!"
}
return &AppConfig{
Message: message,
}, nil
}
Before building or running our application this time, we will want to add the new package to our project go.mod file. Although you can use go get github.com/joho/godotenv, I find it easier to use go mod tidy:
❯ go mod tidy
go: finding module for package github.com/joho/godotenv
go: found github.com/joho/godotenv in github.com/joho/godotenv v1.5.1
Now we need to create a .env file with our settings in it:
APP_MESSAGE= "Hello from .env file!"
And now we can test it:
❯ go run cmd/app/main.go
Hello from .env file!
Before committing our changes, we want to add the .env file to our .gitignore. Just add this line to the end of the file:
.env
Warning
Environment files can contain sensitive information such as database authentication information and api keys. You mustn't commit these to your repository! However, it is a good idea to keep example files that others can reference. Adding a .env.example with all the available settings, just using dummy values that others can replace, is a good practice.
Automating Environment with struct tags
Now, as our application progresses, we need to add more settings, such as our database connection or the address of other servers. We could keep adding to our ReadConfig function like this:
...
db, found := os.LookupEnv("DATABASE_CONNECTION")
if !found {
return fmt.Errorf("DATABASE_CONNECTION is not set.")
}
...
apiKey, found = os.LookupEnv("API_KEY")
if !found {
return fmt.Errorf("API_KEY is not set.")
}
...
Doing so is fine at first, but as we add more settings, this code can become cumbersome. Wouldn't it be great if we had a way to update our AppSettings struct and have the new settings automatically added?
Enter struct tags. In Go, struct tags on their own don't do anything, but when paired with the reflect package, they let you automate a lot of code. If you've ever worked with the json package, you've likely encountered struct tags - here's an example from Go by Example - where they use struct tags to name the associated json property.
type response2 struct {
Page int `json: "page"`
Fruits []string `json: "fruits"`
}
We're going to do something similar by adding custom tags to our configuration struct to make it easier to add new settings:
// Primary application configuration structure
type AppConfig struct {
Message string `env:"APP_MESSAGE,default=Hello, World!"`
ApiKey string `env: "API_KEY,required" `
TimeoutSeconds int `env: "TIMEOUT_SECONDS,default=200" `
}
Now we can use the reflect module to read values into our struct automatically. Additionally, we can also inspect the type of our configuration fields to determine how to parse them:
package config
import (
"fmt"
"os"
"reflect"
"strconv"
"strings"
"github.com/joho/godotenv"
)
// Primary application configuration structure
type AppConfig struct {
Message string `env:"APP_MESSAGE,default=Hello, World!"`
ApiKey string `env: "API_KEY,required" `
TimeoutSeconds int `env: "TIMEOUT_SECONDS,default=200" `
}
func ReadConfigInto(target any) error {
// Load environment variables from .env file if it exists
_ = godotenv.Load()
val := reflect.ValueOf(target).Elem()
typ := val.Type()
for i := 0; i < val.NumField(); i++ {
field := val.Field(i)
structField := typ.Field(i)
tag := structField.Tag.Get("env")
if tag == "" {
continue
}
// Parse the tag
parts := strings.Split(tag, ",")
envVar := parts[0]
var defaultValue string
required := false
for _, part := range parts[1:] {
if part == "required" {
required = true
} else if after, ok := strings.CutPrefix(part, "default="); ok {
defaultValue = after
}
}
// Get environment variable value
envValue, exists := os.LookupEnv(envVar)
if !exists {
if required {
return fmt.Errorf("required environment variable %s is not set", envVar)
}
envValue = defaultValue
}
// Set the field based on its type
if !field.CanSet() {
return fmt.Errorf("cannot set field %s", structField.Name)
}
switch field.Kind() {
case reflect.String:
field.SetString(envValue)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
intVal, err := strconv.ParseInt(envValue, 10, 64)
if err != nil {
return fmt.Errorf("invalid integer value for %s: %s", envVar, envValue)
}
field.SetInt(intVal)
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
uintVal, err := strconv.ParseUint(envValue, 10, 64)
if err != nil {
return fmt.Errorf("invalid unsigned integer value for %s: %s", envVar, envValue)
}
field.SetUint(uintVal)
case reflect.Float32, reflect.Float64:
floatVal, err := strconv.ParseFloat(envValue, 64)
if err != nil {
return fmt.Errorf("invalid float value for %s: %s", envVar, envValue)
}
field.SetFloat(floatVal)
case reflect.Bool:
boolVal, err := strconv.ParseBool(envValue)
if err != nil {
return fmt.Errorf("invalid boolean value for %s: %s", envVar, envValue)
}
field.SetBool(boolVal)
default:
return fmt.Errorf("unsupported field type %s for %s", field.Kind(), structField.Name)
}
}
return nil
}
Info
With this new function, notice how we are passing in a pointer to our configuration structure instead of making the function generate a new one? We can now have multiple configuration structs that we can read as needed in the future.
With this new ReadConfigInto function, we can use our struct tags to determine the environment variables that provide our config values. The underlying types of the struct field determine how to parse them. We've set this up to handle int, floats, string, and boolean types. In the future, we could add additional types or more complex scenarios, like comma-separated fields for arrays.
Now let's update our main.go to use this new function call:
package main
import (
"fmt"
"github.com/ymiseddy/go-getting-started/internal/config"
)
func main() {
appConfig := config.AppConfig{}
err := config.ReadConfigInto(&appConfig)
if err != nil {
panic(err)
}
fmt.Println(appConfig.Message)
fmt.Printf("Here's our config: %+v\n", appConfig)
}
Let's add more settings to our .env file:
APP_MESSAGE= "Hello from .env file!"
API_KEY="12345-ABCDE"
TIMEOUT_SECONDS=25
And run everything to make sure it still works:
❯ go run cmd/app/main.go
Hello from .env file!
Here's our config: {Message:Hello from .env file! ApiKey:12345-ABCDE TimeoutSeconds:25}
Although this code might seem like overkill, as our application complexity increases, we can easily add additional configuration fields to our AppConfig struct. But the real power comes when our application's complexity grows to a point where a single config struct for the entire application is no longer sufficient.
Multiple Config Structs for Separation of Concerns
As complexity grows, we may get to a point where it is necessary to split our configuration into multiple structs.
Keeping configuration settings separate is part of a concept known as Separation of Concerns. This software development principle helps us arrive at a good application design - if we are configuring a database connection, it should not need to know which port our web server is listening on - and vice versa.
Let's add some more and different configuration settings structs to our application:
type AppConfig struct {
Message string `env:"APP_MESSAGE,default=Hello, World!"`
ApiKey string `env: "API_KEY,required" `
TimeoutSeconds int `env: "TIMEOUT_SECONDS,default=200" `
}
type DatabaseConfig struct {
DataSourceName string `env: "DB_DSN,required" `
}
type WebServerConfig struct {
Host string `env: "WEB_HOST,default=0.0.0.0" `
Port int `env: "WEB_PORT,default=8080" `
}
Let's go ahead and update our main.go to use these separate structures:
func main() {
appConfig := config.AppConfig{}
databaseConfig := config.DatabaseConfig{}
webServerConfig := config.WebServerConfig{}
err := config.ReadConfigInto(&appConfig)
if err != nil {
panic(err)
}
err = config.ReadConfigInto(&databaseConfig)
if err != nil {
panic(err)
}
err = config.ReadConfigInto(&webServerConfig)
if err != nil {
panic(err)
}
fmt.Printf("Database Config: %+v\n", databaseConfig)
fmt.Printf("Web Server Config: %+v\n", webServerConfig)
println(appConfig.Message)
}
And of course, update our .env file to test these out fully:
APP_MESSAGE= "Hello from .env file!"
API_KEY="12345-ABCDE"
TIMEOUT_SECONDS=25
DB_DSN="sqlite:///mydatabase.db"
WEB_HOST=192.168.1.21
WEB_PORT=8193
We can now run what we have so far to ensure everything is working as it should:
❯ go run cmd/app/main.go
Application config: {Message:Hello from .env file! ApiKey:12345-ABCDE TimeoutSeconds:25}
Database Config: {DataSourceName:sqlite:///mydatabase.db}
Web Server Config: {Host:192.168.1.21 Port:8193}
Hello from .env file!
As always, never forget to commit and push your work:
git add .
git commit -m "feat: adding environment-based configuration"
git push
Conclusion
We've extended our application by adding a robust way to incorporate settings using environment variables while still making it easy to develop. We've hinted at the importance of keeping the different parts of our configuration separate from each other. Next time, we will gain more insight into how this comes into play as we explore Inversion of Control and Dependency Injection.