Go and JSON: Tags, Empty Strings and Nil Values

I noticed that many struggled with encoding and decoding JSON in Go. Here I cover the issues developers run into and how to solve them.

Recently, I have been spending a lot of time working with Go. While helping other developers I noticed that many struggled with encoding and decoding JSON. In this guide I cover a lot of the common issues developers run into and how to solve them.

Naming keys

In the following example I will define a struct and then marshal it using the encoding/json package.

package main

import (
	"encoding/json"
	"fmt"
)

type Item struct {
	Foo   bool
	Bar string
}

func main() {
	itemJson, _ := json.Marshal(&Item{Foo: true, Bar: "Test"})
	fmt.Println(string(itemJson))
}

The code above will output the following string.

{"Foo":true,"Bar":"Test"}

The keys Foo and Bar are using the same format as our struct. You may want these to be lowercase or different from your struct. To do that we can use struct tags like so.

type Item struct {
	Foo   bool `json:"foo"`
	Bar string `json:"bar_key"`
}

By only changing the struct and running the same code again we will get the following output.

{"foo":true,"bar_key":"Test"}

Excluding keys

We can exclude the keys by adding a - to our json tag in our struct. Any tags which use a - will be excluded when marshalling a struct. So let’s change one of our tags to - like this:

type Item struct {
	Foo   bool `json:"foo"`
	Bar string `json:"-"`
}

When we run our code again we will get the following output.

{"foo":true}

As you can see Bar is now excluded from the marshalled string. This is useful if you have structs that serve multiple purposes such as database queries and api responses.

Optional or empty keys when unmarshalling

In some situations you may send or receive JSON with optional keys which are sometimes present and sometimes not. The issue you may run into is that bools are false, ints are 0 and strings are "" when they are not set. I have seen some developers make huge workarounds to deal with this issue but there is a simple solution: pointers. Let’s take a look at an example without pointers.

package main

import (
	"encoding/json"
	"fmt"
)

type Item struct {
	Foo bool `json:"foo"`
	Bar int  `json:"bar"`
}

func main() {
	itemJson, _ := json.Marshal(&Item{})
	fmt.Println(string(itemJson))
	item := Item{}
	emptyJson := "{}"
	_ = json.Unmarshal([]byte(emptyJson), &item)
	fmt.Println(item.Bar, item.Foo)
}

In the above example we are doing two things. First, we marshal en empty Item struct without any values. Then we take an empty JSON object and Unmarshal this to a new Item struct and then print the values of Bar and Foo. The following is the output we get from running the above code:

{"foo":false,"bar":0}
0 false

Even though we did not set any values, Bar has a value of 0 and Foo has a value of false. This can be a problem with optional parameters because we don’t know if the data we received was actually set to a value or omitted. In the same way, we might end up sending values which we did not intend to send. Now, let’s try changing our struct to use pointers instead:

type Item struct {
	Foo *bool `json:"foo"`
	Bar *int  `json:"bar"`
}

If we run the same code but only adding * we get the following output:

{"foo":null,"bar":null}
<nil> <nil>

Now we have have nil values since we did not set anything. This allows us to check if a parameter was actually set to a value. For example, we might use the following code to check the value of item.Foo after unmarshalling.

if item.Foo == nil {
	// Foo was not set
} else if *item.Foo {
	// Foo was true
} else if !*item.Foo {
	// Foo was false
}

Conclusion

I hope this post has helped cover the common issues with Go and JSON. If I missed anything or made a mistake please let me know in the comments. Thank you for reading!

How To Run A Go Web Server On AWS

Amazon Web Services (AWS) offers free tier web servers where you can run simple go programs for API backends or websites. In this guide we will make a simple web server using a go program. This guide can also be used as a reference to set up AWS instances for larger go services.

To begin you will need to set up a new instance on AWS. To do this simply login to your AWS account and click on Services at the top of the screen and then on EC2. EC2 should be the first option under the Compute category.

Click on Instances on the left hand side and then in the top left there should be a large button called Launch Instance which will start a creation wizard. I typically search for Ubuntu and pick 18.04 as shown in the screenshot below.

Continue reading “How To Run A Go Web Server On AWS”

How to Install Go 1.12 on Ubuntu 18

This guide is also available as a video from Youtube.

Begin by running the following command to add the golang backports ppa:

sudo add-apt-repository ppa:longsleep/golang-backports

If you get an error such as “add-apt-repository command not found” then you need to run the following command to install add-apt-repository:

Continue reading “How to Install Go 1.12 on Ubuntu 18”

Developing Slack Bots with Go: Part 1

In my previous post I wrote a guide on how to post a simple Slack message using a bot with PHP. Now we will do the same thing but instead with Go. I have also decided to send a json request instead of a form-url encoded request like we did with PHP. There are some differences, let’s take a look.

Continue reading “Developing Slack Bots with Go: Part 1”

Install Go 1.11 on Ubuntu 18.04

Begin by running the following command to add the golang backports ppa:

sudo add-apt-repository ppa:longsleep/golang-backports

If you get an error such as “add-apt-repository command not found” then you need to run the following command to install add-apt-repository:

Continue reading “Install Go 1.11 on Ubuntu 18.04”

Install Go 1.10 on Ubuntu 18.04

Also: See my guide on installing Go 1.11 here.

Begin by running the following command to add the golang ppa:

sudo add-apt-repository ppa:gophers/archive

If you get an error such as “add-apt-repository command not found” then you need to run the following command to install add-apt-repository:

Continue reading “Install Go 1.10 on Ubuntu 18.04”

How To Make A Cryptocurrency Trading Bot With Golang

A Quick Disclaimer: I'm a programmer and have very little experience trading. Please do your own research before using the algorithm I create for the cryptocurrency trading bot or any other algorithm. Only trade with money you can afford to lose.

In this guide I will show you how to create¬† a cryptocurrency trading bot with Go. I highly recommend an IDE such as Goland but you can use a simple text editor if you would like. Feel free to take a look at my guide on how to develop with Go on Linux if you don’t have Go installed.

Continue reading “How To Make A Cryptocurrency Trading Bot With Golang”

Parsing Websites with Golang and Colly

I stumbled across a scraper and crawler framework written in Go called Colly. Colly makes it really easy to scrape content from web pages with it’s fast speed and easy interface. I have always been interested in web scrapers ever since I did a project for my university studies and you can read about that project here. Before continuing, please note that scraping of websites is not always allowed and sometimes even illegal. In the guide below we will be parsing this blog, GoPHP.io.

Continue reading “Parsing Websites with Golang and Colly”

How to: Developing with Go on Linux

This guide assumes you are using Ubuntu Linux 16.04 (Xenial) but will most likely work for setting up a Go development environment on other versions of linux. You can still follow this guide if you are using OSX but some small things will differ such as the binary to download. You could always run Go on Linux in a Virtual Machine with Virtual Box.

Continue reading “How to: Developing with Go on Linux”

I made a simple Bcrypt hash generator website with Golang

Sometimes when I am working with dummy database I might need to change the password of a generated user. The easiest way to do this is to got to google for a bcrypt generator and then paste the generated hash into the database in order to update the password. This is much quicker than having to write a function to do this for me or to create a new user from scratch. It is also useful if I need to reset a password to an account which I have totally lost access to. Turns out I don’t like many of the bcrypt generators that are out there and the domain bcrypt.fun was available so I figured, why not make my own generator since I already created password.fun.

Continue reading “I made a simple Bcrypt hash generator website with Golang”