Run your own NameServers with CoreDNS

I have a lot of domains and I wanted an easy way to manage all of them without having to deal with some kind of API from a third party. For learning purposes I decided to give CoreDNS a try. All of my domains are for my own hobby projects and I decided to try running my own nameservers to learn. There are plenty of free options available and it is not recommended to run your own nameservers.

The things I will be using in this guide are mainly:

  • CoreDNS
  • Ansible
  • Hetzner cloud – instances are in Finland, costs around 6 euro per month
  • Namesilo (affiliate link, use code GOPHP to get $1 off ) – for registering our nameservers

Hosting

I went ahead and deployed 2 Hetzner cloud instances since they are cheap and easy to deploy. I am also based in Europe which puts the NameServers close to me which is good for me, not so good for my US users. You could also use AWS EC2 or Google Cloud Compute instances or anything else you prefer.

Domains

I wanted a super simple way to manage my domains so I decided to make use of Ansible to deploy my NameServers. This allows me to keep track of all configurations in one big YAML file. Here is an example of how I have defined my domains.

---
# roles/common/vars/main.yml
domains:
  - domain: 'uberswe.com'
    mx:
      - '10 in1-smtp.messagingengine.com'
      - '20 in2-smtp.messagingengine.com'
    txt:
      - 'v=spf1 include:spf.messagingengine.com ?all'
    sub:
      - domain: 'fm1._domainkey'
        cname:
          - 'fm1.uberswe.com.dkim.fmhosted.com'
      - domain: 'fm2._domainkey'
        cname:
          - 'fm2.uberswe.com.dkim.fmhosted.com'
      - domain: 'fm3._domainkey'
        cname:
          - 'fm3.uberswe.com.dkim.fmhosted.com'
  - # bokföring.xyz
    domain: 'xn--bokfring-q4a.xyz'
  - # bolån.xyz
    domain: 'xn--boln-soa.xyz'

Here we have a variable called domains with a list of objects. Each object needs to specify a domain and the rest is optional.

CoreDNS Configuration

I can use the domains variable to create dns zone files and a CoreDNS configuration file which our dns server will read. We can use a Jinja template to make our configuration file. Below is the template I made.

.:53 {
    forward . 8.8.8.8
    log
    errors
    cache
}

{% for d in domains %}
{{d.domain}} {% if 'sub' in d %}{% for s in d.sub %}{{s.domain}}.{{d.domain}} {% endfor %}{% endif %}{
    file /etc/coredns/zones/{{d.domain}}.db
    log
    cache
    errors
}

{% endfor %}

Our nameservers will forward any requests it doesn’t match to 8.8.8.8 as a fallback. This file will take the domains list we defined in our YAML file and loop through each domain object specifying the db file for each domain.

I decided to use files as CoreDNS has a file plugin which seemed easy to use. I would like to try using the redis plugin in the future to load zone data from a redis instance. Below is my Jinja file which takes each object in the YAML file above and uses it to generate a dns zone file.

$ORIGIN {{ item.domain }}.
@	3600 IN	SOA ns1.beubo.com. admin.beubo.com. 2017042745 7200 3600 1209600 3600
	3600 IN NS ns1.beubo.com.
	3600 IN NS ns2.beubo.com.
        IN A	95.216.12.246
        IN AAAA	2a01:4f9:2a:d09::2
{% if 'mx' in item %}{% for mx in item.mx %}        IN MX  {{mx}}.
{% endfor %}{% endif %}
{% if 'txt' in item %}{% for txt in item.txt %}        IN TXT  "{{txt}}"
{% endfor %}{% endif %}
{% if 'cname' in item %}{% for cname in item.cname %}        IN CNAME  {{cname}}
{% endfor %}{% endif %}

*       IN A     95.216.12.246
        IN AAAA  2a01:4f9:2a:d09::2

{% if 'sub' in item %}{% for d in item.sub %}
{% if 'ipv4' in d %}{{d.domain}}       IN A  {{d.ipv4}}
{% endif %}
{% if 'ipv6' in d %}{{d.domain}}       IN AAAA  {{d.ipv6}}
{% endif %}
{% if 'mx' in d %}{% for mx in d.mx %}{{d.domain}}        IN MX  {{mx}}.
{% endfor %}{% endif %}
{% if 'txt' in d %}{% for txt in d.txt %}{{d.domain}}        IN TXT  "{{txt}}"
{% endfor %}{% endif %}
{% if 'cname' in d %}{% for cname in d.cname %}{{d.domain}}        IN CNAME  {{cname}}.
{% endfor %}{% endif %}

{% endfor %}{% endif %}

Currently the nameservers, SOA record and IP addresses for the root domains are hardcoded but could easily be replaced with variables. The basic idea here is that I check if an item has a key then I echo the value. I also added the sub item which contains any sub domains so I use a loop for that item. Here is an example zone file which I have generated for my dns server.

$ORIGIN uberswe.com.
@	3600 IN	SOA ns1.beubo.com. admin.beubo.com. 2017042745 7200 3600 1209600 3600
	3600 IN NS ns1.beubo.com.
	3600 IN NS ns2.beubo.com.
        IN A	95.216.12.246
        IN AAAA	2a01:4f9:2a:d09::2
        IN MX  10 in1-smtp.messagingengine.com.
        IN MX  20 in2-smtp.messagingengine.com.
        IN TXT  "v=spf1 include:spf.messagingengine.com ?all"

*       IN A     95.216.12.246
        IN AAAA  2a01:4f9:2a:d09::2

fm1._domainkey        IN CNAME  fm1.uberswe.com.dkim.fmhosted.com.

fm2._domainkey        IN CNAME  fm2.uberswe.com.dkim.fmhosted.com.

fm3._domainkey        IN CNAME  fm3.uberswe.com.dkim.fmhosted.com.

We can write an ansible test to ensure that this is working as it should before we try to run this on our nameservers. Below is my test file.

---
# tests/test.yml
- hosts: 127.0.0.1
  vars_files:
    - ../roles/common/vars/main.yml
  tasks:
    - name: Create a directory
      file:
        path: zones
        state: directory
      delegate_to: localhost

    - name: Test dns zone files generation
      template: src=../roles/dnstier/templates/dnszone.db.j2 dest=zones/{{ item.domain }}.db
      with_items: "{{ domains }}"
    
    - name: Create a directory
      file:
        path: files
        state: directory
      delegate_to: localhost

    - name: Test Corefile for coredns generation
      template: src=../roles/dnstier/templates/Corefile.j2 dest=files/Corefile

You can run this with ansible-playbook tests/dnszones.yml and it should make a folder called zones which will contain a .db file for each domain. A files directory with the Corefile will also be created.

Deployment Configuration

To setup and configure CoreDNS I found ansible role which was already ready to use at https://github.com/cloudalchemy/ansible-coredns. We can install this role using the command ansible-galaxy install cloudalchemy.coredns.

We need to generate the zone files locally so I created a role called dnstier which will prepare these files using the following task file.

---
# file: roles/dnstier/tasks/main.yml
- name: Create a directory
  file:
    path: /tmp/zones
    state: directory
  delegate_to: localhost

- name: dns zone files generation
  template: src=../templates/dnszone.db.j2 dest=/tmp/zones/{{ item.domain }}.db
  with_items: "{{ domains }}"
  delegate_to: localhost

Now we can set up our main playbook which will run both of these roles.

---
# file: dnsservers.yml
- hosts: dnsservers
  roles:
    - dnstier
    - cloudalchemy.coredns

We also need to define our variables for the cloudalchemy.coredns role. This is how I defined them.

---
# file: group_vars/all.yml

coredns_config_file: "roles/dnstier/templates/Corefile.j2"

coredns_zone_files_paths:
  - "/tmp/zones/*"

coredns_system_group: "coredns"
coredns_system_user: "{{ coredns_system_group }}"

The variables I have defined here come from the readme of the coredns role we installed with galaxy. The config file is the path to the template file we defined earlier and the zone file path is the same as we defined in our dnstier task. I also have a hosts file defined like so.

---
# host_vars/all.yml
all:
  children:
    dnsservers:
      hosts:
        beubo:
        ns2beubo:

Here beubo and ns2beubo are the host names I have defined in my ssh config for my two nameservers. We could add more hosts here and ansible will deploy this same setup to each one.

Deploying the Nameservers

Now we can finally go ahead and deploy using the command ansible-playbook dnsservers.yml which will:

  • Run the dnstier role, generating the zone files.
  • Run the cloudalchemy.coredns role which installs our coredns server .

After the deployment is done we can check that it is working by specifying the IP of each dns server with a @ symbol in the dig command. You can run this for example to query my domain and server dig @95.216.12.246 uberswe.com

We can now register our nameservers. I am doing this with Namesilo but most registrars should provide an easy way of doing this. I will enter ns1 and ns2 as the prefixes and enter a ipv4 and ipv6 address for each nameserver. You can find the ip addresses on your Hetzner cloud instance page.

Namesilo page to add a new registered nameserver.

Once you have done that and waited some time for the dns to propagate you can query a domain using your server. We can replace the ip in our previous dig query with the domain like so dig @ns1.beubo.com uberswe.com.

If that is working like it should, you should then be ready to point your domains to your new nameservers. Feel free to leave a comment if you have questions, see something I did wrong or just want to wish me a good day.

If this stuff interests you, you may be interested in some relevant topics I found while setting up my nameservers:

Minimalism in Web Development

As I spend more and more time working with bloated frameworks in web development I am starting to miss the minimalism I saw when I got started. I wish websites didn’t need to load megabytes of data to display simple pages. I believe a lot of websites do not benefit at all from frameworks like React, Vue and Angular. Some have told me that these pages just need to be optimized but as far as I can tell this just spreads the loading of files out over time, you will still end up with a site of several megabytes after navigating pages.

How do we define minimalism?

I believe we need less javascript and css for the web. Things can still look great and be done quickly without a bunch of frameworks. I believe this is what we like to call minimalism in web development. Looking at Wikipedia I found a good definition for minimalism and it’s defined like so:

In software and user interface design, minimalism describes the usage of fewer design elements, flat design, fewer options and features, and tendencially less occupied screen space. 

This definition fits well and I believe it’s good to follow this principle. However, I have seen many minimalist user interfaces that look great but take ages to load and use megabytes of data. If we try to use minimalism in web design shouldn’t we also use minimalism when it comes to the underlying code? I found an article called My Minimalist Approach To Software Engineering where the author touches on this topic. Libraries and frameworks can be good but there is a lot we can do in web development without them. In this article and hopefully future articles I would like to explore what we can do with minimalist web development.

In addition to what I have written here there is an article called A Short Guide To Minimal Web Development which I found while researching this topic. I think it outlines a lot of the things for frontend development which I would like to cover but I also want to apply the principle to backend development. Frameworks and libraries have their uses but they are not needed for most web applications.

The idea, however, is that we can only tell what’s really needed once we can tell what’s really important, and why it’s really important.

Jens Oliver Meiert (https://meiert.com/en/blog/minimal-web-development/)

When do we need a backend?

You may need a backend to a web application when you have dynamic content. Now, there is no simple definition for dynamic content. Basically, if you have any form of user authentication or storage shared between users you will need a backend. A lot of things such as blogs, games, live chats and more may have dynamic content but do not need any backend to perform their function if authentication is not needed. Most sites will fall into the first category of authentication but there are some sites out there which need a backend but do not require authentication. These types of sites may exists to provide simple functions like uploading and sharing files or tracking and storing user behaviour.

Many sites like blogs or even news sites will do fine with basic HTML files with no extra code. However, for convenience some people do like to use a backend such as WordPress. WordPress lets you easily create new posts and pages. This does however come with the added cost of complexity and performance. HTML files tend to load quickly and there is very little that can go wrong. Once you add a backend with authentication there may be hundreds of checks that execute on every page load. You also have to maintain a backend, keep it updated and ensure that malicious users don’t gain access to it.

When making a new site you should always begin by asking if you really need a backend.

When is a backend framework needed?

A backend framework is never really needed. But a framework can help speed things up and provide a ready made structure. It’s similar to building a house, you build everything from scratch or use prefabricated houses where all the pieces are already made.

A framework therefore helps standardise things so that it is easier for multiple developers to work on the same project. However, there should always be some consideration as to if a framework is really needed. If you’re building a landing page you don’t need frameworks. You could probably use a simple PHP file for this instead of a full framework like Laravel. Using a framework will add overhead and complexity to the application, especially if it’s small. In the end you are trading development speed for flexibility, stability, performance and security.

When is a frontend framework needed?

Frontend frameworks like React, Angular and Vue have become very popular in the last years. I think these frameworks are needed even less than backend frameworks. Most websites do not need any javascript at all to provide their core functionality. The backend can generate html pages with any needed data and this can be styled with a bit of css.

One argument I have heard is that these frameworks allow you to load a page once and then only make requests for the data that you need. This is referred to as a single page application and I can see how it provides benefits for large applications with lots of data. However, most websites do not load large sets of data in this way and can provide pages that are small in size and load quickly. Many single page applications load so many different libraries that they often reach over a megabyte in size which is huge.

You end up trading a slow initial load for many smaller pages that load quickly. It’s also tends to become impossible to keep track of all the libraries that these frontend applications use. There have been several recent incidents where a small package had a bug or security issue which affected millions of applications. Frontend frameworks should be avoided unless they provide substantial benefits. Frontend frameworks might allow you to get something produced more quickly but once again you trade development speed for flexibility, stability, performance and security.

Conclusion

As I gain more and more experience in web development I am starting to miss the simplicity and speed of the old days. I think we need to find a way back to the simple application. I am personally starting to simplify the websites I run.

As I write this I am using WordPress to manage my posts. However, I am working on a new backend called Beubo. With Beubo I want to simplify the backend to something very basic and rethink how the backend should work. I want to write guides that teach minimalism in web development from the very start. Too many web developers start with frameworks and never learn the simplicity of vanilla CSS, JS and HTML.

I think there is more and more push for simplicity and minimalism in web development. This is as companies become more and more liable for the security of their applications. For a long time there were no real consequences of running insecure web applications. Focus has shifted towards development speed. However, with the rise of GDPR and similar laws we will see more companies looking to simplify. To reduce the amount of libraries and frameworks that are used in order to keep applications more secure and stable.

I would love to hear what you guys think about minimalism in web development. Please leave your comment below.

How Switching Domain Registrars Saved Me $300+/Year

When registering a domain it’s typically easy to find domain registrars with good deals, perhaps just $1 to get any domain name you want. That’s a trick to reel you in in the hopes that you will keep that domain for many years without switching registrar. The renewal fees and any additional offerings such as hosting and email is where most domain registrars earn the most money. Today, however, I want to simply take a look at renewal fees and how much extra we spend on renewals. If you are a company or someone like me with 100+ domains these renewal fees can get very expensive. Here I will show how much you can actually save on renewal fees by switching domain registrars and how I went about figuring this out.

Continue reading “How Switching Domain Registrars Saved Me $300+/Year”

Customer Feedback via SMS using Go and Slack

Using an SMS Gateway called 46elks and some Go we can automate a customer feedback process which is usually done manually by calling a customer. This was an idea I had when I worked at a large sales company in Sweden a few years ago and my task was to find things that could be automated to save money and speed things up. To keep track of feedback and manually handle negative comments I also piped the messages to Slack which we will be doing in this guide as a bonus at the end. The following is a how to guide describing how you can create this yourself. You can find the full source code on Github.

SMS Gateway

The first thing we need to is find a SMS Gateway for sending our text messages to our customers. I met 46elks at HackForSweden this year and it seems to perform really well when I used them for this guide. However, you could go with any SMS Gateway and some of the more popular options are Twilio, Amazon or ClockworkSMS. Most gateways will provide similar APIs so it shouldn’t be too hard to replace my 46elks implementation with another provider.

Sending texts and receiving customer feedback

Continue reading “Customer Feedback via SMS using Go and Slack”

How To Run A Go Web Server On AWS

Amazon Web Services (AWS) offers free tier web servers where you can run simple go programs for API backends or websites. In this guide we will make a simple web server using a go program. This guide can also be used as a reference to set up AWS instances for larger go services.

To begin you will need to set up a new instance on AWS. To do this simply login to your AWS account and click on Services at the top of the screen and then on EC2. EC2 should be the first option under the Compute category.

Click on Instances on the left hand side and then in the top left there should be a large button called Launch Instance which will start a creation wizard. I typically search for Ubuntu and pick 18.04 as shown in the screenshot below.

Continue reading “How To Run A Go Web Server On AWS”

How to use Pi-hole to block all ads on your local network

Recently I decided to set up a Pi-hole installation on my local network. Initially I was going to use a Raspberry Pi 3 B+ but the company I work for was replacing all the PCs and I was able to get my hands on one of them.

Pi-hole is a simple tool which can be configured to handle the DNS of all the devices on a network. It can be installed on a linux environment and is also available as a docker image. Basically it will check any DNS queries against various blacklists and block queries that match ad networks or tracking domains. The end result is that you get a browsing experience without advertisements and user tracking. Pi-hole is configurable and you can whitelist domains or even devices on the network if you would like. The admin interface shown in the image of this post shows the dashboard which gives a nice overview of queries that go through Pi-hole.

Continue reading “How to use Pi-hole to block all ads on your local network”

How to Install Go 1.12 on Ubuntu 18

This guide is also available as a video from Youtube.

Begin by running the following command to add the golang backports ppa:

sudo add-apt-repository ppa:longsleep/golang-backports

If you get an error such as “add-apt-repository command not found” then you need to run the following command to install add-apt-repository:

Continue reading “How to Install Go 1.12 on Ubuntu 18”

Create a Public Slack Community with Go

I recently stumbled across a tweet by Melvin Davis on Twitter where he mentioned a simple tool that he had created using Go that instantly sends a Slack invite when an email is entered. This allows us to make a public slack community that anyone can join.

Check-out the auto invite sender to join #slack written in #Golang https://t.co/fwcurIsZ3w This tool is extremely helpful when you maintain an community slack channel— Melvin Davis (@melvinodsa) January 2, 2019

Continue reading “Create a Public Slack Community with Go”

Developing Slack Bots with Go: Part 1

In my previous post I wrote a guide on how to post a simple Slack message using a bot with PHP. Now we will do the same thing but instead with Go. I have also decided to send a json request instead of a form-url encoded request like we did with PHP. There are some differences, let’s take a look.

Continue reading “Developing Slack Bots with Go: Part 1”

Developing Slack Bots with PHP: Part 1

I have done a lot of work with slack bots in the past both professionally and in my spare time. In the past I build a plugin for WHMCS called WHMCS Slack which was originally a paid addon but is now open source and available for free on Github. This post is a first part in a series of posts i would like to write about both Go and PHP and how we can use these programming languages to create useful bots for Slack.

Continue reading “Developing Slack Bots with PHP: Part 1”