Gitlab Pages Domain and Subdomain verification
I am writing this blog in hope that someone else who is trying to do the same can some hours of their time and not go through the frustration I went through. In hindsight, it was right there but I couldn't see it until I had the right tools to debug the issue.
Enough of the ramble. Let's get to it.
GitLab Pages is a service that allows you to host your static web site from the git repository hosted on the same site. It allows you to have custom domain names along with TLS certificates etc. I will write a seperate blog about how to do this in seperate post.
When you add custom domain name to gitlab pages (settings -> pages -> domains), it will ask you to verify that you own the domain by creating a DNS TXT record in the domain. It will give you of the form
_gitlab-pages-verification-code.yourdomain.com TXT gitlab-pages-verification-code=xxxxxxxxxxxxxxxx
I immediately went to godaddy.com where my domain is hosted and fired up the manage DNS for my domain. Sure enough it was easy click on Add in godady.
For Name I entered _gitlab-pages-verification-code.yourdomain.com For Value I entered gitlab-pages-verification-code=xxxxxxxxxxxxxxxx
I left other fields to default. The record got added to my domain.
However, gitlab was having trouble verifying my domain because it would query for TXT record to see if it available as part of DNS and it would always say that it is not available.
I initially thought it was because of DNS propagation delay so I left it at that. I came after more than 24 hours thinking that DNS would have been updated but I still got unable to verify error message.
Frustrating thing about this exercise was that Gitlab pages doesn' give you any error message. It doesn't give log of verification process so that you can see what is going on and try to fix it. Nothing.
Being a developer, I went to figure out what is going on and to debug the issue. For that I needed a way to see what is going on.
Dig stands for domain information gopher. It is tool for querying DNS servers. It is available in most MAC and Unix systems. In Windows, you need to install Bind tools to get them.
For exmaple, let's use Dig to query Google.com domain
$ dig google.com ; <<9.10.6 <<>> google.com ;; global options: +cmd ;; Got answer: ;; - HEADER<<- opcode: QUERY, status: NOERROR, id: 25516 ;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;google.com. IN A ;; ANSWER SECTION: google.com. 99 IN A 22.214.171.124 google.com. 99 IN A 126.96.36.199 google.com. 99 IN A 188.8.131.52 google.com. 99 IN A 184.108.40.206 google.com. 99 IN A 220.127.116.11 google.com. 99 IN A 18.104.22.168 ;; Query time: 2 msec ;; SERVER: 192.168.1.1#53(192.168.1.1) ;; WHEN: Sun Nov 04 21:38:33 +08 2018 ;; MSG SIZE rcvd: 135DiG
There are three sections to dig command
- Question section
- Answer section
- Addition information section
We can use Dig to query for specific type of record. Let's ask Dig to provide all MX records of Google.com
$ dig MX google.com ; <<9.10.6 <<>> MX google.com ;; global options: +cmd ;; Got answer: ;; - HEADER<<- opcode: QUERY, status: NOERROR, id: 60401 ;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 3 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;google.com. IN MX ;; ANSWER SECTION: google.com. 559 IN MX 20 alt1.aspmx.l.google.com. google.com. 559 IN MX 10 aspmx.l.google.com. google.com. 559 IN MX 40 alt3.aspmx.l.google.com. google.com. 559 IN MX 30 alt2.aspmx.l.google.com. google.com. 559 IN MX 50 alt4.aspmx.l.google.com. ;; ADDITIONAL SECTION: aspmx.l.google.com. 299 IN A 22.214.171.124 aspmx.l.google.com. 645 IN AAAA 2404:6800:4003:c03::1b ;; Query time: 2 msec ;; SERVER: 192.168.1.1#53(192.168.1.1) ;; WHEN: Sun Nov 04 21:40:28 +08 2018 ;; MSG SIZE rcvd: 191DiG
As you can see this is such a powerful tool to debug domain query related issues.
Now coming to my original problem. I tried to do
$ dig TXT _gitlab-pages-verification-code.yourdomain.com
and it returned nothing. Now, I know why gitlab pages wasn't able to verify because it was not getting the response back from DNS server. However, I do see that the record is added to the DNS server in my godaddy manage domains dashboard.
As I was looking at other records in Manage domains page, it dawned on me that I only specify the prefix part of the subdomain and not the whole domain. For example, I only specify www and not mydomain.com.
if you look at the KEY value for TXT record, it has the domain value appended to the end of it. Is it why it wasn't working? To check it, I tried to do the following
$ dig TXT _gitlab-pages-verification-code.yourdomain.com.yourdomain.com
Notice that I have appended yourdomain.com one more time. Boom!, I got response back.
See what happens is that whatever you enter as key gets appended with your domain name when godaddy writes it to the zone file.
The fix is simply to remove the domain name at the end when adding using UI.
Yeah I know it is stupid of me for not seeing it but it wasn't very obvious. Hopefully this post helps people not spend the hours I spend and hopefully use the tool like dig to debug the issue