GitLab Mattermost Token request failed

Including some addtional information that I found userful.

I have this same issue, but wanted to document how I solved this issue since this is one of the top google search results regarding the x509: certificate signed by unknown authority issue.

I read through all the other links / issues mentioned here and did run across a couple others

• http://vaelen.org/2016/02/05/adding-certificates-to-ubuntu-and-gitlab/

But ultimately, I would recommend reading the following blog post that describes why SSL certs are different on different systems and why this causes so much headache:

• https://www.happyassassin.net/2015/01/12/a-note-about-ssltls-trusted-certificate-stores-and-platforms/

In my specific use case, we run a RHEL / Centos 6 server.

While I had already added my Internal CA root certificate to the /etc/pki/ca-trust/source/anchors/ directory and then ran the update-ca-trust as root. This updates certificates in the /etc/pki/ca-trust/extracted directory.

After updating this and forcing the GitLab CA certificate to point to this extracted certificate, I still could not get Mattermost to connect properly:

[root@server /]# ls -la /opt/gitlab/embedded/ssl/certs/
total 8
drwxr-xr-x. 2 root root 4096 Jul 21 13:19 .
drwxr-xr-x. 4 root root 4096 Jul 21 12:47 ..
lrwxrwxrwx. 1 root root   55 Jul 21 13:02 cacert.pem -> /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt

It wasn't until I saw that there was another certs directory from the HappyAssassin blog post that was not being updated. The /etc/ssl/certs is a sym link to /etc/pki/tls/certs and in here these certificates did not have my internal root CA certificate.

So I removed these and pointed this these to use certificates from the extracted directory:

[root@server /]# ls -la /etc/ssl/
total 16
drwxr-xr-x.   2 root root  4096 Jul 21 08:30 .
drwxr-xr-x. 122 root root 12288 Jul 21 13:29 ..
lrwxrwxrwx.   1 root root    16 Jul 21 08:30 certs -> ../pki/tls/cert
[root@server /]# ls -la /etc/ssl/certs/
total 20
drwxr-xr-x. 2 root root 4096 Jul 21 13:27 .
drwxr-xr-x. 5 root root 4096 Jul 21 08:30 ..
lrwxrwxrwx. 1 root root   49 Jul 21 13:27 ca-bundle.crt -> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
lrwxrwxrwx. 1 root root   55 Jul 21 13:27 ca-bundle.trust.crt -> /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
-rwxr-xr-x. 1 root root  610 May  9 08:32 make-dummy-cert
-rw-r--r--. 1 root root 2242 May  9 08:32 Makefile
-rwxr-xr-x. 1 root root  829 May  9 08:32 renew-dummy-cert

It wasn't until I updated these then did one more gitlab reconfigure that Mattermost would finally recognize the certificates from GitLab.

I'm not sure if this is something that is encoded into Mattermost or possibly one of its dependencies, but this was pretty difficult to track down especially since every system does SSL certificates differently and applications/software all look for these in different ways.

Hopefully this helps someone else or at least provides them with background on the SSL issues since that blog post really helped me figure out the core issue here.