Oursky Code
  • AI
  • Auth
  • More from Oursky
    • Oursky Business Blog
    • GitHub
    • About Us
  • AI
  • Auth
  • More from Oursky
0
Subscribe
Oursky Code
Oursky Code
  • AI
  • Auth
  • More from Oursky
    • Oursky Business Blog
    • GitHub
    • About Us
  • Opensource

How to set up an internal team forum in half a day using Discourse

  • January 29, 2018
  • No comments
  • 5 minute read
  • Ben Cheng
discourse forum deployed on kubernetes k8s
Oursky’s Discourse platform — deployed in our own Kubernetes (K8s) cloud.
Total
0
Shares
0
0
0

TL;DR: I’ve deployed Discourse on Kubernetes (K8s) for my company’s internal discussion platform. Because I couldn’t find a simple tutorial, I documented my steps to help other developers do it too.

Why would I want to deploy Discourse on Kubernetes?

  1. Our company already has a Kubernetes cluster for random tools and staging deployment, so it is cheaper to deploy on the existing cluster for an internal Discourse usage.
  2. As a founder, I don’t get many chances to code anymore. I wanted to learn how to use Kubernetes because my team has been using it a lot lately.

A quick overview of this tutorial

The tutorial and the sample config below shows how to deploy a single Discourse web-server. This server needs to connect to a PostgreSQL and Redis server. We are using Google Cloud Registry and gcePersistentDisk for storage.

So let’s begin:

Create Discourse app Docker image

We will “misuse” the launcher provided by discourse_docker to create the docker image for the Discourse web server. And by “misuse” I mean that we have over-used the launcher script to create docker images for production use.

1.Clone from https://github.com/discourse/discourse_docker to your local environment.

2.Setup a temporary Redis server and PostgresSQL database in the local environment.

3.Create a containers/web_only.yml (as shown below)

  • The env var is not relevant to K8s. It’s just for building the local image, so let’s fill in something that works for your local environment.
  • Determine the plugins you want to install with your Discourse setting here.

4.Tip: The local Redis instances might be in protected-mode, and won’t allow a Docker guest to host the connection. For this case, you should start your Redis server with the protected-mode turned off: redis-server --protected-mode no

5.Create the Docker images and upload the images to your K8s Docker registry. In this case, we are using Google Cloud Registry:

  • Create Docker image with Discourse’s launcher: ./launcher bootstrap web_only
  • Verify that the image is created: docker images. You should see the Discourse image in the list if successful.
  • Upload the image to the registry with this command:


docker tag local_discourse/web_only gcr.io/**my-cluster**/discourse:latest
gcloud docker — push gcr.io/**my-cluster**/discourse:latest

view raw

gistfile1.txt

hosted with ❤ by GitHub

Sample config in web_only.yml :


templates:
– "templates/web.template.yml"
– "templates/web.ratelimited.template.yml"
env:
LANG: en_US.UTF-8
UNICORN_WORKERS: 2
DISCOURSE_DB_USERNAME: chpapa
DISCOURSE_DB_PASSWORD: ''
DISCOURSE_DB_HOST: docker.for.mac.localhost
DISCOURSE_DB_NAME: chpapa
DISCOURSE_DEVELOPER_EMAILS: 'bencheng@oursky.com'
DISCOURSE_HOSTNAME: 'localhost'
DISCOURSE_REDIS_HOST: docker.for.mac.localhost
hooks:
after_code:
– exec:
cd: $home/plugins
cmd:
– mkdir -p plugins
– git clone https://github.com/discourse/docker_manager.git
– git clone https://github.com/discourse/discourse-solved.git
– git clone https://github.com/discourse/discourse-voting.git
– git clone https://github.com/discourse/discourse-slack-official.git
– git clone https://github.com/discourse/discourse-assign.git
run:
– exec:
cd: /var/www/discourse
cmd:
– sed -i 's/GlobalSetting.serve_static_assets/true/' config/environments/production.rb
– bash -c "touch -a /shared/log/rails/{sidekiq,puma.err,puma}.log"
– bash -c "ln -s /shared/log/rails/{sidekiq,puma.err,puma}.log log/"
– sed -i 's/default \$scheme;/default https;/' /etc/nginx/conf.d/discourse.conf

view raw

web_only.yml

hosted with ❤ by GitHub

Now we’re ready to deploy to K8s

1. Prepare a persistent volume

We will need a persistent volume as the database to store the user info and the discussion items. We are using GCEPersistentDisk for the persistent disk on the K8s cluster. Now, let’s create two 10GB disks for the app and database respectively. You may consider your Discourse usage to adjust the disk size configuration.


gcloud compute disks create –size=**10GB** –type=**pd-ssd** –zone=**us-east1-b** **discourse**
gcloud compute disks create –size=**10GB** –type=**pd-ssd** –zone=**us-east1-b** **discourse-pgsql**

view raw

gcloud_disksettings.sh

hosted with ❤ by GitHub

2. Deploy to Kubernetes

Next, we will configure the deployment settings of the K8s cloud. Customize the sample K8s file. Here are some variables you probably want to tweak:

volumes.yaml

  • For both persistent volumes:
    • metadata.name
    • spec.capacity.storage
    • spec.gcePersistentDisk.pdName (for the persistent disk name above)
    • spec.claimRef.namespace (for the namespace you’re using in K8s)
  • The sample file here assumes you are using gcePersistentDisk. volumes.yaml needs to change heavily depending on what type of persistent disk you plan to use.

redis.yaml

  • Redis Deployment:
    • spec.template.spec.containers.resources.* (CPU and Memory resources for cache server)

pgsql.yaml

  • PersistentVolumeClaim (pgsql-pv-claim):
    • spec.resources.requests.storage (storage of DB server)

discourse.yaml

  • PersistentVolumeClaim (discourse-pv-claim)
    • spec.resources.requests.storage (storage of web-server disk for logs and backups)
  • Deployment (web-server)
    –spec.template.spec.containers.image (Set the URL to point to your Docker image)
    –spec.template.spec.containers.env

DISCOURSE_DEVELOPER_EMAILS
DISCOURSE_HOSTNAME
DISCOURSE_SMTP_ADDRESS
DISCOURSE_SMTP_PORT

  • spec.template.spec.containers.resources.* (CPU and Memory resources for your web-server)

ingress.yaml

  • spec.rules.host
  • spec.tls.hosts

Recommended: From there, you might want to create your own namespace for the deployment. Also assume you have set the right context to run the kubectl command in the namespace. (For details, read Kubernetes documentation). Otherwise, you should rename most of the names in the config files above to unique names and add some labels.

Apply secrets. dbUsername and dbPassword can be anything you want. Please set the right smtpUsername and smtpPassword for the mail delivery services you use.

Another note on HTTPS for Ingress: you should read this document and the Ingress controllers specific to your cluster and update ingress.yamlaccordingly.


apiVersion: v1
kind: Secret
metadata:
name: secret
type: Opaque
data:
dbUsername:
dbPassword:
smtpUsername:
smtpPassword:

view raw

secret.yaml

hosted with ❤ by GitHub

Apply all config files:


kubectl apply -f secret.yaml
kubectl apply -f volumes.yaml
kubectl apply -f redis.yaml
kubectl apply -f pgsql.yaml

view raw

apply_configs.sh

hosted with ❤ by GitHub

Before starting the app, run the following on PostgreSQL instance to initialize the database properly. You can find your pod name by running kubectl get pods.


kubectl exec **pgsql** — su postgres -c 'psql template1 -c "create extension if not exists hstore;"'
kubectl exec **pgsql** — su postgres -c 'psql template1 -c "create extension if not exists pg_trgm;"'
kubectl exec **pgsql** — su postgres -c 'psql **discourse** -c "create extension if not exists hstore;"'
kubectl exec **pgsql** — su postgres -c 'psql **discourse** -c "create extension if not exists pg_trgm;"'

view raw

init_pgsql.sh

hosted with ❤ by GitHub

Create Discourse deployment and Ingress with these commands:


kubectl apply -f discourse.yaml
kubectl apply -f ingress.yaml

view raw

create_deployment.sh

hosted with ❤ by GitHub

From here, your Discourse instance should be up and running. Below are some useful commands in case things don’t work and require debugging:


# Check logs of k8s pod
kubectl logs –since=1h –tail=50 -lapp=web-server
# Open an interactive terminal into the web-server / pgsql server:
kubectl exec -it **web-server** — /bin/bash
kubectl exec -it **pg-sql** — /bin/bash
# You might want to do the following:
# * Delete / create the pgsql database
# * Check logs under /shared/log/rails
# * Start stop unicorn by sv stop unicorn
# * Run bundle exec rake db:migrate / admin:create in case something went wrong under /var/www/discourse
# * You can also check the logs with an admin account at /admin/logs of your deployment

view raw

k8s_debugg_message.sh

hosted with ❤ by GitHub

Setup S3 backup and file upload

Discourse can use AWS S3 for backup and file upload. Here are the steps to enable it:

  1. Create two S3 buckets: one for backup and one for file upload. Set them as private.

  2. Create an IAM user with API access only and attach the AWS inline policy below:

https://gist.github.com/ourskycode/12a1e1f77883fd590c615ed9be73676f#file-aws-s3-json

  1. Fill in the Access Key and Key ID in Discourse Setting.

Then Discourse can upload files to the S3 bucket you’ve specified, so you can attach image and file attachments in each post.

That’s it!

I hope this piece is helpful for you to set up your own Discourse platform. It’s also a practical exercise for me to try deploying an app on K8s.

Potential improvement and scaling up:

  • It’s possible to run multiple replicas for Discourse web-server for scaling up. It should work, but I haven’t tried yet.
  • We could also deploy Master-Slave PostgreSQL for scaling up. We’re using Bitnami’s PostgreSQL docker image and you can read the relevant instructions here.

Building an app? Our free developer tools and open source backend will make your job easier.

Read more from Oursky

Total
0
Shares
Share 0
Tweet 0
Pin it 0
Related Topics
  • discourse
  • k8s
  • Kubernetes
  • open source project
Ben Cheng

Previous Article
https://unsplash.com/@dsmacinnes
  • UX/UI Design

Should you use Sticky Header?

  • January 3, 2018
  • Ken Chan
View Post
Next Article
carmen female developer
  • Software Testing

A recipe for website automated tests with Python Selenium & Headless Chrome in Docker

  • March 30, 2018
  • Joyz Ng
View Post
You May Also Like
View Post
  • DevSecOps
  • Opensource
  • Security and Privacy

Kubernetes Security – Network Encryption between k8s Deployments and Ingress

  • September 4, 2020
  • Elliot Wong
oursky opensource github
View Post
  • Android
  • Opensource

An Opensourced Recipe for Intializing Redux x Android Native Apps

  • May 9, 2018
  • David Ng
View Post
  • Opensource

A spreadsheet-based API for building MVPs.

  • July 20, 2017
  • David Ng
View Post
  • DevOps
  • Opensource

How I built a Kubernetes cluster so my coworkers could deploy apps faster

  • May 26, 2017
  • cheungpat
View Post
  • iOS
  • Opensource

Continuously Delivering iOS Beta Builds Automated with Travis CI

  • March 26, 2015
  • David Ng

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Subscribe Us

Join our mailing list to never miss an update!

Oursky Code Blog
A team of Developers, Designers and Geeks. All about hacking and building things.

Input your search keywords and press Enter.