Stateful Containers using Portworx and Couchbase

Arun GuptaMarch 10th, 2017Last Updated: March 10th, 2017

0 32 4 minutes read

Containers are meant to be ephemeral and so scale pretty well for stateless applications. Stateful containers, such as Couchbase, need to be treated differently. Managing Persistence for Docker Containers provide a great overview of how to manage persistence for stateful containers.

This blog will explain how to use Docker Volume Plugins and Portworx to create a stateful container.

Why Portworx?

Portworx is an easy-to-deploy container data services that provide persistence, replication, snapshots, encryption, secure RBAC and much more. Some of the benefits are:

Container granular volumes – Portworx can take multiple EBS volumes per host and aggregate the capacity and derive container granular virtual (soft) volumes per container.
Cross Availability Zone HA – Portworx will protect the data, at block level, across multiple compute instances across availability zones. As replication controllers restart pods on different nodes, the data will still be highly available on those nodes.
Support for enterprise data operations – Portworx implements container granular snapshots, class of service, tiering on top of the available physical volumes.
Ease of deployment and provisioning – Portworx itself is deployed as a container and integrated with the orchestration tools. DevOps can programmatically provision container granular storage with any property such as size, class of service, encryption key etc.

Setup AWS EC2 Instance

Portworx runs only on Linux or CoreOS. Setup an Ubuntu instance on AWS EC2:

Start Ubuntu 14.04 instance with m3.medium instance type. Make sure to add port 8091 to inbound security rules. This allows Couchbase Web Console to be accessible afterwards.
Login to the EC2 instance using the command: ssh -i ~/.ssh/arun-cb-west1.pem ubuntu@<public-ip>
Update the Ubuntu instance: sudo apt-get update
Install Docker: curl -sSL https://get.docker.com/ | sh. More detailed instructions are available at Get Docker for Ubuntu.
Enable non-root access for the docker command: sudo usermod -aG docker ubuntu
Logout from the EC2 instance and log back in

Create AWS EBS Volume

Create an EBS volume for 10GB using EC2 console as explained in docs.
Get the instance id from the EC2 console. Attach this volume to EC2 instance using this instance id, use the default device name /dev/sdf.

Use lsblk command in EC2 instance to verify that the volume is attached to the instance:

NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda    202:0    0   8G  0 disk
└─xvda1 202:1    0   8G  0 part /
xvdb    202:16   0  30G  0 disk /mnt
xvdf    202:80   0  10G  0 disk

Portworx Container

Physical storage makeup of each node, all the provisioned volumes in the cluster as well as their container mappings is stored in an etcd cluster. Start an etcd cluster:
```
docker run -v \
  /data/varlib/etcd \
  -p 4001:4001 \
  -d \
  portworx/etcd:latest
```
By default root mounted volumes are not allowed to be shared. Enable this using the command:
```
sudo mount --make-shared /
```
This is explained more at Ubuntu Configuration and Shared Mounts.

PX-Developer (px-dev) container on a server with Docker Engine turns that server into a scale-out storage node. PX-Enterprise, on the other hand, provides multi-cluster and multi-cloud support, where storage under management can be on-premise or in a public cloud like AWS.
For this blog, we’ll start a px-dev container:

docker run --restart=always --name px -d --net=host \
  --privileged=true                             \
  -v /run/docker/plugins:/run/docker/plugins    \
  -v /var/lib/osd:/var/lib/osd:shared           \
  -v /dev:/dev                                  \
  -v /etc/pwx:/etc/pwx                          \
  -v /opt/pwx/bin:/export_bin:shared            \
  -v /var/run/docker.sock:/var/run/docker.sock  \
  -v /var/cores:/var/cores                      \
  -v /usr/src:/usr/src                           \
  --ipc=host                                    \
  portworx/px-dev -daemon -k etcd://localhost:4001 -c cluster1 -s /dev/xvdf

Complete details about this command are available at Run PX with Docker.

Look for logs using docker container logs -f px and watch out for the following statements:

time="2017-02-16T05:33:26Z" level=info msg="Initialize the scheduler client and the scheduler watch" 
time="2017-02-16T05:33:26Z" level=info msg="Started a kvdb watch on key : scheduler/containers" 
time="2017-02-16T05:33:26Z" level=info msg="Started a kvdb watch on key : scheduler/volumes" 
time="2017-02-16T05:33:26Z" level=info msg="Started a kvdb watch on key : scheduler/nodes/list"

Check the status of attached volumes that are available to Portworx using sudo /opt/pwx/bin/pxctl status to see the output:

Status: PX is operational
Node ID: 679b79b1-f4c3-413e-a8e0-c527348647c9
    IP: 172.31.25.21 
     Local Storage Pool: 1 pool
    Pool    IO_Priority    Size    Used    Status    Zone    Region
    0    LOW        10 GiB    266 MiB    Online    a    us-west-1
    Local Storage Devices: 1 device
    Device    Path        Media Type        Size        Last-Scan
    0:1    /dev/xvdf    STORAGE_MEDIUM_SSD    10 GiB        16 Feb 17 05:33 UTC
    total            -            10 GiB
Cluster Summary
    Cluster ID: cluster1
    Node IP: 172.31.25.21 - Capacity: 266 MiB/10 GiB Online (This node)
Global Storage Pool
    Total Used        :  266 MiB
    Total Capacity    :  10 GiB

It shows the total capacity available and used.

Docker Volume

Let’s create a Docker volume:
```
docker volume create -d pxd -o size=10G -o fs=ext4 --name cbvol
```
More details about this command are at Create Volumes with Docker.

Check the list of volumes available using docker volume ls command:

DRIVER              VOLUME NAME
local               70f7b9a356df4c1f0c08e13a4e813f1ef3e174a91001f277a63b62d683a27159
pxd                 cbvol
local               f7bc5fa455a88638c106881f1bce98244b670e094d5fdc47917b53a88e46c073

As shown, cbvol is created with pxd driver.

Couchbase with Portworx Volume

Create a Couchbase container using the Portworx volume:
```
docker container run \
  -d \
  --name db \
  -v cbvol:/opt/couchbase/var \
  -p 8091-8094:8091-8094 \
  -p 11210:11210 \
  arungupta/couchbase
```
Notice how /opt/couchbase/var where all Couchbase data is stored in the container is mapped to the cbvol volume on the host. This volume is mapped by Portworx.
Login to Couchbase Web Console at http://<public-ip>:8091, use the login Administrator and password as password.
Go to Data Buckets and create a new data bucket pwx:

In EC2 instance, see the list of containers:

ubuntu@ip-172-31-25-21:~$ docker container ls
CONTAINER ID        IMAGE                  COMMAND                  CREATED             STATUS              PORTS                                                                                               NAMES
8ae763d9d53b        arungupta/couchbase    "/entrypoint.sh /o..."   5 minutes ago       Up 5 minutes        0.0.0.0:8091-8094->8091-8094/tcp, 11207/tcp, 11211/tcp, 0.0.0.0:11210->11210/tcp, 18091-18093/tcp   db
5423bcd9b426        portworx/px-dev        "/docker-entry-poi..."   14 minutes ago      Up 14 minutes                                                                                                           px
cf3c779a4459        portworx/etcd:latest   "/entrypoint.sh /b..."   21 minutes ago      Up 21 minutes       2379-2380/tcp, 7001/tcp, 0.0.0.0:4001->4001/tcp                                                     youthful_jepsen

etcd, px-dev and db containers are running.

Kill the db container:
```
docker container rm -f db
```
Restart the database container as:
```
docker container run \
  -d \
  --name db \
  -v cbvol:/opt/couchbase/var \
  -p 8091-8094:8091-8094 \
  -p 11210:11210 \
  arungupta/couchbase
```
Now, because cbvol is mapped to /opt/couchbase/var again, the data is preserved across restarts. This can be verified by accessing the Couchbase Web Consoleand checking on the pwx bucket created earlier.

Another interesting perspective is also at why database are not for containers?. Just because there is Docker, does not mean all your database needs should be Dockerized. But if you need to, then there are plenty of options and can be used in production-grade applications.

Want to learn more about running Couchbase in containers?

Reference:

Stateful Containers using Portworx and Couchbase from our JCG partner Arun Gupta at the Miles to go 3.0 … blog.