Nextcloud: running in Docker Compose on Debian with Let’s Encrypt SSL

I while ago I’d tested the Nextcloud, see the NextCloud: installing server on Debian behind NGINX with PHP-FPM and client on Arch Linux post.

In general, it looks good, so it’s time to try to run in a production environment and finally migrate from Dropbox to it.

Today, let’s spin up a Nextcloud instance using Docker Compose on the Debian 10 IS on a droplet running in Digital Ocean.

This droplet has an additional volume mounted to make easier to migrate data between instances and to backup it.

To run all necessary services let’s create a Docker Compose file with the next containers:

NGINX: a proxying service
Lets Encrypt: an SSL agent
MariaDB: a database server to store Nextcloud’s settings
Nextcloud: a Nextcloud container with Apache and Nextcloud’s source code

See the documentation here>>>.

For the SSL will use Lets Encrypt client from the docker-letsencrypt-nginx-proxy-companion image .

Docker and Docker Compose installation

Install Docker:

root@setevoy-do-nextcloud-production:~# curl https://get.docker.com/ | bash

And Docker Compose (check the version first – download/1.24.1/ on the releases page):

root@setevoy-do-nextcloud-production:~# curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
root@setevoy-do-nextcloud-production:~# chmod +x /usr/local/bin/docker-compose

Create a new directory to store our future Compose file:

root@setevoy-do-nextcloud-production:~# mkdir /opt/nextcloud
root@setevoy-do-nextcloud-production:~# cd /opt/nextcloud/

And let’s start with the stack creation.

Running Nexcloud

`nginx-proxy`

Create directories to keep the NGINX a Lets Encrypt files:

root@setevoy-do-nextcloud-production:/data/nextcloud# mkdir -p /data/nextcloud/nginx/{certs,vhost.d,html}

Create the Compose file /opt/nextcloud/nextcloud-compose.yml, add a network first:

networks:
  nextcloud_network:

Now, add the first container with the nginx-proxy image which will use this network, add volumes from the directories created above so the file will be the next:

version: '3'  
services:
  nginx-proxy:
    image: jwilder/nginx-proxy:alpine
    labels:
      - "com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy=true"
    container_name: nextcloud-proxy
    networks:
      - nextcloud_network
    ports:
      - 80:80
      - 443:443
    volumes:
      - /data/nextcloud/nginx/vhost.d:/etc/nginx/vhost.d:rw
      - /data/nextcloud/nginx/html:/usr/share/nginx/html:rw
      - /data/nextcloud/nginx/certs:/etc/nginx/certs:ro
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/tmp/docker.sock:ro
    restart: unless-stopped
networks:
  nextcloud_network:

The labels: com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy=true string will be used by the Let's Encrypt container to find “its own” proxy-service.

The directories here:

/etc/nginx/certs: to store SSL’s certificates and private keys (read-only for the nginx-proxy container as those files will be generated by the Lets Encrypt container).
/etc/nginx/vhost.d: virtual hosts configs
/usr/share/nginx/html: for a domain verification during issuing a new SSL certificate

Run it:

root@setevoy-do-nextcloud-production:/opt/nextcloud# docker-compose -f nextcloud-compose.yml up
Creating nextcloud-proxy ... done
Attaching to nextcloud-proxy
nextcloud-proxy | WARNING: /etc/nginx/dhparam/dhparam.pem was not found. A pre-generated dhparam.pem will be used for now while a new one
nextcloud-proxy | is being generated in the background.  Once the new dhparam.pem is in place, nginx will be reloaded.
nextcloud-proxy | forego     | starting dockergen.1 on port 5000
nextcloud-proxy | forego     | starting nginx.1 on port 5100
nextcloud-proxy | Generating DH parameters, 2048 bit long safe prime, generator 2
nextcloud-proxy | dockergen.1 | 2019/11/27 08:15:10 Generated '/etc/nginx/conf.d/default.conf' from 1 containers
nextcloud-proxy | dockergen.1 | 2019/11/27 08:15:10 Running 'nginx -s reload'
nextcloud-proxy | dockergen.1 | 2019/11/27 08:15:10 Watching docker events
nextcloud-proxy | dockergen.1 | 2019/11/27 08:15:10 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload'
nextcloud-proxy | 2019/11/27 08:15:41 [notice] 41#41: signal process started
nextcloud-proxy | This is going to take a long time
nextcloud-proxy | dhparam generation complete, reloading nginx

And try to connect:

[setevoy@setevoy-arch-work ~] $ curl -I cloud.example.org.ua
HTTP/1.1 503 Service Temporarily Unavailable
Server: nginx/1.17.5
Date: Wed, 27 Nov 2019 07:57:28 GMT
Content-Type: text/html
Content-Length: 197
Connection: keep-alive

Cool. The 503 error has no sense for us at this moment as we have no other services started yet.

Let’s Encrypt Docker

Now, add the Let’s Encrypt container to this Compose file:

...
  letsencrypt:
    image: jrcs/letsencrypt-nginx-proxy-companion
    container_name: nextcloud-letsencrypt
    depends_on:
      - nginx-proxy
    networks:
      - nextcloud_network
    volumes:
      - /data/nextcloud/nginx/vhost.d:/etc/nginx/vhost.d:rw
      - /data/nextcloud/nginx/html:/usr/share/nginx/html:rw
      - /data/nextcloud/nginx/certs:/etc/nginx/certs:rw
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
    restart: unless-stopped
...

Re-create the stack:

root@setevoy-do-nextcloud-production:/opt/nextcloud# docker-compose -f nextcloud-compose.yml up
Starting nextcloud-proxy ... done
Creating nextcloud-letsencrypt ... done
Attaching to nextcloud-proxy, nextcloud-letsencrypt
nextcloud-proxy | Custom dhparam.pem file found, generation skipped
nextcloud-proxy | forego     | starting dockergen.1 on port 5000
nextcloud-proxy | forego     | starting nginx.1 on port 5100
nextcloud-proxy | dockergen.1 | 2019/11/27 08:31:01 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload'
nextcloud-proxy | dockergen.1 | 2019/11/27 08:31:01 Watching docker events
nextcloud-proxy | dockergen.1 | 2019/11/27 08:31:01 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload'
nextcloud-proxy | dockergen.1 | 2019/11/27 08:31:02 Received event start for container 2f40fa5f50ea
nextcloud-proxy | dockergen.1 | 2019/11/27 08:31:02 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload'
nextcloud-letsencrypt | Generating a RSA private key
nextcloud-letsencrypt | ...........................................................................................................................++++
nextcloud-letsencrypt | ...........++++
nextcloud-letsencrypt | writing new private key to '/etc/nginx/certs/default.key.new'
nextcloud-letsencrypt | -----
nextcloud-letsencrypt | Info: a default key and certificate have been created at /etc/nginx/certs/default.key and /etc/nginx/certs/default.crt.
nextcloud-letsencrypt | Info: Creating Diffie-Hellman group in the background.
nextcloud-letsencrypt | A pre-generated Diffie-Hellman group will be used for now while the new one
nextcloud-letsencrypt | is being created.
nextcloud-letsencrypt | Generating DH parameters, 2048 bit long safe prime, generator 2
nextcloud-letsencrypt | Reloading nginx proxy (2e665ad175d4e2dbd270b4616bbe5d0e1c5f78421d25da55d163cc15836e859c)...
nextcloud-letsencrypt | 2019/11/27 08:31:03 Generated '/etc/nginx/conf.d/default.conf' from 2 containers
nextcloud-letsencrypt | 2019/11/27 08:31:03 [notice] 34#34: signal process started
nextcloud-letsencrypt | 2019/11/27 08:31:03 Generated '/app/letsencrypt_service_data' from 2 containers
nextcloud-letsencrypt | 2019/11/27 08:31:03 Running '/app/signal_le_service'
nextcloud-letsencrypt | 2019/11/27 08:31:04 Watching docker events
nextcloud-letsencrypt | 2019/11/27 08:31:04 Contents of /app/letsencrypt_service_data did not change. Skipping notification '/app/signal_le_service'
nextcloud-letsencrypt | Sleep for 3600s
nextcloud-letsencrypt | This is going to take a long time
nextcloud-letsencrypt | Info: Diffie-Hellman group creation complete, reloading nginx.
nextcloud-letsencrypt | Reloading nginx proxy (2e665ad175d4e2dbd270b4616bbe5d0e1c5f78421d25da55d163cc15836e859c)...
nextcloud-letsencrypt | 2019/11/27 08:31:12 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification ''
nextcloud-letsencrypt | 2019/11/27 08:31:12 [notice] 54#54: signal process started

Let’s Encrypt container generated a certificate for us – check it:

root@setevoy-do-nextcloud-production:/data/nextcloud# ll /data/nextcloud/nginx/certs/
total 12
-rw-r--r-- 1 root root 1870 Nov 27 08:31 default.crt
-rw-r--r-- 1 root root 3272 Nov 27 08:31 default.key
-rw-r--r-- 1 root root  424 Nov 27 08:31 dhparam.pem

And check availability via HTTPS/443:

[setevoy@setevoy-arch-work ~] $ curl -I https://cloud.example.org.ua
curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Okay – connection works, just a certificate’s name is not valid. We will fix it later when will add a web-app with the Nextcloud instance

For now, can skip this check by using the curl -k:

[setevoy@setevoy-arch-work ~] $ curl -kI https://cloud.example.org.ua
HTTP/2 503
server: nginx/1.17.5
date: Wed, 27 Nov 2019 08:32:58 GMT
content-type: text/html
content-length: 197

Still 503, but the main is the fact that HTTPS is working, all good so far.

MariaDB in Docker

The next thing is to add a MariaDB instance.

To make its data persistent – create a new directory on the host:


root@setevoy-do-nextcloud-production:/data/nextcloud# mkdir /data/nextcloud/mysql

Add its service to the Compose file:

...
  mysql:
    image: mariadb
    container_name: nextcloud-mysql
    networks:
      - nextcloud_network
    volumes:
      - /data/nextcloud/mysql:/var/lib/mysql
      - /etc/localtime:/etc/localtime:ro
    environment:
      - MYSQL_ROOT_PASSWORD=mysql-root-p@ssw0rd
      - MYSQL_PASSWORD=nextcloud-p@ssw0rd
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
    restart: unless-stopped
...

Run the stack:

...
nextcloud-mysql | 2019-11-27 09:16:38+00:00 [Note] [Entrypoint]: Database files initialized
nextcloud-mysql | 2019-11-27 09:16:38+00:00 [Note] [Entrypoint]: Starting temporary server
nextcloud-mysql | 2019-11-27 09:16:38+00:00 [Note] [Entrypoint]: Waiting for server startup
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] mysqld (mysqld 10.4.10-MariaDB-1:10.4.10+maria~bionic) starting as process 121 ...
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: Using Linux native AIO
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: Uses event mutexes
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: Number of pools: 1
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: Using SSE2 crc32 instructions
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: Completed initialization of buffer pool
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: Creating shared tablespace for temporary tables
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: Waiting for purge to start
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: 10.4.10 started; log sequence number 139827; transaction id 21
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] Plugin 'FEEDBACK' is disabled.
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
nextcloud-mysql | 2019-11-27  9:16:38 0 [Warning] 'user' entry 'root@de5e3e9dd106' ignored in --skip-name-resolve mode.
nextcloud-mysql | 2019-11-27  9:16:38 0 [Warning] 'user' entry '@de5e3e9dd106' ignored in --skip-name-resolve mode.
nextcloud-mysql | 2019-11-27  9:16:38 0 [Warning] 'proxies_priv' entry '@% root@de5e3e9dd106' ignored in --skip-name-resolve mode.
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] InnoDB: Buffer pool(s) load completed at 191127  9:16:38
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] Reading of all Master_info entries succeeded
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] Added new Master_info '' to hash table
nextcloud-mysql | 2019-11-27  9:16:38 0 [Note] mysqld: ready for connections.
nextcloud-mysql | Version: '10.4.10-MariaDB-1:10.4.10+maria~bionic'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
nextcloud-mysql | 2019-11-27 09:16:39+00:00 [Note] [Entrypoint]: Temporary server started.
...

Check if data is present on the host:

root@setevoy-do-nextcloud-production:~# ll /data/nextcloud/mysql/
total 122936
-rw-rw---- 1 systemd-coredump systemd-coredump    32768 Nov 27 09:17 aria_log.00000001
-rw-rw---- 1 systemd-coredump systemd-coredump       52 Nov 27 09:17 aria_log_control
-rw-rw---- 1 systemd-coredump systemd-coredump     6176 Nov 27 09:17 ib_buffer_pool
-rw-rw---- 1 systemd-coredump systemd-coredump 12582912 Nov 27 09:17 ibdata1
-rw-rw---- 1 systemd-coredump systemd-coredump 50331648 Nov 27 09:17 ib_logfile0
-rw-rw---- 1 systemd-coredump systemd-coredump 50331648 Nov 27 09:16 ib_logfile1
-rw-rw---- 1 systemd-coredump systemd-coredump 12582912 Nov 27 09:17 ibtmp1
-rw-rw---- 1 systemd-coredump systemd-coredump        0 Nov 27 09:16 multi-master.info
drwx------ 2 systemd-coredump systemd-coredump     4096 Nov 27 09:17 mysql
drwx------ 2 systemd-coredump systemd-coredump     4096 Nov 27 09:17 nextcloud
drwx------ 2 systemd-coredump systemd-coredump     4096 Nov 27 09:16 performance_schema

Okay.

Try to connect locally:

root@setevoy-do-nextcloud-production:/data/nextcloud# mysql -h localhost -u root
...
MariaDB [(none)]>

Nice, it’s working.

Nextcloud

And, finally, let’s add the Nextcloud application.

Create directories:

root@setevoy-do-nextcloud-production:~# mkdir -p /data/nextcloud/app/{config,custom\_apps,data,themes,html}

Update the Compose file:

...
  nextcloud-app:
    image: nextcloud:latest
    container_name: nextcloud-app
    networks:
      - nextcloud_network
    depends_on:
      - letsencrypt
      - nginx-proxy
      - mysql
    volumes:
      - /data/nextcloud/app/html:/var/www/html
      - /data/nextcloud/app/config:/var/www/html/config
      - /data/nextcloud/app/custom_apps:/var/www/html/custom_apps
      - /data/nextcloud/app/data:/var/www/html/data
      - /data/nextcloud/app/themes:/var/www/html/themes
      - /etc/localtime:/etc/localtime:ro
    environment:
      - VIRTUAL_HOST=cloud.example.org.ua
      - LETSENCRYPT_HOST=cloud.example.org.ua
      - LETSENCRYPT_EMAIL=root@example.org.ua
    restart: unless-stopped
...

The VIRTUAL_HOST will be used by the nginx-proxy to chose what and where to proxy, an the LETSENCRYPT_HOST will be used by the Let’s Ecnrypt container to determina a domain name to create an SSL certificate for. See the documentation here>>>.

Run the stack

root@setevoy-do-nextcloud-production:/opt/nextcloud# docker-compose -f nextcloud-compose.yml up --force-recreate
Recreating nextcloud-proxy ... done
Recreating nextcloud-mysql ... done
Recreating nextcloud-letsencrypt ... done
Creating nextcloud-app           ... done
Attaching to nextcloud-mysql, nextcloud-proxy, nextcloud-letsencrypt, nextcloud-app
...

Check the HTTPS but without -k now as have to have a valid SSL certificate now, after we specified a domain nave in the variables mentioned above:

[setevoy@setevoy-arch-work ~] $ curl -I https://cloud.example.org.ua
HTTP/2 200
server: nginx/1.17.5
...

And in a browser:

Nextcloud configuration

The next steps are the same as in the NextCloud: installing server on Debian behind NGINX with PHP-FPM and client on Arch Linux post, just with the MySQL’s host specified as the service in the Docker Compose file, in the current example it will be mysql – the Docker will perform its DNS-resolution by the service’s name to the corresponding container’s IP from the nextcloud_network:

And:

Tags and the full docker-compose file

Before finalizing this setup, let’s update our Compose file and instead of using the latest tag – set strict versions to be used to pull images – this is good practice in any production-like setup.

At the moment of writing they were :

jwilder/nginx-proxy:0.4.0
jrcs/letsencrypt-nginx-proxy-companion:v1.12
mariadb:10.4.10
nextcloud:17.0.1-apache

So, the full file will be the next:

version: '3'
services:
  nginx-proxy:
    image: jwilder/nginx-proxy:0.4.0
    labels:
      - "com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy=true"
    container_name: nextcloud-proxy
    networks:
      - nextcloud_network
    ports:
      - 80:80
      - 443:443
    volumes:
      - /data/nextcloud/nginx/vhost.d:/etc/nginx/vhost.d:rw
      - /data/nextcloud/nginx/html:/usr/share/nginx/html:rw
      - /data/nextcloud/nginx/certs:/etc/nginx/certs:ro
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/tmp/docker.sock:ro
    restart: unless-stopped
  letsencrypt:
    image: jrcs/letsencrypt-nginx-proxy-companion:v1.12
    container_name: nextcloud-letsencrypt
    depends_on:
      - nginx-proxy
    networks:
      - nextcloud_network
    volumes:
      - /data/nextcloud/nginx/vhost.d:/etc/nginx/vhost.d:rw
      - /data/nextcloud/nginx/html:/usr/share/nginx/html:rw
      - /data/nextcloud/nginx/certs:/etc/nginx/certs:rw
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
    restart: unless-stopped
  mysql:
    image: mariadb:10.4.10
    container_name: nextcloud-mysql
    networks:
      - nextcloud_network
    volumes:
      - /data/nextcloud/mysql:/var/lib/mysql
      - /etc/localtime:/etc/localtime:ro
    environment:
      - MYSQL_ROOT_PASSWORD=mysql-root-p@ssw0rd
      - MYSQL_PASSWORD=nextcloud-p@ssw0rd
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
    restart: unless-stopped
  nextcloud-app:
    image: nextcloud:17.0.1-apache
    container_name: nextcloud-app
    networks:
      - nextcloud_network
    depends_on:
      - letsencrypt
      - nginx-proxy
      - mysql
    volumes:
      - /data/nextcloud/app/html:/var/www/html
      - /data/nextcloud/app/config:/var/www/html/config
      - /data/nextcloud/app/custom_apps:/var/www/html/custom_apps
      - /data/nextcloud/app/data:/var/www/html/data
      - /data/nextcloud/app/themes:/var/www/html/themes
      - /etc/localtime:/etc/localtime:ro
    environment:
      - VIRTUAL_HOST=cloud.example.org.ua
      - LETSENCRYPT_HOST=cloud.example.org.ua
      - LETSENCRYPT_EMAIL=root@example.org.ua
    restart: unless-stopped
networks:
  nextcloud_network:

Pull images:

root@setevoy-do-nextcloud-production:/opt/nextcloud# docker-compose -f nextcloud-compose.yml pull

Re-create the stack:

root@setevoy-do-nextcloud-production:/opt/nextcloud# docker-compose -f nextcloud-compose.yml up --force-recreate

And test if everything is working.

`systemd`

The last step here will be to create a systemd unit-file for a service, as it’s described in the Linux: systemd сервис для Docker Compose post (Rus), let’s save it as /etc/systemd/system/nextcloud.service:

[Unit]
Description=Nextcloud stack
Requires=docker.service
After=docker.service

[Service]
Restart=always
WorkingDirectory=/opt/nextcloud
ExecStart=/usr/local/bin/docker-compose -f nextcloud-compose.yml up
ExecStop=/usr/local/bin/docker-compose -f nextcloud-compose.yml down  

[Install]
WantedBy=multi-user.target

Run the service:

root@setevoy-do-nextcloud-production:~# systemctl start nextcloud
root@setevoy-do-nextcloud-production:~# systemctl status nextcloud
● nextcloud.service - Nextcloud stack
Loaded: loaded (/etc/systemd/system/nextcloud.service; disabled; vendor preset: enabled)
Active: active (running) since Wed 2019-11-27 13:56:37 UTC; 4s ago
Main PID: 16599 (docker-compose)
Tasks: 7 (limit: 1167)
Memory: 74.3M
CGroup: /system.slice/nextcloud.service
├─16599 /usr/local/bin/docker-compose -f nextcloud-compose.yml up
...

Add it to autostart:

root@setevoy-do-nextcloud-production:~# systemctl enable nextcloud
Created symlink /etc/systemd/system/multi-user.target.wants/nextcloud.service → /etc/systemd/system/nextcloud.service.

Done.

Blog