Why this project?

I discovered Monica CRM a few months ago and I am a big fan of this tool. While I am making many new acquaintances these days, I always found it overwhelming knowing that I already struggle to keep up with nurturing existing relationships.

Monica CRM was the perfect answer because of its simplicity, yet it allows me to quickly log a few notes about the discussions I have with my contacts. If I remember someone told me about an insect farming project last month but I can't remember whom, then Monica will help me with that. I am not using it for personal relationships through, unless they are also business contacts. No worries if we are 'only' friends or family, I don't have files on you ;-)

I had to do a small hack to make the contact details more searchable. I will cover that later in this post.

There are different options for using Monica:

Use the online version provided by Monica as a SaaS. The free version is limited to 10 contacts - unusable beyond testing unless you've been taking social distancing a bit too far. The paid one has no limit and is offered for $9/month or $7.50/month if paid upfront for 1 year.
Install your own. The project is completely open-source and there are different ways to deploy it, including using docker.

As I am a bit obsessed with AWS, it was a natural choice for me. it was also a good timing because I decided it was time for me to learn more about Docker in practice and to find a good use case to get my hands dirty.

I looked up on GitHub, DockerHub and all over the web but couldn't find any post about running Monica on AWS. My aim was to rely on low-cost managed AWS services as much as possible. I would also like to use this example as an assignment for future AWS training.

The total monthly cost for my deployment is as follow:

EC2 t3a.macro: 1$/month (paid upfront as a reserved instance)
EBS Magnetic 30GB: 1.50$/month (can be even free if using free tier)
MySQL Aurora: less than 1$/month (I am using the Monica 1-2 times a day in average)
S3 and SES: near $0 (negligible)

TOTAL cost: 3.50$/month (or less if based on free tier)

However, my EC2 instance used for ECS only hosts the containers used for this project. The EC2 and EBS costs will be shared with other project if I manage to run other small containers on the same instance.

What are the building blocks?

The project uses 2 docker containers:

Monica CRM app (the standard monica container)
Traefik reverse-proxy as the SSL endpoint

I used traefik instead of nginx mostly out of curiosity. It is a much younger project but I heard it is gaining popularity as a great reverse proxy for micro-services and containers architectures. The SSL certificate will be taken care of by traefik via Let's Encrypt (pre-configured in the container's bootstrap).

ECS running on EC2 will be used to manage the docker containers.

MySQL is not included as a container because it will be configured as a serverless Aurora service.

In this example, we will use a VPC with a public subnet for the instance hosting the containers and a private subnet for the MySQL endpoint.

Prerequisites

A public domain is needed otherwise you will encounter SSL validation errors.

You also need a set of SMTP credentials that will be used by Monica to send email information such as reminders. Those can be created using the AWS SES services.

All the other components will be created following this tutorial.

Create an S3 bucket with an associated AWS user

This bucket will be used by Monica to store uploaded files such as avatars and profile pictures for your contacts.

Once your bucket is created, configure a specific user that will be used by Monica to access the bucket. Don't forget to assign the necessary S3 privileges and create the S3 policy.

Record the full S3 name, as well as the new user's key and secret. They will be needed in the next steps.

IAM privileges:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:GetObjectRetention",
                "s3:DeleteObjectVersion",
                "s3:GetObjectVersionTagging",
                "s3:ListBucket",
                "s3:PutObjectLegalHold",
                "s3:ReplicateObject",
                "s3:GetBucketObjectLockConfiguration",
                "s3:PutObject",
                "s3:GetObjectAcl",
                "s3:GetObject",
                "s3:PutObjectRetention",
                "s3:GetObjectVersionAcl",
                "s3:GetObjectTagging",
                "s3:GetObjectVersionForReplication",
                "s3:DeleteObject",
                "s3:GetBucketLocation",
                "s3:GetObjectVersion"
            ],
            "Resource": [
                "arn:aws:s3:::monicacrmlaurent",
                "arn:aws:s3:::monicacrmlaurent/*"
            ]
        }
    ]
}

S3 policy example:

{
    "Version": "2012-10-17",
    "Id": "Policy1598008770540",
    "Statement": [
        {
            "Sid": "Stmt1598008766523",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::759642557178:user/monicahq"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::monicacrmlaurent/*"
        }
    ]
}

Note that we can probably further narrow down the restrictions there.

Set-up your VPC environment

Create the VPC. You can use the wizard for that.

Create a security group for your future instance and another one to allow access to your future MySQL endpoint.

Initiate your Aurora database

First, you need to create a DB subnet group for your Aurora endpoint. In this example I had to create a second private subnet in an additional AZ because Aurora does not allow a DB subnet group with a single AZ.

Then create your Aurora cluster.

Make sure you select Serverless.

Choose a username and password. Remember them as they will be needed as environment variables for the Monica app container.

You can control you costs in the Capacity settings. In this example, I am forcing Aurora not to exceed 1 capacity unit (2GB RAM). It is more than enough considering I am the only user of my personal CRM!
I am also defining there that the Aurora cluster will pause after 45 minutes of activity so that I only pay for my usage and it won't cost me anything while I'm not using it.

In the Connectivity section, you can assign the DB subnet group, VPC, subnet and security group.

Prepare your ECS cluster (optional)

You can skip this part if you already have an ECS cluster ready to host your containers. If not, follow the steps below to create a new one.

First, Create a new ECS cluster.

Then choose an EC2 instance type. I am selecting the t3a.nano as it is the cheapest one available and it is more than enough for my 2 containers.

Assign the security group that was previously created to allow SSL from outside.

The ECS cluster will automatically launch en EC2 instance.

You can also assign an Elastic IP to this instance so it can be pointed to as an A entry in your public domain DNS.

Another cheap-tweak is to modify the associated EBS volume to Magnetic, which is cheaper than the default gp2 (SSD) and sufficient in terms of performance for a personal CRM. You can change it in the existing instance, as well as in the Launch Configuration auto-scaling group that was created by ECS.

Define the task definition to run the containers

Create a new Task Definition based on EC2.

In the next window, select Configure via JSON to load the JSON file.

Use the file MonicaECSTaskDefinition.json and change the marked parameters as explained below.

<<AWS REGION>> = AWS Region where the containers are running
<<APP KEY>> = choose a random string that unique for your Monica app
<<BUCKET NAME>> = Full bucket name
<<BUCKET REGION>> = Bucket region
<<AWS KEY TO ACCESS BUCKET>> = Key of the user created for the bucket
<<AWS SECRET TO ACCESS BUCKET>> = Secret of the user
<<AURORA CLUSTER NAME>> = Full name of the Aurora cluster
<<MYSQL USERNAME>> = MySQL username created earlier
<<MYSQL PASSWORD>> = MySQL password created earlier
<<FROM EMAIL ADDRESS>> = FROM address used for sending email notifications
<<FROM EMAIL NAME>> = FROM name used for sending email notifications
<<SMTP HOST>> = SMTP host obtained from SES
<<SMTP USERNAME>> = SMTP username obtained from SES
<<SMTP PASSWORD>> = SMTP password obtained from SES
<<FQDN OF MONICA CRM>> = Full public name of your Monica app
<<EMAIL ADDRESS CERTIFICATE REGISTRATION>> = Email used to request a free Let's Encrypt certificate

Task definition JSON:

{
    "ipcMode": null,
    "executionRoleArn": "<create_new>",
    "containerDefinitions": [
        {
            "dnsSearchDomains": null,
            "environmentFiles": [],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "/ecs/MonicaCRM",
                    "awslogs-region": "<<AWS REGION>>",
                    "awslogs-stream-prefix": "ecs"
                }
            },
            "entryPoint": [],
            "portMappings": [],
            "command": [],
            "linuxParameters": null,
            "cpu": 0,
            "environment": [
                {
                    "name": "APP_DISABLE_SIGNUP",
                    "value": "false"
                },
                {
                    "name": "APP_KEY",
                    "value": "<<APP KEY>>"
                },
                {
                    "name": "APP_TRUSTED_PROXIES",
                    "value": "*"
                },
                {
                    "name": "AWS_BUCKET",
                    "value": "<<BUCKET NAME>>"
                },
                {
                    "name": "AWS_REGION",
                    "value": "<<BUCKET REGION>>"
                },
                {
                    "name": "AWS_KEY",
                    "value": "<<AWS KEY TO ACCESS BUCKET>>"
                },
                {
                    "name": "AWS_SECRET",
                    "value": "<<AWS SECRET TO ACCESS BUCKET>>"
                },
                {
                    "name": "AWS_SERVER",
                    "value": ""
                },
                {
                    "name": "DAV_ENABLED",
                    "value": "true"
                },
                {
                    "name": "DB_HOST",
                    "value": "<<AURORA CLUSTER NAME>>"
                },
                {
                    "name": "DB_USERNAME",
                    "value": "<<MYSQL USERNAME>>"
                },
                {
                    "name": "DB_PASSWORD",
                    "value": "<<MYSQL PASSWORD>>"
                },
                {
                    "name": "DEFAULT_FILESYSTEM",
                    "value": "s3"
                },
                {
                    "name": "MAIL_ENCRYPTION",
                    "value": "tls"
                },
                {
                    "name": "MAIL_FROM_ADDRESS",
                    "value": "<<FROM EMAIL ADDRESS>>"
                },
                {
                    "name": "MAIL_FROM_NAME",
                    "value": "<<FROM EMAIL NAME>>"
                },
                {
                    "name": "MAIL_HOST",
                    "value": "<<SMTP HOST>>"
                },
                {
                    "name": "MAIL_MAILER",
                    "value": "smtp"
                },
                {
                    "name": "MAIL_USERNAME",
                    "value": "<<SMTP USERNAME>>"
                },
                {
                    "name": "MAIL_PASSWORD",
                    "value": "<<SMTP PASSWORD>>"
                },
                {
                    "name": "MAIL_PORT",
                    "value": "587"
                },
                {
                    "name": "MFA_ENABLED",
                    "value": "true"
                }
            ],
            "resourceRequirements": null,
            "ulimits": null,
            "dnsServers": null,
            "mountPoints": null,
            "workingDirectory": null,
            "secrets": null,
            "dockerSecurityOptions": null,
            "memoryReservation": 250,
            "volumesFrom": null,
            "stopTimeout": null,
            "image": "monicahq/monicahq",
            "startTimeout": null,
            "firelensConfiguration": null,
            "dependsOn": null,
            "disableNetworking": null,
            "interactive": null,
            "healthCheck": null,
            "essential": true,
            "links": [],
            "hostname": null,
            "extraHosts": null,
            "pseudoTerminal": null,
            "user": null,
            "readonlyRootFilesystem": null,
            "dockerLabels": {
                "traefik.enable": "true",
                "traefik.http.routers.app.entrypoints": "app",
                "traefik.http.routers.app.rule": "Host(`<<FQDN OF MONICA CRM>>`)",
                "traefik.http.routers.app.tls.certresolver": "mytls"
            },
            "systemControls": null,
            "privileged": null,
            "name": "app",
            "repositoryCredentials": {
                "credentialsParameter": ""
            }
        },
        {
            "dnsSearchDomains": null,
            "environmentFiles": [],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "/ecs/MonicaCRM",
                    "awslogs-region": "<<AWS REGION>>",
                    "awslogs-stream-prefix": "ecs"
                }
            },
            "entryPoint": null,
            "portMappings": [
                {
                    "hostPort": 443,
                    "protocol": "tcp",
                    "containerPort": 443
                },
                {
                    "hostPort": 8080,
                    "protocol": "tcp",
                    "containerPort": 8080
                }
            ],
            "command": [],
            "linuxParameters": null,
            "cpu": 0,
            "environment": [
                {
                    "name": "TRAEFIK_API_INSECURE",
                    "value": "true"
                },
                {
                    "name": "TRAEFIK_CERTIFICATESRESOLVERS_MYTLS_ACME_EMAIL",
                    "value": "<<EMAIL ADDRESS CERTIFICATE REGISTRATION>>"
                },
                {
                    "name": "TRAEFIK_CERTIFICATESRESOLVERS_MYTLS_ACME_TLSCHALLENGE",
                    "value": "true"
                },
                {
                    "name": "TRAEFIK_CERTIFICATESRSOLVERS_MYTLS_ACME_STORAGE",
                    "value": "/letsencrypt/acme.json"
                },
                {
                    "name": "TRAEFIK_ENTRYPOINTS_APP_ADDRESS",
                    "value": ":443"
                },
                {
                    "name": "TRAEFIK_PROVIDERS_DOCKER",
                    "value": "true"
                },
                {
                    "name": "TRAEFIK_PROVIDERS_DOCKER_EXPOSEDBYDEFAULT",
                    "value": "false"
                }
            ],
            "resourceRequirements": null,
            "ulimits": null,
            "dnsServers": null,
            "mountPoints": [
                {
                    "readOnly": true,
                    "containerPath": "/var/run/docker.sock",
                    "sourceVolume": "dockersock"
                },
                {
                    "readOnly": null,
                    "containerPath": "/letsencrypt",
                    "sourceVolume": "tmp"
                }
            ],
            "workingDirectory": null,
            "secrets": null,
            "dockerSecurityOptions": null,
            "memoryReservation": 200,
            "volumesFrom": null,
            "stopTimeout": null,
            "image": "traefik:v2.3.0-rc2",
            "startTimeout": null,
            "firelensConfiguration": null,
            "dependsOn": [
                {
                    "containerName": "app",
                    "condition": "START"
                }
            ],
            "disableNetworking": null,
            "interactive": null,
            "healthCheck": null,
            "essential": true,
            "links": null,
            "hostname": null,
            "extraHosts": null,
            "pseudoTerminal": null,
            "user": null,
            "readonlyRootFilesystem": null,
            "dockerLabels": null,
            "systemControls": null,
            "privileged": null,
            "name": "traefik",
            "repositoryCredentials": {
                "credentialsParameter": ""
            }
        }
    ],
    "memory": null,
    "taskRoleArn": "",
    "family": "MonicaCRM",
    "pidMode": null,
    "requiresCompatibilities": [
        "EC2"
    ],
    "networkMode": null,
    "cpu": null,
    "inferenceAccelerators": null,
    "proxyConfiguration": null,
    "volumes": [
        {
            "efsVolumeConfiguration": null,
            "name": "dockersock",
            "host": {
                "sourcePath": "/var/run/docker.sock"
            },
            "dockerVolumeConfiguration": null
        },
        {
            "efsVolumeConfiguration": null,
            "name": "tmp",
            "host": {
                "sourcePath": "/tmp/"
            },
            "dockerVolumeConfiguration": null
        }
    ],
    "placementConstraints": [],
    "tags": []
}

Once the task definition is created, run the task to start the containers.

Check that the task has the RUNNING status.

Configure your public domain

At this point you should add an A record in your public DNS to point to the IP address of the EC2 instance on which Monica is running. In my example, I have an A record for the following entry:
crm.mydomain.com

Done!

Now you should be able to access Monica CRM via HTTPS using its DNS record crm.mydomain.com.

One small trick to make your contact more searchable

I find it very limiting that we cannot search all the contact details in Monica's search bar. For example, you are not able to search by company name. There are a few open issues on the project's GitHub page for that, however they have not been resolved so far.
https://github.com/monicahq/monica/issues/2069
https://github.com/monicahq/monica/issues/1017

As a workaround, you can tweak the PHP code in the Contact.php file by adding additional searchable parameter. I usually don't like modifying directly within the app's code but there is no other way than hardcoding it.

The only useful parameters you can add are the contact description, first met information and company name. Unfortunately we cannot search for conversation or activity using this workaround because they are stored in a different table. If someone find a workaround for this, please share.

Type the following 3 commands inside the container to implement the workaround. You can use the docker exec command syntax from the EC2 instance used by ECS to execute the commands from within a container.

sed -i '/searchable_columns / a '\'description''\'\,'' /var/www/monica/app/Models/Contact/Contact.php
sed -i '/searchable_columns / a '\'first_met_additional_info''\'\,'' /var/www/monica/app/Models/Contact/Contact.php
sed -i '/searchable_columns / a '\'company''\'\,'' /var/www/monica/app/Models/Contact/Contact.php

You may need to redo this procedure every time you start a new monica app container.

Ideas for next steps...

Here are a few things I might be working on in the future. Hopefully I will find more spare time since my favourite personal CRM is already helping me with being more efficient :-)

CloudFormation stack

I would like to develop a stack to automate the deployment of all these steps. The idea is to include the S3 bucket, IAM users and policies, Aurora serverless MySQL, VPC, EC2 instance, ECS cluster and tasks definitions. The only part that will not be created by the stack is the DNS alias because it is specific to the domain name and provider.

Go entirely serverless

Although PHP does not appear as a natively supported Lambda runcode environment, there are some tutorials out there to build your own. I would like to move entirely from the Monica docker to a serverless architecture based on AWS Lambda and API gateway using the SAM framework.