When looking at any AWS Perscriptive guidance for deploying Multi Account Landing Zones, centralised logging is always a central pillar in an Well Architected envrionment.

A centralised Logging Account with all other accounts streaming logs to CloudWatch are the standard way of doing things.

While there is nothing wrong with doing this CloudWatch does have some short comings. Another architectural pattern that can be used to provide enhanced capabilities is to centralise logs into Amazon Opensearch.

There are some key benefits of using Opensearch:-

Fine-grained Data Access Control

• Amazon OpenSearch Service allows you to limit access to data down to the field level and anonymize sensitive data based on user permissions. This is useful if you want to enable troubleshooting access without exposing sensitive information.

Aggregating Logs Across Accounts and Regions

• By streaming logs from multiple AWS accounts and Regions into a centralized Amazon OpenSearch Service cluster, you can analyze trends, issues, and perform analytics across your entire infrastructure in a single location.

Flexible Querying and Analytics

• Amazon OpenSearch Service provides a powerful search and analytics engine that goes beyond the capabilities of CloudWatch Logs. You can use advanced querying, aggregation, and visualization features to gain deeper insights from your log data.

Scalability and Performance

• Amazon OpenSearch Service is a fully managed service that can automatically scale to handle large volumes of log data and provide low-latency search and analytics capabilities.

Integration with Other AWS Services

• Amazon OpenSearch Service integrates with a wide range of AWS services, such as Amazon S3, Amazon DynamoDB, and Amazon Kinesis Data Firehose, allowing you to consolidate data from multiple sources into a unified platform.

How do I build an OpenSearch Setup?

I setup a very basic setup in my AWS account, three VPC's, Two subnets per VPC, One Instance deployed into each subnet, and VPC Flow Logs created two three log groups. I then created a Amazon OpenSearch Domain.

Below is a diagram of my simple setup

Lets look at the setup of the OpenSearch Domain

Create it with the minimum required

I set mine to public access, feel free to play with VPC Access

This is important, make sure if you are doing public access you add a source IP address for your public home broadband address so you can access the OpenSearch Kibana Dashboard. Also make sure that you add the IAM ARN of the Lambda function that is being used to transport logs from CLoudWatch to Opensearch, and also make sure it has all the relevant permissions for CloudWatch and Opensearch.

Once you can got to this point you can go ahead and create your domain. (this will take a while)

Once your domain is active we need to create the subscription filters in CloudWatch.

Select your log group and then select 'Actions' > 'Subscription Filters' > 'Create Amazon OpenSearch Service subscription filter'

note yours may look slightly different and ask you to select a Lambda Role you may have to create one.

Once you have done this you need to create soe traffic. I found the easiest way to do this was to just select all my instances and Stop them and then Start them again.

I would wait 10-15 mins to allow logs to get to the OpenSeach Cluster

The easiest way to check the logs are getting to OpenSearch is to click on your domain in OpenSearch and Click the indexes tab, if you see an index with the prefix cwl... they are getting there succesfully.

To Access the dashboad click on the Kibana URL link and this will take you in.

Once in you will be presented with the below screen, you need to click 'Connect to your Elasticsearch index'

you will then be asked to create an index pattern, cwl-* is the best one to use to see all the CloudWatch data.

Once you have created the index pattern you will be able to see your data

From here you can start querying your logs using the filter options, You can build dashboards to visualise the data.

Unfortunately I have not had much chance to play with Kibana but would encourage you to have a look at the differnt features it has.

If you run into issues with getting your data into OpenSeach I would check your Lambda logs to see if there are any permissions issues with the execution role and also check the security policy on the domain settings for OpenSearch to ensure that the Lambda IAM ARN is listed to be allowed to carry out actions against the OpenSearch domain.

Hope you find this whistle stop tour of setting up Amazon OpenSearch for Centralised Logging.