TheLazy.Dev
Portfolio GitHub

Blog

logstash |elasticsearch |kibana |docker

Creating and Running Docker Containers with Logstash, Elasticsearch, and Kibana

shunmare

shun

Posted on August 3, 2023

Creating and Running Docker Containers with Logstash, Elasticsearch, and Kibana

Creating and Running Docker Containers with Logstash, Elasticsearch, and Kibana

Create docker-compose.yml 

version: '3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.9.0
    environment:
      - discovery.type=single-node
    volumes:
      - esdata:/usr/share/elasticsearch/data
    ports:
      - 9200:9200

  logstash:
    image: docker.elastic.co/logstash/logstash:8.9.0
    volumes:
      - ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml
      - ./logstash/pipeline:/usr/share/logstash/pipeline
    ports:
      - 5044:5044
    depends_on:
      - elasticsearch

  kibana:
    image: docker.elastic.co/kibana/kibana:8.9.0
    ports:
      - 5601:5601
    depends_on:
      - elasticsearch

volumes:
  esdata:
Enter fullscreen mode Exit fullscreen mode

If you want to mount log files, use the following code:

volumes:
  - ./logstash/logs:/usr/share/logstash/logs
Enter fullscreen mode Exit fullscreen mode

Creating logstash/config/logstash.yml and logstash/pipeline/logstash.conf

Create a folder named "logstash" in the same directory as the docker-compose.yml. Inside the "logstash" folder, create two subfolders named "config" and "pipeline". Save the following logstash.yml and logstash.conf files into their respective folders.
File Structure:

/my_project
  docker-compose.yml
  /logstash
    /config
      logstash.yml
    /pipeline
      logstash.conf
Enter fullscreen mode Exit fullscreen mode

logstash/config/logstash.yml:

http.host: "0.0.0.0"
path.config: /usr/share/logstash/pipeline
Enter fullscreen mode Exit fullscreen mode

logstash/pipeline/logstash.conf:

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
  }
  stdout {
    codec => rubydebug
  }
}
Enter fullscreen mode Exit fullscreen mode

Here is the basic Logstash configuration to read the content of a text file and output it to standard output (stdout):

input {
  file {
    path => "/path/to/your/file.txt"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

output {
  stdout { codec => rubydebug }
}
Enter fullscreen mode Exit fullscreen mode

Starting the Docker Container 

docker-compose up
Enter fullscreen mode Exit fullscreen mode

Stopping and Removing All Running Containers 

docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
Enter fullscreen mode Exit fullscreen mode

Displaying Docker Container Logs 

docker logs <your-container-id>
Enter fullscreen mode Exit fullscreen mode

or

docker-compose logs logstash
Enter fullscreen mode Exit fullscreen mode

Stopping Docker Containers 

docker-compose down
Enter fullscreen mode Exit fullscreen mode

Restarting Containers 

docker-compose up
Enter fullscreen mode Exit fullscreen mode

Entering a Running Docker Container 

docker exec -it <container_id> /bin/bash
Enter fullscreen mode Exit fullscreen mode

or

docker exec -it <container_id> /bin/sh
Enter fullscreen mode Exit fullscreen mode
💖 💪 🙅 🚩
shunmare
shun

Posted on August 3, 2023

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Creating and Running Docker Containers with Logstash, Elasticsearch, and Kibana
logstash Creating and Running Docker Containers with Logstash, Elasticsearch, and Kibana

August 3, 2023