TheLazy.Dev
Portfolio GitHub

Blog

Essential Debugging Techniques for Network and Service Connectivity

saniyathossain

Saniyat Hossain

Posted on June 20, 2024

Essential Debugging Techniques for Network and Service Connectivity

In the fast-paced world of software development, effective debugging is crucial for ensuring smooth operations and quick resolutions to issues. This article provides practical examples and tips for using essential tools like curl, telnet, and tcpdump, along with connectivity checks for services such as Redis, MySQL, RabbitMQ, Minio, and more. We'll also cover additional tricks for extensive debugging and discuss tools like NGINX and HAProxy.

Tools and Techniques Overview

Curl: The Versatile HTTP Client

curl is a powerful command-line tool for transferring data with URLs. It supports various protocols and provides extensive options for debugging.

Basic Usage:

curl -I http://example.com
Enter fullscreen mode Exit fullscreen mode

Verbose Mode with SSL Verification and Proxy:

curl -kvvv --proxy http://proxy.example.com:8080 https://example.com
Enter fullscreen mode Exit fullscreen mode

Example Response:

* Rebuilt URL to: https://example.com/
*   Trying 93.184.216.34...
* TCP_NODELAY set
* Connected to example.com (93.184.216.34) port 443 (#0)
> GET / HTTP/1.1
> Host: example.com
> User-Agent: curl/7.68.0
> Accept: */*
< HTTP/1.1 200 OK
...
* Connection #0 to host example.com left intact
Enter fullscreen mode Exit fullscreen mode

Tips and Tricks:

  1. Check Headers and Content: 
   curl -i http://example.com
Enter fullscreen mode Exit fullscreen mode
  1. Follow Redirects: 
   curl -L http://example.com
Enter fullscreen mode Exit fullscreen mode
  1. Download File: 
   curl -O http://example.com/file.zip
Enter fullscreen mode Exit fullscreen mode
  1. Send Data with POST: 
   curl -d "param1=value1&param2=value2" -X POST http://example.com/submit
Enter fullscreen mode Exit fullscreen mode
  1. Use with Authentication: 
   curl -u username:password http://example.com
Enter fullscreen mode Exit fullscreen mode
  1. Custom Headers: 
   curl -H "Custom-Header: value" http://example.com
Enter fullscreen mode Exit fullscreen mode
  1. Debugging DNS: 
   curl --dns-servers 8.8.8.8 http://example.com
Enter fullscreen mode Exit fullscreen mode
  1. Output to File: 
   curl http://example.com -o output.txt
Enter fullscreen mode Exit fullscreen mode
  1. Testing APIs: 
   curl -X POST -H "Content-Type: application/json" -d '{"key1":"value1", "key2":"value2"}' http://api.example.com/endpoint
Enter fullscreen mode Exit fullscreen mode

Telnet: Simple Connectivity Testing

telnet is useful for testing connectivity to remote hosts and services.

Basic Usage:

telnet example.com 80
Enter fullscreen mode Exit fullscreen mode

Example Response:

Trying 93.184.216.34...
Connected to example.com.
Escape character is '^]'.
Enter fullscreen mode Exit fullscreen mode

Tips and Tricks:

  1. Check SMTP Server: 
   telnet smtp.example.com 25
Enter fullscreen mode Exit fullscreen mode
  1. Test HTTP Service: 
   telnet example.com 80
   GET / HTTP/1.1
   Host: example.com
Enter fullscreen mode Exit fullscreen mode
  1. Escape Character: 
   telnet example.com 80
   (Ctrl + ])
   quit
Enter fullscreen mode Exit fullscreen mode

Tcpdump: Network Packet Analyzer

tcpdump captures network packets for detailed analysis.

Basic Usage:

sudo tcpdump -i eth0
Enter fullscreen mode Exit fullscreen mode

Capture Specific Traffic:

sudo tcpdump -i eth0 'tcp port 80'
Enter fullscreen mode Exit fullscreen mode

Tips and Tricks:

  1. Save to File: 
   sudo tcpdump -i eth0 -w capture.pcap
Enter fullscreen mode Exit fullscreen mode
  1. Read from File: 
   sudo tcpdump -r capture.pcap
Enter fullscreen mode Exit fullscreen mode
  1. Filter by Host: 
   sudo tcpdump -i eth0 host example.com
Enter fullscreen mode Exit fullscreen mode
  1. Filter by Port: 
   sudo tcpdump -i eth0 port 443
Enter fullscreen mode Exit fullscreen mode
  1. Analyze HTTP Traffic: 
   sudo tcpdump -i eth0 -A -s 0 'tcp port 80'
Enter fullscreen mode Exit fullscreen mode
  1. Limit Packet Capture: 
   sudo tcpdump -c 100 -i eth0
Enter fullscreen mode Exit fullscreen mode

Ping: Network Latency Checker

Basic Usage:

ping example.com
Enter fullscreen mode Exit fullscreen mode

Example Response:

PING example.com (93.184.216.34) 56(84) bytes of data.
64 bytes from example.com (93.184.216.34): icmp_seq=1 ttl=54 time=6.67 ms
64 bytes from example.com (93.184.216.34): icmp_seq=2 ttl=54 time=6.73 ms
...
Enter fullscreen mode Exit fullscreen mode

SFTP: Secure File Transfer Protocol

Basic Usage:

sftp user@example.com
Enter fullscreen mode Exit fullscreen mode

Example Response:

Connected to example.com.
sftp>
Enter fullscreen mode Exit fullscreen mode

Wireshark: Network Protocol Analyzer

Wireshark is a GUI tool for capturing and analyzing network traffic.

Basic Usage:

  1. Open Wireshark.
  2. Select the network interface to capture traffic.
  3. Use filters (e.g., http, tcp.port == 80) to refine the capture.

Elasticsearch and Kibana: Search and Analytics Engine

Elasticsearch Basic Usage:

curl -X GET "localhost:9200"
Enter fullscreen mode Exit fullscreen mode

Example Response:

{
  "name" : "node-1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "XYZ123",
  ...
}
Enter fullscreen mode Exit fullscreen mode

Kibana Basic Usage:

Open a web browser and navigate to http://localhost:5601.

Jenkins: Continuous Integration and Delivery

Basic Usage:

Navigate to Jenkins web interface at http://localhost:8080.

Service Connectivity and Authentication Checks

Docker Container Setup

Dockerfile Example 

# Use a slim base image
FROM debian:slim-buster

# Install necessary packages
RUN apt-get update && apt-get install -y \
    curl \
    telnet \
    tcpdump \
    mysql-client \
    redis-tools \
    rabbitmq-server \
    mc \
    td-agent \
    postgresql-client \
    --no-install-recommends && \
    rm -rf /var/lib/apt/lists/*

CMD ["bash"]
Enter fullscreen mode Exit fullscreen mode

Connectivity Checks

Redis

Installation:

Inside Docker Container:

RUN apt-get update && apt-get install -y redis-tools
Enter fullscreen mode Exit fullscreen mode

Host Machines (Ubuntu, Windows, CentOS/RedHat, Kali Linux):

sudo apt-get update && sudo apt-get install -y redis-tools
Enter fullscreen mode Exit fullscreen mode

Check Connection:

redis-cli -h redis.example.com -p 6379 ping
Enter fullscreen mode Exit fullscreen mode

Possible Response:

PONG
Enter fullscreen mode Exit fullscreen mode

Tips and Tricks:

  1. Check Connection: 
   redis-cli -h redis.example.com -p 6379 ping
Enter fullscreen mode Exit fullscreen mode
  1. Test with Password: 
   redis-cli -h redis.example.com -p 6379 -a password ping
Enter fullscreen mode Exit fullscreen mode

MySQL

Installation:

Inside Docker Container:

RUN apt-get update && apt-get install -y mysql-client
Enter fullscreen mode Exit fullscreen mode

Host Machines (Ubuntu, Windows, CentOS/RedHat, Kali Linux):

sudo apt-get update && sudo apt-get install -y mysql-client
Enter fullscreen mode Exit fullscreen mode

Check Connection:

mysql -h mysql.example.com -P 3306 -u username -p
Enter fullscreen mode Exit fullscreen mode

Possible Response:

Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 5.7.33 MySQL Community Server (GPL)
...
Enter fullscreen mode Exit fullscreen mode

Tips and Tricks:

  1. Check Connection: 
   mysql -h mysql.example.com -P 3306 -u username -p
Enter fullscreen mode Exit fullscreen mode
  1. Test with Database: 
   mysql -h mysql.example.com -P 3306 -u username -p database_name
Enter fullscreen mode Exit fullscreen mode

RabbitMQ

Installation:

Inside Docker Container:

RUN apt-get update && apt-get install -y rabbitmq-server
Enter fullscreen mode Exit fullscreen mode

Host Machines (Ubuntu, Windows, CentOS/RedHat, Kali Linux):

sudo apt-get update && sudo apt-get install -y rabbitmq-server
Enter fullscreen mode Exit fullscreen mode

Check Connection:

curl -kvvv -u guest:guest http://rabbitmq.example.com:15672/api/overview
Enter fullscreen mode Exit fullscreen mode

Possible Response:

*   Trying rabbitmq.example.com...
* TCP_NODELAY set
* Connected to rabbitmq.example.com (93.184.216.34) port 15672 (#0)
> GET /api/overview HTTP/1.1
> Host: rabbitmq.example.com
> Authorization: Basic Z3Vlc3Q6Z3Vlc3Q=
> User-Agent: curl/

7.68.0
> Accept: */*
< HTTP/1.1 200 OK
...
Enter fullscreen mode Exit fullscreen mode

Tips and Tricks:

  1. Check Connection: 
   curl -kvvv -u guest:guest http://rabbitmq.example.com:15672/api/overview
Enter fullscreen mode Exit fullscreen mode
  1. Check Queues: 
   curl -kvvv -u guest:guest http://rabbitmq.example.com:15672/api/queues
Enter fullscreen mode Exit fullscreen mode

Minio

Installation:

Inside Docker Container:

RUN apt-get update && apt-get install -y mc
Enter fullscreen mode Exit fullscreen mode

Host Machines (Ubuntu, Windows, CentOS/RedHat, Kali Linux):

sudo apt-get update && sudo apt-get install -y mc
Enter fullscreen mode Exit fullscreen mode

Check Connection:

curl -kvvv http://minio.example.com:9000
Enter fullscreen mode Exit fullscreen mode

Possible Response:

*   Trying minio.example.com...
* TCP_NODELAY set
* Connected to minio.example.com (93.184.216.34) port 9000 (#0)
> GET / HTTP/1.1
> Host: minio.example.com
> User-Agent: curl/7.68.0
> Accept: */*
< HTTP/1.1 200 OK
...
Enter fullscreen mode Exit fullscreen mode

Tips and Tricks:

  1. Check Connection: 
   curl -kvvv http://minio.example.com:9000
Enter fullscreen mode Exit fullscreen mode
  1. List Buckets: 
   curl -kvvv http://minio.example.com:9000/minio/health/ready
Enter fullscreen mode Exit fullscreen mode

Fluentd/TD-Agent: Log Management

Installation:

Inside Docker Container:

RUN apt-get update && apt-get install -y td-agent
Enter fullscreen mode Exit fullscreen mode

Host Machines (Ubuntu, Windows, CentOS/RedHat, Kali Linux):

curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-bionic-td-agent3.sh | sh
Enter fullscreen mode Exit fullscreen mode

Check Connection:

curl -kvvv http://fluentd.example.com:24224
Enter fullscreen mode Exit fullscreen mode

Possible Response:

*   Trying fluentd.example.com...
* TCP_NODELAY set
* Connected to fluentd.example.com (93.184.216.34) port 24224 (#0)
> GET / HTTP/1.1
> Host: fluentd.example.com
> User-Agent: curl/7.68.0
> Accept: */*
< HTTP/1.1 200 OK
...
Enter fullscreen mode Exit fullscreen mode

Tips and Tricks:

  1. Check Connection: 
   curl -kvvv http://fluentd.example.com:24224
Enter fullscreen mode Exit fullscreen mode
  1. Check Logs: 
   tail -f /var/log/td-agent/td-agent.log
Enter fullscreen mode Exit fullscreen mode

Kafka: Distributed Streaming Platform

Installation:

Inside Docker Container:

RUN apt-get update && apt-get install -y kafka
Enter fullscreen mode Exit fullscreen mode

Host Machines (Ubuntu, Windows, CentOS/RedHat, Kali Linux):

sudo apt-get update && sudo apt-get install -y kafka
Enter fullscreen mode Exit fullscreen mode

Check Connection:

kafka-console-consumer.sh --bootstrap-server kafka.example.com:9092 --topic test --from-beginning
Enter fullscreen mode Exit fullscreen mode

Possible Response:

...
Message 1
Message 2
...
Enter fullscreen mode Exit fullscreen mode

Tips and Tricks:

  1. Check Connection: 
   kafka-console-consumer.sh --bootstrap-server kafka.example.com:9092 --topic test --from-beginning
Enter fullscreen mode Exit fullscreen mode
  1. Check Topics: 
   kafka-topics.sh --list --bootstrap-server kafka.example.com:9092
Enter fullscreen mode Exit fullscreen mode

PostgreSQL

Installation:

Inside Docker Container:

RUN apt-get update && apt-get install -y postgresql-client
Enter fullscreen mode Exit fullscreen mode

Host Machines (Ubuntu, Windows, CentOS/RedHat, Kali Linux):

sudo apt-get update && sudo apt-get install -y postgresql-client
Enter fullscreen mode Exit fullscreen mode

Check Connection:

psql -h postgres.example.com -p 5432 -U username -d database_name
Enter fullscreen mode Exit fullscreen mode

Possible Response:

Password for user username:
psql (13.3 (Ubuntu 13.3-1.pgdg20.04+1))
Type "help" for help.

database_name=>
Enter fullscreen mode Exit fullscreen mode

Tips and Tricks:

  1. Check Connection: 
   psql -h postgres.example.com -p 5432 -U username -d database_name
Enter fullscreen mode Exit fullscreen mode
  1. List Databases: 
   psql -h postgres.example.com -p 5432 -U username -c "\l"
Enter fullscreen mode Exit fullscreen mode

NGINX: Web Server Debugging

Configuration Testing:

nginx -t
Enter fullscreen mode Exit fullscreen mode

Reload Configuration:

sudo systemctl reload nginx
Enter fullscreen mode Exit fullscreen mode

Log Files:

Access Log:

tail -f /var/log/nginx/access.log
Enter fullscreen mode Exit fullscreen mode

Error Log:

tail -f /var/log/nginx/error.log
Enter fullscreen mode Exit fullscreen mode

Debugging Tips:

  1. Check Server Status: 
   curl -I http://localhost
Enter fullscreen mode Exit fullscreen mode
  1. Verify Configuration Syntax: 
   nginx -t
Enter fullscreen mode Exit fullscreen mode
  1. Test Specific Configuration File: 
   nginx -c /etc/nginx/nginx.conf
Enter fullscreen mode Exit fullscreen mode
  1. Check Connection Limit: 
   curl -I --limit-rate 1k http://localhost
Enter fullscreen mode Exit fullscreen mode
  1. Analyze Request Headers: 
   curl -I http://localhost -H "Host: example.com"
Enter fullscreen mode Exit fullscreen mode

HAProxy: Load Balancer Debugging

Configuration Testing:

haproxy -c -f /etc/haproxy/haproxy.cfg
Enter fullscreen mode Exit fullscreen mode

Reload Configuration:

sudo systemctl reload haproxy
Enter fullscreen mode Exit fullscreen mode

Log Files:

Access Log:

tail -f /var/log/haproxy.log
Enter fullscreen mode Exit fullscreen mode

Debugging Tips:

  1. Check Frontend Status: 
   curl -I http://haproxy.example.com
Enter fullscreen mode Exit fullscreen mode
  1. Verify Configuration Syntax: 
   haproxy -c -f /etc/haproxy/haproxy.cfg
Enter fullscreen mode Exit fullscreen mode
  1. View Statistics: 
   curl http://haproxy.example.com/stats
Enter fullscreen mode Exit fullscreen mode
  1. Check Backend Health: 
   curl http://haproxy.example.com:8080/health
Enter fullscreen mode Exit fullscreen mode
  1. Analyze Headers: 
   curl -I http://haproxy.example.com -H "X-Forwarded-For: 1.2.3.4"
Enter fullscreen mode Exit fullscreen mode

Wireshark: Network Protocol Analyzer

Wireshark is a GUI tool for capturing and analyzing network traffic.

Basic Usage:

  1. Open Wireshark.
  2. Select the network interface to capture traffic.
  3. Use filters (e.g., http, tcp.port == 80) to refine the capture.

Elasticsearch and Kibana: Search and Analytics Engine

Elasticsearch Basic Usage:

curl -X GET "localhost:9200"
Enter fullscreen mode Exit fullscreen mode

Example Response:

{
  "name" : "node-1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "XYZ123",
  ...
}
Enter fullscreen mode Exit fullscreen mode

Kibana Basic Usage:

Open a web browser and navigate to http://localhost:5601.

Jenkins: Continuous Integration and Delivery

Basic Usage:

Navigate to Jenkins web interface at http://localhost:8080.

Summary Table

Here's the summarized table of tools and connectivity checks with proper formatting:

Tool / Service Usage Example Command / Response Tips and Tricks
Curl HTTP Client curl -I http://example.com Check headers, follow redirects, send data with POST, use with authentication, custom headers, debugging DNS
Telnet Simple Connectivity Testing telnet example.com 80 Check SMTP server, test HTTP service, escape character
Tcpdump Network Packet Analyzer sudo tcpdump -i eth0 Save to file, read from file, filter by host and port, analyze HTTP traffic
Ping Network Latency Checker ping example.com Basic usage, measure latency
SFTP Secure File Transfer Protocol sftp user@example.com Basic usage, file transfers
Wireshark Network Protocol Analyzer GUI-based tool Capture network traffic, use filters, analyze protocols
Elasticsearch Search and Analytics Engine curl -X GET "localhost:9200" Check cluster status, use Kibana for visual analytics
Jenkins Continuous Integration and Delivery Navigate to Jenkins web interface at http://localhost:8080 Automate builds and deployments, integrate with various tools
Redis Key-Value Store redis-cli -h redis.example.com -p 6379 ping Check connection, test with password
MySQL Relational Database mysql -h mysql.example.com -P 3306 -u username -p Check connection, test with database
RabbitMQ Message Broker curl -kvvv -u guest:guest http://rabbitmq.example.com:15672/api/overview Check connection, check queues
Minio Object Storage curl -kvvv http://minio.example.com:9000 Check connection, list buckets
Fluentd / TD-Agent Log Management curl -kvvv http://fluentd.example.com:24224 Check connection, check logs
Kafka Distributed Streaming Platform kafka-console-consumer.sh --bootstrap-server kafka.example.com:9092 --topic test --from-beginning Check connection, check topics
PostgreSQL Relational Database psql -h postgres.example.com -p 5432 -U username -d database_name Check connection, list databases
NGINX Web Server Debugging nginx -t Verify configuration, reload configuration, check logs
HAProxy Load Balancer Debugging haproxy -c -f /etc/haproxy/haproxy.cfg Verify configuration, reload configuration, view statistics

Conclusion

By leveraging tools like curl, telnet, tcpdump, and others, and understanding how to check connectivity for services such as Redis, MySQL, RabbitMQ, and Minio, you can effectively diagnose and resolve network and service-related issues. Additionally, using NGINX and HAProxy debugging tips can help you maintain a smooth and efficient development workflow.

Feel free to share your thoughts and experiences in the comments below!

💖 💪 🙅 🚩
saniyathossain
Saniyat Hossain

Posted on June 20, 2024

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

I am purposefully super nice to AI because I imagine they are trained on data where nice people get better help. Doing this, though, has an existential uneasiness to it.
ai I am purposefully super nice to AI because I imagine they are trained on data where nice people get better help. Doing this, though, has an existential uneasiness to it.

November 28, 2024

The best way to get better at writing code is...
development The best way to get better at writing code is...

November 28, 2024

If you're a beginner, definitely check this open source guide. I've explained almost everything you need to know.
undefined If you're a beginner, definitely check this open source guide. I've explained almost everything you need to know.

November 28, 2024

HMPL has a new logo. What do you think?
webdev HMPL has a new logo. What do you think?

November 28, 2024

15 Best Chrome Extensions for DEVs🧑‍💻
webdev 15 Best Chrome Extensions for DEVs🧑‍💻

November 28, 2024