Posted on

Troubleshooting Web Applications (DevOps)

Web Application Troubleshooter is a lightweight diagnostic tool for software developers, DevOps, and system administrators. It automates troubleshooting tasks for popular web servers and generates a summary HTML report shareable via email. By running a suite of automated tests, this streamlines root cause analysis and enables troubleshooting collaboration across IT teams.  

  • DNS and network connectivity problems
  • SSL/TLS certificate errors
  • Server resource usage bottlenecks
  • Application-layer HTTP/HTTPS problems
  • Firewall filtering of application ports

Whether your deploying a new application or investigating an incident, this tool quickly provides an actionable report to accelerate the troubleshooting process. When a web application goes down or performance is slow, where do you even start? Is it DNS? network latency? misconfigured server? SSL certificate expired? For software developers finding the root cause can be like searching for a needle in a haystack, especially when time is critical.

Start with some internet tests (google.com, udemy.com etc.) and then private domain web servers. SSH login includes key-based and password-based authentication to servers where an account exists. This is to check server resources (CPU, memory, disk usage) and fetch error log. Download is available with troubleshooting web applications course.

  • Software Developers/DevOps: Run diagnostics for troubleshooting application problems.
  • System Administrators: Check server health and investigate performance issues.
  • Network Engineers: Perform quick network and application-layer diagnostics.
  • Students: Learn real-world web server diagnostics and troubleshooting.

System Requirements

1. Native Linux or Windows WSL
2. sudo privileges
3. Install traceroute, nslookup, ping, curl
– sudo apt install traceroute
– sudo apt install dnsutils
– sudo apt install curl

4. Download Web Application Troubleshooter

Linux Install

1. Copy serverdiags.tar.gz from downloads directory to /home/username directory.

2. Extract the binary and README.txt to local working directory.

    tar -xzvf serverdiags.tar.gz

3. Run application from the extracted directory or create a desktop shortcut.

    ./serverdiags

Windows WSL Install

This option is recommended if you do not have native Linux desktop. ChatGPT can help troubleshoot your install based on error messages.

1. Run Windows PowerShell as administrator with right-click menu and type this command to install Windows Subsystem for Linux (WSL).

wsl --install

2. Copy serverdiags.tar.gz from downloads directory to \Ubuntu\home\username\ directory.

3. Extract the binary and README.txt to local working directory.

    tar -xzvf serverdiags.tar.gz

4. Run application from the extracted directory or create a desktop shortcut.

    ./serverdiags

Web Diagnostic Tests

Security Check

This diagnostic test confirms user has sudo privileges for running traceroute and server resources check commands. Some Linux utilities need elevated permissions to retrieve low-level system and network data.

DNS Resolution

This diagnostic test prompts the user for domain name of web server (example.com) and attempts to resolve it to an IP address. The nslookup command is used to query DNS records based on the provided domain name. The command output includes IP address of DNS server used to resolve hostname and one or more DNS A records. This could be a mix of IPv4 and/or IPv6 addresses. Select the first authoritative or non-authoritative answer (IP address) to use for ping test.

If successful, the test confirms the domain name is registered and DNS server is reachable. The test also confirms there is an A record configured on the DNS server. This does not necessarily verify that the DNS record is correct, only that one exists. For additional troubleshooting you could do reverse DNS lookup or verify registered IP address. Failure here usually means there is a DNS misconfiguration or network connectivity issues with your DNS server. The DNS query is sent to the IP address of DNS server configured on local machine (/etc/resolv.conf) for Linux. Common issues include misconfigured or missing DNS records (A, AAAA, CNAME) and DNS propagation delays after recent changes. There is also misspelled or expired domain names that will cause DNS failure.

✅ If you see authoritative or non-authoritative answers then DNS has resolved hostname.
❌ If you get a “NXDOMAIN” or timeout, there is a DNS configuration error or server is offline.

Ping Test

Verify basic connectivity between local machine and web server to confirm it is reachable. The ping command sends four ICMP echo request packets to web server and waits for echo replies. The output includes round-trip time (latency) in milliseconds and packet loss (%). If ping fails entirely, the server may be offline, blocking ICMP, or behind a firewall blocking ICMP.

✅ Packet loss = 0% and some or all ICMP packets returned = good connectivity. Network latency that is more than 100 msec could indicate some network congestion.
❌ “100% packet loss” or timeouts suggest the server is down, host unreachable, ICMP filtering, or incorrect IP address.

Traceroute

Check the network path from local machine to web server to identify routing issues or bottlenecks. Traceroute command sends ICMP packets and logs the IP address of each router (hop) along the route to server. The results reveal if there are delays, packet drops, or unreachable routers that could be offline for example. This is useful for pinpointing latency issues within an ISP or internal network routing loops.

✅ Each hop shows the assigned IP address and latency.
❌ Timeouts or abrupt ends may indicate ICMP filtering, routing issues, or device offline.

Open Port Check

Confirm that a firewall is not blocking HTTP/HTTPS application port (443, 80, 8080) and listening on web server. The diagnostic test attempts to open a TCP connection to the user selected port based on IP address from ping test. The result confirms whether there is a firewall rule filtering an application port or server misconfiguration.

✅ Connection succeeds message is returned when the server is listening on the selected application port and no firewall blocking.
❌ Connection timeouts occur when server is offline, overloaded, or firewall is filtering application port. Connection refused errors occur when application port is closed or server misconfiguration.

HTTP Server Status Check

Verify that HTTP service is running on the web server and detect response code (200 OK, 404 Not Found, 500 Internal Server Error). This diagnostic test also detects the server type whether it is Apache, NGINX, LiteSpeed, or other type. HTTP GET request is sent to the web server based on hostname from DNS check and redirects are followed. The test attempts three separate requests and returns the first successful response it receives starting with https://domain, then http://domain, and finally http://domain:8080 URL. The purpose of this test is to detect specific error codes returned by web server when troubleshooting application issues.

200 OK: ✅ Normal operation
301/302: Redirects
4xx: ❌ Client error (e.g., 404 not found, 403 forbidden, 400 bad request etc.)
5xx: ❌ Server error (e.g., 500 internal server error, 503 service unavailable, 504 bad gateway)

Status Error Name Common Root Causes / Examples
400 Bad Request Malformed URL or JSON, Missing required fields, Invalid query parameters
401 Unauthorized Missing or invalid API token, No Authorization header, User not authenticated
403 Forbidden Insufficient permissions, IP blocked, Access control restrictions
404 Not Found Incorrect URL, Resource removed, File or page doesn’t exist
405 Method Not Allowed POST on GET-only route, Unsupported method, Misconfigured route
408 Request Timeout Client took too long, Server timed out, Slow connection
409 Conflict Duplicate resource, Version conflict, Simultaneous edits
429 Too Many Requests Rate limit exceeded, API throttling, Bot activity
500 Internal Server Error Code bugs, Crash in backend, Misconfigured server
501 Not Implemented Feature not available, Method not supported, Stub endpoint
502 Bad Gateway Upstream server failure, Reverse proxy issue, DNS failure
503 Service Unavailable Overloaded server, Maintenance mode, Rate limiting
504 Gateway Timeout Backend server too slow, DB query timeout, Load balancer delay
505 HTTP Version Not Supported Old HTTP version, Client incompatibility, Legacy tool in use

SSL Certificate Check

Check SSL/TLS certificate validity period, expiry date, and number of days remaining. The diagnostic test connects over port 443 and inspects the SSL certificate presented by the server. Expired SSL/TLS certificates compromise web application security. In fact, when HSTS is enabled, an expired certificate would prevent access to an application since HTTPS is mandatory.

✅ Days remaining until expiration.
❌ Errors for expired, invalid, or unreachable certificate.

Check HTTP Headers

Sends an HTTP request to the specified domain to fetch response headers from web server. The web server responds with headers that are based on server configuration and operational state. The diagnostic test also analyzes security headers and generates a posture score. HTTP header check is used to detect web server configuration settings that are key to troubleshooting web applications.

  • Identify server software (Server, X-Powered-By)
  • Check caching behavior (Cache-Control, ETag, Expires)
  • Content settings (Content-Type, Content-Encoding)
  • Authentication or session controls (Set-Cookie, WWW-Authenticate)
  • Debugging reverse proxies (Via, X-Forwarded-For)
  • Server overload or rate limiting (Retry-After)
  • HTTPS enforcement (Strict-Transport-Security)

Application Performance Metrics

This diagnostic test measures connect time latency based on DNS lookup, TCP handshake, and SSL handshake. The results are used to detect problems with network congestion and busy servers. There is also Time to First Byte (TTFB) metric that measures amount of time from when a request is made to when server sends first byte. This is a key application metric that essentially represents server processing delay. Application response time measures how long for a web page to load and is mostly affected by network latency between client and server.

DNS lookup query is done first to resolve web server domain name (hostname) to an IP address. This is required so that client has a destination IP address for routing packets to web server. Firewall filtering of UDP 53 will prevent DNS query through firewall to DNS server. TCP handshake from client then makes a host connection to the web server socket with IP address and port 443. After that, SSL/TLS protocol handshake is initiated to web server for HTTPS data encryption. Connect time is the cumulative amount of time for all three protocols to finish successfully.

Client sends GET request for a web page based on HTTPS to the web server. At this point there is some server processing delay that could include backend server processing for dynamic content. The web server finally starts sending data to the client and when the first byte arrives this is TTFB. There is a direct correlation between server processing delay and TTFB, since an overloaded server will increase TTFB.

Public DNS servers such as Cloudflare for example are recommended for maximum speed and reliability. Network latency is mostly eliminated with CDNs that provide local internet access points globally with cached content. Implementing HTTP/2 with TLS 1.3 will provide the fastest SSL handshake with session resumption.

  • DNS Lookup (UDP 53)
  • TCP Handshake (port number)
  • SSL Handshake (HTTPS)
  • Server Processing Delay = TTFB – Connect Time
  • Time to First Byte (TTFB)

DNS lookup latency causes:
– Slow or overloaded DNS servers
– No DNS caching (e.g., first-time lookup)
– Poor internet connectivity or routing issues
– Use fast reliable DNS servers (Cloudflare etc.)

TCP handshake latency causes:
– Long physical distance to origin web server
– Network congestion
– Firewalls or proxies slowing handshake
– Packet loss or TCP retransmissions

SSL/TLS certificate handshake latency causes:
– Server-side TLS misconfiguration
– Large certificate chains
– No session resumption (TLS 1.3 required)

Server Resources Check

Login to the web server via SSH and checks CPU, memory, disk usage, and fetch error log. Many application problems are caused by overloaded servers based on CPU, memory, and disk saturation. Error log inspection is among the most recommended troubleshooting tools to resolve application problems.

This diagnostic test supports SSH key-based or password-based login authentication to a web server. There is also the option to skip this test if you do not have an SSH account on server. SSH key-based authentication prompts for username, key path (e.g., ~/.ssh/id_rsa), and optional passphrase. You would then select from NGINX, Apache, or LiteSpeed server to fetch the correct error log. The output of report displays CPU (top), memory (free -m), disk usage (df -h), and retrieves last 20 lines of error log.

Log directory based on server platform:
– Apache (Debian): /var/log/apache2/error.log
– Apache (RedHat): /var/log/httpd/error.log
– NGINX: /var/log/nginx/error.log
– LiteSpeed: /usr/local/lsws/logs/error.log

Common Web Application Errors and Diagnostic Tests

Error Type Diagnostic Test from Script
500 Internal Server Error ✅ Fetch error log
DNS Resolution Failures ✅ nslookup
Port Blocked / Connection Refused ✅ open port check
SSL/TLS Certificate Errors ✅ SSL/TLS certificate status and expiry date
404 Not Found ✅ HTTP server status, server type, and error codes
CORS Errors ✅ HTTP response headers and security score
Timeout or Latency Issues ✅ measure DNS, TCP, SSL (connect time) and TTFB latency
Network Connectivity and Routing Issues ✅ ping and traceroute
Authentication Errors ✅ HTTP server status and headers
Web Server Misconfigurations ✅ Fetch error log, HTTP server status

HTML Report Generation

Once all diagnostics are complete, the application asks if want to generate an HTML report. The report includes all test results, clearly marked with ✅ (success) or ❌ (failure). Reports are saved to your local working directory with a filename based on the web server’s domain (e.g., example.com.html). You can easily share them with colleagues or use them for documentation.

User prompt to generate HTML report format.
Each test is marked with ✅ (success) or ❌ (failure).
Saved locally (e.g., example.com.html).
Attach to email for troubleshooting collaboration, testing, and production deployment.

Web Server Diagnostics for cisconetsolutions.com

  • DNS Lookup
    ✅ 
    ✅ DNS Lookup successful:
Server: 10.255.255.254
Address: 10.255.255.254#53

Non-authoritative answer:
Name: cisconetsolutions.com
Address: 64.188.2.244
  • Ping Test
    ✅ 
    ✅ Ping test successful:
PING 64.188.2.244 (64.188.2.244) 56(84) bytes of data.
64 bytes from 64.188.2.244: icmp_seq=1 ttl=51 time=49.3 ms
64 bytes from 64.188.2.244: icmp_seq=2 ttl=51 time=48.4 ms
64 bytes from 64.188.2.244: icmp_seq=3 ttl=51 time=49.6 ms
64 bytes from 64.188.2.244: icmp_seq=4 ttl=51 time=49.1 ms

4 packets transmitted, 4 received, 0% packet loss
rtt min/avg/max/mdev = 48.444/49.112/49.609/0.423 ms
  • Traceroute
    ✅ 
    ✅ Traceroute successful:
traceroute to 64.188.2.244 (64.188.2.244), 30 hops max, 60 byte packets
 1  172.27.32.1  0.369 ms  0.352 ms  0.350 ms
 2  192.168.1.254  17.842 ms  17.841 ms  17.839 ms
 3  10.27.146.1  20.598 ms  20.596 ms  20.595 ms
 4  * * *
 5  * * *
 6  * * *
 7  62.115.139.17  152.759 ms * *
 8  62.115.116.41  48.915 ms  46.118 ms  46.085 ms
 9  62.115.112.239  46.094 ms  47.570 ms  47.547 ms
10  * * *
11  * * *
12  23.94.116.10  50.895 ms  50.892 ms  50.890 ms
13  64.188.2.244  50.882 ms  50.878 ms  50.876 ms
  • Port 443 Check (HTTPS)
    ✅ 
    ✅ Port 443 on 64.188.2.244 is open.
  • HTTP Status
    ✅ 
    ✅ HTTPS 200 OK. Service is running correctly on https://cisconetsolutions.com.
Server: LiteSpeed
  • SSL Check
    ✅ 
    ✅ SSL Certificate Information:
- Valid From: 2025-03-29 02:59:24
- Valid Until: 2025-06-27 02:59:23
- SSL Certificate valid, expires in 22 days.
  • HTTP Headers
    ❌ 
    --- All Response Headers ---
Connection: Keep-Alive
Keep-Alive: timeout=5, max=100
x-powered-by: PHP/7.4.33
expires: Thu, 19 Nov 1981 08:52:00 GMT
cache-control: no-store, no-cache, must-revalidate
pragma: no-cache
content-type: text/html; charset=UTF-8
link: https://www.cisconetsolutions.com/wp-json/
transfer-encoding: chunked
content-encoding: gzip
vary: Accept-Encoding
date: Wed, 04 Jun 2025 18:42:03 GMT
server: LiteSpeed

--- Security Header Analysis ---
❌ Missing Strict-Transport-Security
❌ Missing Content-Security-Policy
❌ Missing X-Frame-Options
❌ Missing X-Content-Type-Options
❌ Missing Referrer-Policy
❌ Missing Permissions-Policy

Security Headers Score: 0/6
  • Performance Metrics
    ✅ 
    ✅ HTTP Status Code: 200
✅ Performance Metrics (in milliseconds):
- DNS Lookup: 47.28 ms
- TCP Handshake: 107.82 ms
- SSL Handshake: 127.16 ms
- Server Processing Delay (TTFB - Connect Time): 1135.94 ms
- Time to First Byte (TTFB): 1418.20 ms
  • Server Resources
    ✅ 
    Skipped by user.
Troubleshooting Web Applications and APIs
CISCONET Training Solutions

Web Server Diagnostics for udemy.com

  • DNS Lookup
    ✅ 
    ✅ DNS Lookup successful:
Server: 10.255.255.254
Address: 10.255.255.254#53

Non-authoritative answer:
Name: udemy.com
Address: 104.16.143.237
Name: udemy.com
Address: 104.16.142.237
Name: udemy.com
Address: 2606:4700::6810:8eed
Name: udemy.com
Address: 2606:4700::6810:8fed
  • Ping Test
    ✅ 
    ✅ Ping test successful:
PING 172.27.36.173 (172.27.36.173) 56(84) bytes of data.
64 bytes from 172.27.36.173: icmp_seq=1 ttl=64 time=0.029 ms
64 bytes from 172.27.36.173: icmp_seq=2 ttl=64 time=0.036 ms
64 bytes from 172.27.36.173: icmp_seq=3 ttl=64 time=0.032 ms
64 bytes from 172.27.36.173: icmp_seq=4 ttl=64 time=0.066 ms

4 packets transmitted, 4 received, 0% packet loss
rtt min/avg/max/mdev = 0.029/0.040/0.066/0.014 ms
  • Traceroute
    ✅ 
    ✅ Traceroute successful:
traceroute to 172.27.36.173 (172.27.36.173), 30 hops max, 60 byte packets
1  172.27.36.173  0.026 ms  0.003 ms  0.003 ms
  • Port 80 Check (HTTP)
    ✅ 
    ✅ Port 80 on 172.27.36.173 is open.
  • HTTP Status
    ✅ 
    ✅ HTTPS 200 OK. Service is running correctly on https://udemy.com.
Server: cloudflare
  • SSL Check
    ✅ 
    ✅ SSL Certificate Information for udemy.com:
- Valid From: 2025-05-27 00:00:10
- Valid Until: 2025-08-25 01:00:08
- SSL Certificate valid, expires in 81 days.
  • HTTP Headers
    ❌ 
    --- All Response Headers ---
Date: Wed, 04 Jun 2025 19:24:32 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
CF-Ray: 94a9cfa02eed45a5-YVR
CF-Cache-Status: HIT
Age: 2428
Cache-Control: s-maxage=14400, stale-while-revalidate
Set-Cookie: __udmy_2_v57r=f607e15d6dfa419fabed6126e2feaedb; Domain=.udemy.com; expires=Thu, 04 Jun 2026 19:24:32 GMT; Max-Age=31536000; Path=/; Secure, ud_country_code=CA; Domain=.udemy.com; expires=Wed, 04 Jun 2025 19:25:02 GMT; Max-Age=30; Path=/; Secure
Vary: Accept-Encoding
x-cf-worker-deploy-env: production
x-cf-worker-locale: en
x-cf-worker-name: worker-udemy-com
x-cf-worker-origin-status-code: 200
x-cf-worker-version: bc8485f
X-Content-Type-Options: nosniff
x-envoy-upstream-service-time: 8
x-nextjs-cache: HIT
x-nextjs-pagekey: discovery_logged_out_home
x-nextjs-version: 3025c60
Server: cloudflare
Content-Encoding: gzip

--- Security Header Analysis ---
❌ Missing Strict-Transport-Security
❌ Missing Content-Security-Policy
❌ Missing X-Frame-Options
✅ X-Content-Type-Options: nosniff
❌ Missing Referrer-Policy
❌ Missing Permissions-Policy

Security Headers Score: 1/6
  • Performance Metrics
    ✅ 
    ✅ HTTP Status Code: 200
✅ Performance Metrics (in milliseconds):
- DNS Lookup: 34.80 ms
- TCP Handshake: 63.24 ms
- SSL Handshake: 65.54 ms
- Server Processing Delay (TTFB - Connect Time): 78.81 ms
- Time to First Byte (TTFB): 242.39 ms
  • Server Resources
    ✅ 
    ✅ Server resource check successful:
CPU Usage: 1.60%
Memory Usage: 503 MB
Disk Usage: Used: 954G, Usage: 1%

--- Last 20 lines of Error Log (/var/log/nginx/error.log) ---
2025/06/04 12:22:42 [debug] 428#428: *2 event timer del: 17: 239873
2025/06/04 12:22:42 [debug] 428#428: *2 reusable connection: 0
2025/06/04 12:22:42 [debug] 428#428: *2 free: 000055993BA06920
2025/06/04 12:22:42 [debug] 428#428: *2 free: 000055993BA20880, unused: 136
2025/06/04 12:24:31 [debug] 428#428: accept on 0.0.0.0:80, ready: 0
2025/06/04 12:24:31 [debug] 428#428: posix_memalign: 000055993BA20880:512 @16
2025/06/04 12:24:31 [debug] 428#428: *3 accept: 172.27.36.173:39500 fd:17
2025/06/04 12:24:31 [debug] 428#428: *3 event timer add: 17: 60000:343613
2025/06/04 12:24:31 [debug] 428#428: *3 reusable connection: 1
2025/06/04 12:24:31 [debug] 428#428: *3 epoll add event: fd:17 op:1 ev:80002001
2025/06/04 12:24:31 [debug] 428#428: *3 http wait request handler
2025/06/04 12:24:31 [debug] 428#428: *3 malloc: 000055993BA06920:1024
2025/06/04 12:24:31 [debug] 428#428: *3 recv: eof:1, avail:-1
2025/06/04 12:24:31 [debug] 428#428: *3 recv: fd:17 0 of 1024
2025/06/04 12:24:31 [info] 428#428: *3 client closed connection while waiting for request, client: 172.27.36.173, server: 0.0.0.0:80
2025/06/04 12:24:31 [debug] 428#428: *3 close http connection: 17
2025/06/04 12:24:31 [debug] 428#428: *3 event timer del: 17: 343613
2025/06/04 12:24:31 [debug] 428#428: *3 reusable connection: 0
2025/06/04 12:24:31 [debug] 428#428: *3 free: 000055993BA06920
2025/06/04 12:24:31 [debug] 428#428: *3 free: 000055993BA20880, unused: 136
Troubleshooting Web Applications and APIs
CISCONET Training Solutions

Custom Freeware License and Disclaimer

This software (Web Application Troubleshooter) is provided free of charge for personal and non-commercial use only, subject to the following terms:

Permissions
You may redistribute copies of this software, provided that you retain the copyright notice and this license in all copies. You may use this software on your personal Linux system for non-commercial purposes.

Restrictions
You may not sell, sublicense, or otherwise distribute this software for commercial purposes. You may not modify, reverse-engineer, or decompile this software, unless explicitly permitted by the author. Any redistribution of the software must retain this license, and the software must not be distributed with commercial intent or for profit.

No Warranty
This software is provided “as-is”, without any warranty of any kind, either express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose, or non-infringement. In no event shall the author be liable for any damages arising from the use or misuse of this software. Use at your own risk. Be sure to review the generated reports carefully, as they may contain sensitive system information.