Module

Kibana essentials

The 20% of features that cover 80% of debugging.

You don’t need to learn all of Kibana

Kibana has Discover, Dashboards, Visualize, Lens, Maps, Machine Learning, and a settings panel that could fill a textbook. At 2am you need Discover and maybe one saved dashboard.

Everything else is for building dashboards on a Tuesday afternoon, not for debugging production at night.

Discover: where you actually live

Discover is the log search view. You set a time range, write a query, and read results. That’s 90% of incident debugging in Elasticsearch-backed stacks.

The workflow:

  1. Open Discover
  2. Set time range (start narrow)
  3. Write a KQL filter
  4. Sort by @timestamp descending
  5. Expand one document, find request_id, pivot

Don’t build a visualization. Don’t open Lens. Search, read, pivot.

KQL: the 20% that covers 80%

Kibana Query Language (KQL) is simpler than Lucene. These patterns handle most incidents:

# Errors on a specific service
service:checkout and level:error

# Text search in message field
message:"connection refused"

# Exclude noise
level:error and not message:"health check"

# Time-bounded (use the picker too, but you can combine)
service:api and status_code >= 500

# Exists check — find logs missing request_id (data quality)
not request_id:*

Field:value for exact match. Quotes for phrases. and / or / not for boolean logic. That’s the whole language for emergencies.

The time picker traps

The time picker causes more false conclusions than bad queries.

Trap What goes wrong Fix
“Last 24 hours” default You see old errors mixed with new ones Start with last 15–30 minutes
Relative vs absolute “Last 1 hour” shifts as you investigate Switch to absolute once you find incident start
Refresh interval Results change while you’re reading Pause auto-refresh during triage
Timezone Deploy at “02:14 UTC” vs logs in local Check the timezone indicator top-right

When you find the incident start time, lock it: set absolute range from 10 minutes before to now.

Saved searches that actually get used

The best teams have 3–5 saved searches, not 300:

  • checkout-errors-last-hour — pre-filtered to the critical path
  • 5xx-by-service — grouped view of server errors
  • slow-requests-p99 — latency outliers with request IDs

Name them for the question they answer, not the query syntax. “What broke in checkout?” not “KQL query v3 final FINAL.”

Dashboards vs Discover

Use Discover when… Use Dashboards when…
You don’t know what’s wrong yet You know what to look at and want a snapshot
You need to read individual log lines You need trends over time
You’re correlating request IDs You’re checking if error rate is still climbing

During an active incident, start in Discover. Switch to a dashboard only after you have a hypothesis.

Index patterns: the silent footgun

Kibana reads from index patterns (e.g., logs-*). Common mistakes:

  • Wrong pattern selected — You’re searching staging-* while production burns
  • Too broad — * across all indices is slow and noisy
  • Field mapping mismatch — level in one index, log.level in another; your filter matches nothing

If your query returns zero results but you know errors exist, check the index pattern dropdown first. Seriously. Check it before rewriting the query.

The 5-minute Kibana triage

  1. Confirm index pattern matches the environment (prod, not staging)
  2. Set absolute time range around the alert time
  3. level:error on the alerted service
  4. Sort by count — what’s the top error message?
  5. Expand one hit, copy request_id, search for it
  6. Note first @timestamp — correlate with deploy markers in Grafana

What to learn later (not at 2am)

  • Lens and custom visualizations — for building dashboards, not debugging
  • Index lifecycle management — for the person who owns the cluster
  • Machine learning jobs — for anomaly detection, not live triage

Master Discover and KQL. Ignore the rest until someone asks you to build a dashboard.