Understanding and Debugging DNS in Kubernetes Clusters
Baseline
DNS Components
graph LR
Application-->NodeLocalDNS-->CoreDNS-->Upstream
Problems
- Many hidden systems
- Not easy to trace across clusters
Tools
Demo queries are located in the slides and were executed during the stream
CoreDNS Log Plugin
- Core-Plugin (just needs to be activated)
- Logs all requests to stdout
Hubble
- Cilium observability needs cilium l7 proxy, runs as deamonset
- Needs CiliumNetworkPolicies for AppPod and CoreDNS
- Metrics, UI and cli with jq (and protocol filter)
Inspector Gadget
- Toolset for Kubernetes and Linux that can be customized
- Runns as daemonset or debug pod - gadgets are distributed as containers (via artifactorhub)
- DNS-Gadget: Trace via ebpf, post process with wasm
Overview
- CoreDNS: Great for initial, nut only CoreDNS
- Hubble: Compact overview, but cilium needed with special configs
- Inspector Gadget: Rich DNS traces, limited tcp support