Prometheus Fundamentals: Scraping & PromQL
The data model, scrape config, and the query language that ties it together.
Now that you know why we monitor, letβs get concrete about how Prometheus stores data and how you ask it questions with PromQL, its query language.
π― Learning Objectives
- Understand the Prometheus data model (metric name + labels + value + timestamp)
- Configure a scrape job in
prometheus.yml - Know the four metric types
- Write basic PromQL queries, including
rate()
The Data Model
Every sample Prometheus stores is a time series identified by a metric name plus a set of labels:
http_requests_total{method="GET", path="/api", status="200"}
ββββββββ¬βββββββ ββββββββββββββββββββ¬ββββββββββββββββββββ
metric name labelsEach unique combination of name + labels is its own series, sampled over time. Labels are what make metrics powerful β you can later filter and aggregate across any dimension.
β Watch your label cardinality
Never put unbounded values (like a user ID, email, or request ID) in a label. Each unique value creates a new time series, and millions of series will overwhelm Prometheus. Keep labels to small, finite sets like status, method, or region.
The Four Metric Types
| Type | Meaning | Example |
|---|---|---|
| Counter | Only goes up (reset on restart) | Total requests served |
| Gauge | Goes up and down | Current memory in use, temperature |
| Histogram | Buckets of observations | Request duration distribution |
| Summary | Similar to histogram, client-side quantiles | Request latency percentiles |
The rule of thumb: use a counter for things you accumulate, a gauge for things that fluctuate, and a histogram for distributions like latency.
Configuring a Scrape
Prometheus reads a YAML config that lists what to scrape and how often:
global:
scrape_interval: 15s # how often to scrape every target
scrape_configs:
- job_name: "prometheus" # Prometheus scraping itself
static_configs:
- targets: ["localhost:9090"]
- job_name: "my-app"
static_configs:
- targets: ["app:8080"] # scrapes http://app:8080/metricsStart Prometheus pointing at that file and it begins scraping immediately:
π‘ Exporters do the work for you
Canβt add metrics to something you donβt control (like a Linux host or PostgreSQL)? Run an exporter β a small sidecar that translates system stats into a /metrics endpoint. node_exporter (hosts) and blackbox_exporter (probing URLs) are the most common.
Querying with PromQL
Open the Prometheus UI at port 9090 and type queries. The simplest query is just a metric name β but the real power is filtering and functions.
Filter by label:
http_requests_total{status="500"}Rate of increase β the single most useful function. Counters always climb, so you rarely graph them raw; instead you graph how fast they climb with rate():
rate(http_requests_total{status="500"}[5m])This reads: the per-second rate of 500 errors, averaged over the last 5 minutes.
Aggregate across series with sum, avg, by:
sum(rate(http_requests_total[5m])) by (status)That gives you total request rate broken down by status code β exactly what youβd graph on a dashboard.
π§ͺ Hands-on Lab
Write the Queries
Write PromQL for each:
- Current value of the gauge
node_memory_active_bytes - Per-second rate of all HTTP requests over the last 5 minutes
- Total request rate grouped by
path
π§ Knowledge Check
Which metric type should you use for a value that only ever increases, like total requests served?
Why do we usually wrap counters in rate() before graphing them?
πΌ Interview Preparation
What is label cardinality and why does it matter?
Summary
You can now read the Prometheus data model, configure a scrape job, pick the right metric type, and write PromQL β including the all-important rate(). Next, we take these queries and turn them into beautiful, shareable dashboards in Grafana.