How to read this book, what Go brings to web services, and what we will build — an HTTP server with routing, middleware, PostgreSQL, authentication, and server-side rendered dashboards.
Shipping Go Web Services
23 lessons/ source code available
Build a production log aggregator from scratch. HTTP services, PostgreSQL, authentication, middleware, HTML dashboards, testing, and deployment — no frameworks, just Go.
what you'll learn
- Build an HTTP server from scratch — routing, JSON APIs, request validation, and structured logging with slog
- PostgreSQL — connection pools, parameterized queries, batch inserts, COPY protocol, full-text search with GIN indexes, cursor-based pagination, and schema migrations with goose
- Project structure — cmd/, internal/, dependency injection through the Server struct, and compiler-enforced boundaries
- Security — API key auth with SHA-256, bcrypt password hashing, session cookies, per-key rate limiting with token buckets, CORS, and SQL injection prevention
- Middleware — the func(http.Handler) http.Handler pattern for logging, panic recovery, auth, and composable handler chains
- Server-rendered dashboard — Go templates, Chart.js analytics, filter bar, embedded static assets with ETag caching via go:embed
- Tests at every layer — httptest, real PostgreSQL, benchmarks, transaction-based isolation, and the DBTX interface pattern
- Graceful shutdown — signal handling, request draining, buffer flushing, health and readiness probes for orchestrators
- Ship to production — multi-stage Dockerfile, ldflags version embedding, Makefile, systemd, and Caddy with auto-TLS
The Ingestion Server
Parse incoming log entries from the request body. Validate required fields. Return proper status codes and JSON error responses.
Define the LogEntry struct with JSON tags. Custom time.Time unmarshaling, input validation, max body size, and enveloped error responses.
Set up go.mod for logline. Add third-party packages, pin versions, vendor dependencies. The real-project workflow for dependency management.
Move from a single main.go to cmd/, internal/, config. Dependency injection without frameworks. Why clean architecture is overkill for Go.
The three layers: defaults, env vars, CLI flags. Config structs with validation. Secrets management. The twelve-factor app in Go.
Use log/slog for structured logging. JSON and text handlers, log levels, attributes, groups. Request-scoped logging with trace IDs.
The Database Layer
Connect to PostgreSQL with pgx. Why sql.DB is a pool, not a connection. Pool sizing, context-aware queries. Create the logs table with jsonb.
INSERT log entries with parameterized queries. Batch inserts for throughput. COPY protocol for bulk loading. SQL injection prevention.
One INSERT per request caps throughput at a few hundred rows/second. Multi-row INSERT, the COPY protocol, buffering with mutexes, and the durability trade-offs.
GET /logs with filters. Full-text search on message field with GIN indexes. Pagination with cursors. Time-range queries.
Schema versioning with goose. Up/down migrations. sql.Tx for multi-step operations. The defer tx.Rollback() idiom.
Middleware & Security
func(http.Handler) http.Handler — the only pattern you need. Request logging, panic recovery, request ID injection, and composable middleware chains.
Generate API keys for log sources. Store hashed keys in the database. Middleware validates Authorization headers on /ingest.
User accounts for the dashboard. Password hashing with bcrypt. Session tokens in cookies. Login, logout, session expiry.
Token bucket per API key. golang.org/x/time/rate. 429 responses with Retry-After. CORS headers and preflight requests.
The Dashboard
html/template from scratch. Base layout with navigation. Log viewer with level badges. Template inheritance, auto-escaping, custom functions.
Dashboard filter bar with level, service, and date range. Form submission via GET params. Server-side filtering and cursor-based pagination.
SQL aggregation: errors per hour, log volume by service, top error messages. GROUP BY with time buckets. Chart.js integration.
Serve CSS, JS, and images. http.FileServer, http.StripPrefix. go:embed for single-binary deployment. Cache headers and ETags.
Production
Spin up a test database. Test the full flow: create API key, ingest logs, query them back. Transaction-based test isolation.
Graceful Shutdown & Health Checks
Signal handling, http.Server.Shutdown, draining in-flight requests. /health and /ready endpoints. Background log retention worker.
go build with ldflags for version. Multi-stage Dockerfile. Makefile for common tasks. Deploy to a VPS with Caddy and auto-TLS.