Building a web service with go-tool-base, part 2: a gRPC service, with TLS

Tue, 26 May 2026 00:00:00 +0000

The heartbeat from part 1 runs, ticks along, and shuts down politely when you ask it to. It also talks to absolutely no one. A service people can actually call needs an API, and for a typed, fast, streaming-capable one, gRPC is the obvious first move.

The catch is that a production-grade gRPC server is rather more than grpc.NewServer(). You want health checks an orchestrator understands, reflection so you can poke at it without the .proto file in hand, a graceful shutdown that doesn’t guillotine calls that are still in flight, and TLS, which is where most people’s first attempt quietly goes wrong. The good news: part 1 already gave us the thing that carries all of it. A gRPC server is just another service to register on the controller.

Why gRPC, and not just REST

Worth a moment on why we’re reaching for gRPC at all, because for plenty of services a plain JSON-over-HTTP API is the right call and less faff. gRPC earns its place when a few of these matter:

A contract that’s enforced, not hoped for. The .proto is the single source of truth, and both ends are generated from it. You don’t hand-write JSON marshalling, and you don’t find out at runtime that the client and server disagree about a field’s type. Evolve the schema carefully (add fields by number) and old clients keep working.
Clients in any language, for free. The same .proto generates a Go server and a Python, TypeScript, Rust or Java client with nobody writing an SDK by hand. For an internal service that several teams call, that one point can decide it.
It’s built for service-to-service traffic. Binary protobuf is smaller and quicker to encode than JSON, calls multiplex down a single HTTP/2 connection, and streaming (from the client, the server, or both at once) is a first-class thing rather than something you bolt onto REST with websockets.
Deadlines, cancellation and a health protocol come built in, rather than conventions you reinvent for every service.

The trade-offs are real. A browser doesn’t speak gRPC natively, and a binary protocol is fiddlier to poke at than a JSON endpoint you can curl (which is exactly why reflection and grpcurl exist). That’s not a reason to avoid it; it’s the reason this series doesn’t stop at gRPC. In part 4 we put a REST/JSON face on this very service, so the things that call it get the typed, fast core and the things that can’t speak gRPC still get a friendly HTTP surface. You don’t have to pick a side.

Define the contract

gRPC starts with a schema. Here’s a small macguffin service, macguffin.proto:

// proto/macguffin/v1/macguffin.proto

syntax = "proto3";

package macguffin.v1;

option go_package = "gitlab.com/myorg/macguffinsvc/internal/gen/macguffin/v1;macguffinv1";

service MacguffinService {
 rpc GetMacguffin(GetMacguffinRequest) returns (Macguffin);
 rpc ListMacguffins(ListMacguffinsRequest) returns (ListMacguffinsResponse);
 rpc CreateMacguffin(CreateMacguffinRequest) returns (Macguffin);
}

message Macguffin {
 string id = 1;
 string name = 2;
 int32 quantity = 3;
}

message GetMacguffinRequest { string id = 1; }
message ListMacguffinsRequest { int32 page_size = 1; }
message ListMacguffinsResponse { repeated Macguffin macguffins = 1; }
message CreateMacguffinRequest { string name = 1; int32 quantity = 2; }

From proto to Go

If gRPC in Go is new to you, this is the part that catches people out: you don’t write the Go for those messages and that service interface, you generate it from the .proto. The proto is the source of truth; a compiler turns it into Go you import and build against. Same goes for a client in any other language, all from the same file.

That compiler is protoc, and on its own it’s a faff. You install it, then a separate plugin for each output you want (protoc-gen-go for the message types, protoc-gen-go-grpc for the client and server stubs), keep their versions in step, and drive the lot with a command line of -I include paths and --*_out flags that’s easy to get subtly wrong.

buf is the friendlier way to run exactly that. It wraps protoc and its plugins behind a couple of small config files, handles the plugin versions, and turns that gnarly invocation into a single word. It’s become the usual way to work with protobuf in Go, and it’s what we’ll use here.

At a minimum you need three binaries on your PATH: buf itself, and the two plugins it drives. go install is the quickest way to get them:

go install github.com/bufbuild/buf/cmd/buf@latest
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

Those land in $(go env GOPATH)/bin, so make sure that’s on your $PATH. Then describe what you want generated in a buf.gen.yaml:

# buf.gen.yaml
version: v2
plugins:
 - local: protoc-gen-go
 out: internal/gen
 opt: paths=source_relative
 - local: protoc-gen-go-grpc
 out: internal/gen
 opt: paths=source_relative

Each part of that earns its place. version: v2 is buf’s config format. The plugins list names the generators to run, and we run two, because gRPC in Go arrives in two halves:

protoc-gen-go turns the messages into Go structs, the Macguffin type and the request and response types, in a macguffin.pb.go file.
protoc-gen-go-grpc turns the service into the client and server scaffolding, in a macguffin_grpc.pb.go file, including the MacguffinServiceServer interface you’re about to implement.

out: internal/gen is where the files land, and paths=source_relative lays them out mirroring the proto’s own folders (so proto/macguffin/v1/... becomes internal/gen/macguffin/v1/...) rather than deriving the path from the go_package line. Then run it:

buf generate

Both files appear under internal/gen/macguffin/v1, and we’re ready to write the implementation.

Running that by hand once is fine; remembering to run it every time the .proto changes is where it goes wrong, and the generated code quietly drifts out of step, usually right before a demo. Wire it into go generate instead. Drop a one-line directive in a file at your module root, say gen.go:

// gen.go (at your module root)

package macguffinsvc

//go:generate buf generate

Now go generate ./... regenerates everything from the proto, and it’s the same one command for any other generator you add later. Run it whenever the schema changes, and in CI if you want to catch a stale checkout.

If OpenAPI is your map of the territory

If your mental model of an API contract is an OpenAPI (Swagger) document, a .proto is the same idea wearing fewer clothes: a typed, language-neutral description of a service that both ends generate from. The thing you notice first is how much less of it there is. Here’s that Macguffin message again, and the same shape written as an OpenAPI schema:

message Macguffin {
 string id = 1;
 string name = 2;
 int32 quantity = 3;
}

Macguffin:
 type: object
 properties:
 id:
 type: string
 name:
 type: string
 quantity:
 type: integer
 format: int32

And that pattern holds across the whole service. The proto above, three calls and five messages, is about twenty lines. Describe the same surface in OpenAPI and you’re closer to a hundred, because OpenAPI also pins down the HTTP verbs, paths, status codes and content types: the transport details a proto leaves out on purpose. That isn’t OpenAPI being bloated; it’s describing more. But when the contract is the thing you care about, the proto says it with less ceremony, and it doesn’t wed your API to HTTP, which is exactly what lets us serve this same service over gRPC now and REST later. (We’ll generate a real OpenAPI document from this proto in part 5, for the readers who still want one.)

Implement it

Generating the code gave you the message types and, more to the point, an interface to satisfy. Open macguffin_grpc.pb.go and you’ll find MacguffinServiceServer, one method per RPC in the proto:

// internal/gen/macguffin/v1/macguffin_grpc.pb.go (generated)

type MacguffinServiceServer interface {
	GetMacguffin(context.Context, *GetMacguffinRequest) (*Macguffin, error)
	ListMacguffins(context.Context, *ListMacguffinsRequest) (*ListMacguffinsResponse, error)
	CreateMacguffin(context.Context, *CreateMacguffinRequest) (*Macguffin, error)
	mustEmbedUnimplementedMacguffinServiceServer()
}

That interface is the server-side contract. Each method takes the request message you defined and hands back the response message, plus an error. Writing the type that honours it, the actual logic behind each call, is the part that’s yours: the proto pins down the shape of the conversation, and this is what the service actually does when one happens.

The one curious line is mustEmbedUnimplementedMacguffinServiceServer(). Alongside the interface, buf generated an UnimplementedMacguffinServiceServer struct with a do-nothing stub for every method, and you embed it in your own type. It earns its keep twice over. It satisfies that unexported method, so your type counts as a real implementation. And it future-proofs you: add a new RPC to the proto later and your existing server still compiles, falling back to the stub (which returns a clean “unimplemented” error) until you write the real method.

Before we satisfy that interface, one separation worth making up front. The gRPC server is a delivery mechanism, not the place the data lives. If we stuff the map of macguffins straight inside it and then build an HTTP server next part, we’d have two servers each hoarding their own copy of the same data. So keep the domain, the macguffins and what you can do with them, in its own type, and let each transport be a thin layer over it.

Here’s that domain: an in-memory store standing in for the repository a real service would have. Nothing in it knows about gRPC, HTTP, or any wire format.

// internal/macguffin/store.go

package macguffin

import "sync"

// Macguffin is the domain type. The JSON tags will let a hand-written HTTP
// handler serve it directly in part 3.
type Macguffin struct {
	ID string `json:"id"`
	Name string `json:"name"`
	Quantity int32 `json:"quantity"`
}

type Store struct {
	mu sync.Mutex
	items map[string]Macguffin
	seq int
}

func (s *Store) Get(id string) (Macguffin, bool) {
	s.mu.Lock()
	defer s.mu.Unlock()

	m, ok := s.items[id]

	return m, ok
}

List and Create are the same shape, and NewStore seeds it with a single maltese-falcon. Now the gRPC server is thin: it embeds the stub, holds a store, and each method calls the store and translates the result into the generated protobuf type.

// internal/grpcsvc/server.go

package grpcsvc

import (
	"context"

	"google.golang.org/grpc/codes"
	"google.golang.org/grpc/status"

	macguffinv1 "gitlab.com/myorg/macguffinsvc/internal/gen/macguffin/v1"
	"gitlab.com/myorg/macguffinsvc/internal/macguffin"
)

type Server struct {
	macguffinv1.UnimplementedMacguffinServiceServer

	store *macguffin.Store
}

func New(store *macguffin.Store) *Server {
	return &Server{store: store}
}

func (s *Server) GetMacguffin(_ context.Context, req *macguffinv1.GetMacguffinRequest) (*macguffinv1.Macguffin, error) {
	m, ok := s.store.Get(req.GetId())
	if !ok {
		return nil, status.Errorf(codes.NotFound, "macguffin %q not found", req.GetId())
	}

	return toProto(m), nil
}

// toProto maps the domain type to the generated protobuf DTO.
func toProto(m macguffin.Macguffin) *macguffinv1.Macguffin {
	return &macguffinv1.Macguffin{Id: m.ID, Name: m.Name, Quantity: m.Quantity}
}

ListMacguffins and CreateMacguffin are the same: call the store, map the result. The one habit worth keeping is to return real gRPC status codes (codes.NotFound here) rather than bare errors, so callers get something they can branch on.

That toProto step is worth a second look, because it comes back round later. The domain has one shape, the proto has its own generated Macguffin, so the adapter maps between them. It’s a small price for a single transport. In part 3 we add a second transport with its own generated type and pay that price again, and part 4 is where we stop paying it twice.

Wire it onto the controller

This is the part that earns its keep. First, generate a serve command, the same way the CLI series generated its commands:

gtb generate command \
 --name serve \
 --short "Run the macguffin service"

That scaffolds two files: pkg/cmd/serve/cmd.go (generated, and wired into your command tree for you) and pkg/cmd/serve/main.go, which holds a RunServe function for your logic. Fill it in:

// pkg/cmd/serve/main.go

package serve

import (
	"context"

	"gitlab.com/phpboyscout/go-tool-base/pkg/controls"
	gtbgrpc "gitlab.com/phpboyscout/go-tool-base/pkg/grpc"
	"gitlab.com/phpboyscout/go-tool-base/pkg/props"

	macguffinv1 "gitlab.com/myorg/macguffinsvc/internal/gen/macguffin/v1"
	"gitlab.com/myorg/macguffinsvc/internal/grpcsvc"
	"gitlab.com/myorg/macguffinsvc/internal/macguffin"
)

func RunServe(ctx context.Context, p *props.Props, _ *ServeOptions, _ []string) error {
	controller := controls.NewController(ctx, controls.WithLogger(p.Logger))

	store := macguffin.NewStore()

	grpcSrv, err := gtbgrpc.Register(ctx, "grpc", controller, p.Config, p.Logger)
	if err != nil {
		return err
	}

	macguffinv1.RegisterMacguffinServiceServer(grpcSrv, grpcsvc.New(store))

	controller.Start()
	controller.Wait()

	return nil
}

That’s the whole server. gtbgrpc.Register does four things in one call: it builds a *grpc.Server, wires the standard gRPC health service to the controller’s health reports (the ones we met in part 1), registers Start, Stop and Status against the controller so the lifecycle is handled, and hands you back the server to register your own service on, which is the RegisterMacguffinServiceServer line. After that it’s the same Start() / Wait() we used for the heartbeat.

It reads its port from config (server.grpc.port, falling back to server.port), so a minimal config is:

server:
 grpc:
 port: 50051
 reflection: true

Poke it

Build, run mytool serve, and reach for grpcurl. Reflection is on, so you don’t need the .proto to hand:

$ grpcurl -plaintext localhost:50051 list
grpc.health.v1.Health
grpc.reflection.v1.ServerReflection
grpc.reflection.v1alpha.ServerReflection
macguffin.v1.MacguffinService

$ grpcurl -plaintext localhost:50051 macguffin.v1.MacguffinService/ListMacguffins
{
 "macguffins": [
 { "id": "m-1", "name": "maltese-falcon", "quantity": 1 }
 ]
}

And the health service is already answering, wired straight to the controller, without you registering a thing:

$ grpcurl -plaintext localhost:50051 grpc.health.v1.Health/Check
{ "status": "SERVING" }

That’s the lifecycle work from part 1 paying out: the controller’s health is the gRPC health, and a SIGTERM still drains and stops the server cleanly.

Now turn on TLS

Here’s the bit people brace for. Plaintext gRPC is fine on a laptop and unacceptable the moment it leaves one. With go-tool-base it’s a config change, not a code change.

The fiddly part of local TLS is usually the certificate. A hand-rolled self-signed one means passing a -cacert to every client and clicking past browser warnings. mkcert makes that go away: it creates a local certificate authority and installs it into your system’s (and your browser’s) trust stores, so anything it signs is simply trusted. Set the CA up once:

mkcert -install

Then mint a certificate for the names the service answers on:

mkcert localhost 127.0.0.1 ::1

That writes localhost+2.pem (the certificate) and localhost+2-key.pem (the key), signed by your now-trusted local CA. Doing this properly now pays off later: in part 3 the HTTP server, and in part 5 the API docs in a browser, both lean on that certificate being trusted with no warning.

Point the tool’s config at the pair, under the shared server.tls block:

server:
 grpc:
 port: 50051
 reflection: true
 tls:
 enabled: true
 cert: ./localhost+2.pem
 key: ./localhost+2-key.pem

No code changes. Run mytool serve again and it comes up over TLS:

INFO starting gRPC server tls=true addr=:50051

Because the certificate is signed by a CA your machine already trusts, the client needs no extra flags:

$ grpcurl localhost:50051 macguffin.v1.MacguffinService/GetMacguffin -d '{"id":"m-1"}'
{ "id": "m-1", "name": "maltese-falcon", "quantity": 1 }

A plaintext client is now refused, as it should be. (In production you’d point those same two config keys at whatever your real CA issues; the wiring doesn’t change.) Two details are worth knowing about what just happened, because both are easy to get wrong by hand. The server uses a hardened TLS config (1.2 minimum, AEAD cipher suites, X25519), so you’re not accidentally shipping a weak handshake. And the listener advertises HTTP/2 over ALPN, the h2 protocol gRPC rides on, which sounds like a footnote until you discover that recent gRPC clients refuse a TLS connection that doesn’t offer it. The framework sets it for you; it’s the single most common reason a hand-rolled gRPC-over-TLS server works with old tooling and mysteriously rejects a current client. All of that lives in the shared pkg/tls package.

I put the certificate under server.tls rather than server.grpc.tls deliberately. That shared block is the cert every transport falls back to, so the HTTP server in the next part and the transports after it can use the same one, with a per-transport override only where you actually need it.

The short version

A few files in, you have a real gRPC API: a typed contract, an implementation, health an orchestrator understands, reflection for poking, a clean shutdown, and TLS, and the only part that was actually yours to write was the service logic. The reference for the server helpers is the gRPC component doc, and the add-a-gRPC-service how-to has the manual-wiring path if you ever want it.

Next part puts an HTTP face on the very same controller, REST handlers and the same health endpoints an orchestrator probes, sharing that one certificate.

Tls on PHP Boy Scout