A quick tally of where part
3
left us. One domain, the Store. One gRPC service over it, mapping the domain to
proto with toProto. And then a whole second transport, the REST layer, with its
own routing and its own toDTO mapping the very same domain into the very same
shape, by hand. Two encodings of one thing, drifting apart the moment anyone adds
a field and forgets the other side.
I promised that doubling would go away. This is the part where it does, and the thing that does it is the grpc-gateway.
What the gateway actually is
The grpc-gateway is a reverse proxy, generated from your .proto, that speaks
REST on the front and gRPC on the back. A JSON request comes in, the gateway
turns it into the matching gRPC call, hands it to your gRPC server, and turns the
gRPC response back into JSON on the way out.
Read that again with part 3 in mind, because it’s the whole point. The gateway
does the JSON-to-proto-and-back encoding for you, using the proto types your
gRPC server already produces. You wrote domain → proto once, in the gRPC
adapter. The gateway supplies proto → JSON. There is no second hand-written
encoding to keep in step, because there is no second implementation: REST becomes
a generated front door onto the gRPC service you already have.
So the plan is short. Tell the proto which HTTP calls map to which RPCs, regenerate, wire the gateway in, and delete the part-3 REST layer entirely.
Map HTTP onto the proto
gRPC has no opinion about URLs and verbs; REST is all URLs and verbs. The bridge
is an annotation, google.api.http, that you attach to each RPC to say “this one
is GET /v1/macguffins/{id}”. Here’s the service with those rules added:
// proto/macguffin/v1/macguffin.proto
import "google/api/annotations.proto";
service MacguffinService {
rpc GetMacguffin(GetMacguffinRequest) returns (Macguffin) {
option (google.api.http) = {get: "/v1/macguffins/{id}"};
}
rpc ListMacguffins(ListMacguffinsRequest) returns (ListMacguffinsResponse) {
option (google.api.http) = {get: "/v1/macguffins"};
}
rpc CreateMacguffin(CreateMacguffinRequest) returns (Macguffin) {
option (google.api.http) = {
post: "/v1/macguffins"
body: "*"
};
}
}
Each rule is small but exact. {id} in the path binds to the id field of the
request message. body: "*" on the create says the whole JSON body maps onto the
request. The list takes no body and no path parameter, just the verb and path.
This is the same information part 3’s hand-written routes carried, except now it
lives next to the RPC it describes, and a generator reads it instead of you.
These rules go a good deal further than the three cases we need: query-string
parameters, several URL bindings for a single RPC, custom verbs, choosing which
field becomes the response body. When you reach for those, the grpc-gateway
docs walk through
the mapping, and the canonical reference is the HttpRule
message
that google.api.http comes from, its comments document every option.
The annotations.proto import comes from Google’s common protos, so tell buf
where to find them by adding a dependency, then fetch it:
# buf.yaml
deps:
- buf.build/googleapis/googleapis
buf dep update
Generate the gateway
This is another buf plugin, exactly like part 2’s. Install it:
go install github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway@latest
and add it to the generators:
# buf.gen.yaml
version: v2
plugins:
- local: protoc-gen-go
out: internal/gen
opt: paths=source_relative
- local: protoc-gen-go-grpc
out: internal/gen
opt: paths=source_relative
- local: protoc-gen-grpc-gateway
out: internal/gen
opt: paths=source_relative
go generate ./... now also writes macguffin.pb.gw.go, the gateway: a
RegisterMacguffinServiceHandler function that, given a connection to your gRPC
server, mounts the REST routes the annotations described.
Wire it on
The gateway needs to call your gRPC server, which means dialling it like any
other client, over the same TLS, with credentials that trust its certificate.
That’s fiddly to get right by hand, so go-tool-base’s gateway package does it
for you. gateway.New opens the connection (matching your server’s transport
security) and hands you a mux to register the generated handlers on:
// pkg/cmd/serve/main.go
package serve
import (
"context"
stdhttp "net/http"
"github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"google.golang.org/grpc"
"gitlab.com/phpboyscout/go-tool-base/pkg/controls"
"gitlab.com/phpboyscout/go-tool-base/pkg/gateway"
gtbgrpc "gitlab.com/phpboyscout/go-tool-base/pkg/grpc"
gtbhttp "gitlab.com/phpboyscout/go-tool-base/pkg/http"
"gitlab.com/phpboyscout/go-tool-base/pkg/props"
macguffinv1 "gitlab.com/myorg/macguffinsvc/internal/gen/macguffin/v1"
"gitlab.com/myorg/macguffinsvc/internal/grpcsvc"
"gitlab.com/myorg/macguffinsvc/internal/macguffin"
)
func RunServe(ctx context.Context, p *props.Props, _ *ServeOptions, _ []string) error {
controller := controls.NewController(ctx, controls.WithLogger(p.Logger))
store := macguffin.NewStore()
// gRPC: the one implementation, mapping the domain to proto.
grpcSrv, err := gtbgrpc.Register(ctx, "grpc", controller, p.Config, p.Logger)
if err != nil {
return err
}
macguffinv1.RegisterMacguffinServiceServer(grpcSrv, grpcsvc.New(store))
// REST, for free: the gateway proxies JSON/HTTP to the gRPC server above.
gw, err := gateway.New(ctx, p.Config,
func(ctx context.Context, mux *runtime.ServeMux, conn *grpc.ClientConn) error {
return macguffinv1.RegisterMacguffinServiceHandler(ctx, mux, conn)
})
if err != nil {
return err
}
mux := stdhttp.NewServeMux()
mux.Handle("/v1/", gw)
if _, err := gtbhttp.Register(ctx, "http", controller, p.Config, p.Logger, mux); err != nil {
return err
}
controller.Start()
controller.Wait()
return nil
}
The only macguffin-specific line is the one inside the callback,
RegisterMacguffinServiceHandler. Everything around it, the dial, the
credentials, the mux, is the framework’s. Mount the result under /v1/, register
it on the same controller and HTTP server as before, and you’re done.
Delete the duplication
Here’s the satisfying bit. The hand-written REST adapter from part 3, the
resthand package, the routes, the toDTO, all of it, comes out. You don’t need
it: the gateway serves the same REST surface, backed by the gRPC service, from
the proto. The serve command shrinks to one gRPC server and one gateway, and
your codebase now has a single place where a macguffin becomes JSON.
See it work
The gateway answers REST, and it’s the same store the gRPC service uses:
$ curl https://localhost:8443/v1/macguffins
{"macguffins":[{"id":"m-1","name":"maltese-falcon","quantity":1}]}
$ curl -X POST https://localhost:8443/v1/macguffins -d '{"name":"the-grail","quantity":1}'
{"id":"m-2","name":"the-grail","quantity":1}
Create over REST, and the macguffin is there over gRPC a moment later, because
both are the same implementation over the same Store:
$ grpcurl localhost:50051 macguffin.v1.MacguffinService/ListMacguffins
{ "macguffins": [ { "id": "m-1", ... }, { "id": "m-2", "name": "the-grail", ... } ] }
Errors, and changing them
Error handling comes across too. When a gRPC handler returns a status code, the
gateway maps it to the matching HTTP status. The codes.NotFound we returned back
in part 2 arrives as a 404, with a JSON error body, and we wrote none of it:
$ curl -s -o /dev/null -w '%{http_code}\n' https://localhost:8443/v1/macguffins/nope
404
That default mapping is the sensible one you’d reach for anyway. A few of the common codes:
| gRPC code | HTTP |
|---|---|
InvalidArgument | 400 |
Unauthenticated | 401 |
PermissionDenied | 403 |
NotFound | 404 |
AlreadyExists | 409 |
Internal | 500 |
Unavailable | 503 |
So the rule of thumb is simply to return the right codes.* from your gRPC
handlers, and the REST side gets the right status for free.
When the default shape isn’t what your clients expect, a {"error": {…}}
envelope, a trace id header, a tweak to one particular status, you supply your own
error handler. The grpc-gateway takes one as a runtime.ServeMuxOption, and
gateway.New passes those straight through:
gw, err := gateway.New(ctx, p.Config, register,
gateway.WithMuxOptions(runtime.WithErrorHandler(myErrorHandler)),
)
myErrorHandler receives the error and the http.ResponseWriter and writes
whatever your API promises. That same WithMuxOptions hatch takes the gateway’s
other knobs, header matchers, custom marshalers, and the rest; they’re all in the
grpc-gateway docs.
Where this leaves us
This is the shape the series was building towards. One domain, one gRPC
implementation, one domain → proto mapping you wrote by hand, and a REST API
generated from the same proto that needed no second implementation and no second
encoding. The things that speak gRPC get gRPC; the browser, the webhook and the
curl get JSON; and there’s exactly one place to change when a macguffin grows a
field.
Those google.api.http annotations have one more trick in them. They describe
your REST API precisely enough to generate an OpenAPI document, and in part 5 we
serve that as a live, clickable docs site, from the very same proto.
