gRPC之gRPC转换HTTP

1、gRPC转换HTTP

我们通常把RPC用作内部通信，而使用Restful Api进行外部通信。为了避免写两套应用，我们使用grpc-

gateway 把gRPC转成HTTP。服务接收到HTTP请求后，grpc-gateway把它转成gRPC进行处理，然后以JSON

形式返回数据。本篇代码最终转成的Restful Api支持bearer token验证、数据验证，并添加swagger文档。

1.1 编写proto

// simple.proto
// 协议为proto3
syntax = "proto3";
package proto;
option go_package = "./simple;proto";import "validator.proto";
import "google/api/annotations.proto";message InnerMessage {// some_integer can only be in range (1, 100).int32 some_integer = 1 [(validator.field) = {int_gt: 0, int_lt: 100}];// some_float can only be in range (0;1).double some_float = 2 [(validator.field) = {float_gte: 0, float_lte: 1}];
}message OuterMessage {// important_string must be a lowercase alpha-numeric of 5 to 30 characters (RE2 syntax).string important_string = 1 [(validator.field) = {regex: "^[a-z]{2,5}$"}];// proto3 doesn't have `required`, the `msg_exist` enforces presence of InnerMessage.InnerMessage inner = 2 [(validator.field) = {msg_exists : true}];
}service Simple{rpc Route (InnerMessage) returns (OuterMessage){option (google.api.http) ={post:"/v1/example/route"body:"*"};}
}

可以看到，proto变化不大，只是添加了API的路由路径：

option (google.api.http) ={post:"/v1/example/route"body:"*"
};

annotations.proto文件的内容：

// ./proto/google/api/annotations.proto
syntax = "proto3";package google.api;option go_package = "google/api;google_api";import "google/api/http.proto";
import "google/protobuf/descriptor.proto";option java_multiple_files = true;
option java_outer_classname = "AnnotationsProto";
option java_package = "com.google.api";extend google.protobuf.MethodOptions {HttpRule http = 72295728;}

http.proto文件的内容：

// ./proto/google/api/http.proto
syntax = "proto3";package google.api;option go_package = "google/api;google_api";option cc_enable_arenas = true;
option java_multiple_files = true;
option java_outer_classname = "HttpProto";
option java_package = "com.google.api";message Http {repeated HttpRule rules = 1;
}message HttpRule {string selector = 1;oneof pattern {string get = 2;string put = 3;string post = 4;string delete = 5;string patch = 6;CustomHttpPattern custom = 8;}string body = 7;repeated HttpRule additional_bindings = 11;
}message CustomHttpPattern {string kind = 1;string path = 2;
}

1.2 编译proto

$ cd proto# 编译google.api
$ protoc -I . --go_out=plugins=grpc:. google/api/*.proto# 编译simple.proto
$ protoc -I . --go_out=plugins=grpc:. simple/*.proto# 编译simple.proto gateway
$ protoc --grpc-gateway_out=logtostderr=true:. simple/simple.proto# 编译simple.proto validator
$ protoc --govalidators_out=. --go_out=plugins=grpc:. simple/simple.proto

以上完成proto编译，接着修改服务端代码。

1.3 项目结构

# 项目结构
$ tree demo/
demo/
├── cert
│   ├── ca.crt
│   ├── ca.csr
│   ├── ca.key
│   ├── ca.srl
│   ├── client
│   │   ├── client.csr
│   │   ├── client.key
│   │   └── client.pem
│   ├── openssl.cnf
│   └── server
│       ├── server.csr
│       ├── server.key
│       └── server.pem
├── client
│   ├── auth
│   │   └── auth.go
│   └── client.go
├── go.mod
├── go.sum
├── proto
│   ├── google
│   │   └── api
│   │       ├── annotations.pb.go
│   │       ├── annotations.proto
│   │       ├── http.pb.go
│   │       └── http.proto
│   ├── simple
│   │   ├── simple.pb.go
│   │   ├── simple.pb.gw.go
│   │   ├── simple.proto
│   │   └── simple.validator.pb.go
│   └── validator.proto
└── server├── gateway│   └── gateway.go├── log│   └── debug.log├── middleware│   ├── auth│   │   └── auth.go│   ├── cred│   │   └── cred.go│   ├── recovery│   │   └── recovery.go│   └── zap│       └── zap.go└── server.go17 directories, 31 files

1.3.1 gateway.go

package gatewayimport ("context""crypto/tls"pb "demo/proto/simple""github.com/grpc-ecosystem/grpc-gateway/runtime""golang.org/x/net/http2""golang.org/x/net/http2/h2c""google.golang.org/grpc""google.golang.org/grpc/credentials""io/ioutil""log""net/http""strings"
)// ProvideHTTP 把gRPC服务转成HTTP服务，让gRPC同时支持HTTP
func ProvideHTTP(endpoint string, grpcServer *grpc.Server) *http.Server {ctx := context.Background()//获取证书creds, err := credentials.NewClientTLSFromFile("../cert/server/server.pem", "test.example.com")if err != nil {log.Fatalf("Failed to create TLS credentials %v", err)}//添加证书dopts := []grpc.DialOption{grpc.WithTransportCredentials(creds)}//新建gwmux，它是grpc-gateway的请求复用器。它将http请求与模式匹配，并调用相应的处理程序。gwmux := runtime.NewServeMux()//将服务的http处理程序注册到gwmux。处理程序通过endpoint转发请求到grpc端点err = pb.RegisterSimpleHandlerFromEndpoint(ctx, gwmux, endpoint, dopts)if err != nil {log.Fatalf("Register Endpoint err: %v", err)}//新建mux，它是http的请求复用器mux := http.NewServeMux()//注册gwmuxmux.Handle("/", gwmux)log.Println(endpoint + " HTTP.Listing with TLS and token...")return &http.Server{Addr:      endpoint,Handler:   grpcHandlerFunc(grpcServer, mux),TLSConfig: getTLSConfig(),}
}// grpcHandlerFunc 根据不同的请求重定向到指定的Handler处理
func grpcHandlerFunc(grpcServer *grpc.Server, otherHandler http.Handler) http.Handler {return h2c.NewHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {if r.ProtoMajor == 2 && strings.Contains(r.Header.Get("Content-Type"), "application/grpc") {grpcServer.ServeHTTP(w, r)} else {otherHandler.ServeHTTP(w, r)}}), &http2.Server{})
}// getTLSConfig获取TLS配置
func getTLSConfig() *tls.Config {cert, _ := ioutil.ReadFile("../cert/server/server.pem")key, _ := ioutil.ReadFile("../cert/server/server.key")var demoKeyPair *tls.Certificatepair, err := tls.X509KeyPair(cert, key)if err != nil {log.Fatalf("TLS KeyPair err: %v\n", err)}demoKeyPair = &pairreturn &tls.Config{Certificates: []tls.Certificate{*demoKeyPair},NextProtos:   []string{http2.NextProtoTLS}, // HTTP2 TLS支持}
}

1.3.2 server.go

package mainimport ("context""crypto/tls"pb "demo/proto/simple""demo/server/gateway""demo/server/middleware/auth""demo/server/middleware/cred""demo/server/middleware/recovery""demo/server/middleware/zap"grpc_middleware "github.com/grpc-ecosystem/go-grpc-middleware"grpc_auth "github.com/grpc-ecosystem/go-grpc-middleware/auth"grpc_zap "github.com/grpc-ecosystem/go-grpc-middleware/logging/zap"grpc_recovery "github.com/grpc-ecosystem/go-grpc-middleware/recovery"grpc_validator "github.com/grpc-ecosystem/go-grpc-middleware/validator""google.golang.org/grpc""log""net"
)// SimpleService 定义我们的服务
type SimpleService struct{}// Route 实现Route方法
func (s *SimpleService) Route(ctx context.Context, req *pb.InnerMessage) (*pb.OuterMessage, error) {res := pb.OuterMessage{ImportantString: "hello grpc validator",Inner:           req,}return &res, nil
}const (// Address 监听地址Address string = "127.0.0.1:8000"// Network 网络通信协议Network string = "tcp"
)func main() {// 监听本地端口listener, err := net.Listen(Network, Address)if err != nil {log.Fatalf("net.Listen err: %v", err)}// 新建gRPC服务器实例grpcServer := grpc.NewServer(cred.TLSInterceptor(),grpc.StreamInterceptor(grpc_middleware.ChainStreamServer(grpc_validator.StreamServerInterceptor(),grpc_zap.StreamServerInterceptor(zap.ZapInterceptor()),grpc_auth.StreamServerInterceptor(auth.AuthInterceptor),grpc_recovery.StreamServerInterceptor(recovery.RecoveryInterceptor()),)),grpc.UnaryInterceptor(grpc_middleware.ChainUnaryServer(grpc_validator.UnaryServerInterceptor(),grpc_zap.UnaryServerInterceptor(zap.ZapInterceptor()),grpc_auth.UnaryServerInterceptor(auth.AuthInterceptor),grpc_recovery.UnaryServerInterceptor(recovery.RecoveryInterceptor()),)),)// 在gRPC服务器注册我们的服务pb.RegisterSimpleServer(grpcServer, &SimpleService{})log.Println(Address + " net.Listing with TLS and token...")//使用gateway把grpcServer转成httpServerhttpServer := gateway.ProvideHTTP(Address, grpcServer)//用服务器 Serve() 方法以及我们的端口信息区实现阻塞等待，直到进程被杀死或者 Stop() 被调用if err = httpServer.Serve(tls.NewListener(listener, httpServer.TLSConfig)); err != nil {log.Fatal("ListenAndServe: ", err)}
}

1.3.3 auth.go

package authimport ("context""errors"grpc_auth "github.com/grpc-ecosystem/go-grpc-middleware/auth""google.golang.org/grpc""google.golang.org/grpc/codes"
)// TokenInfo 用户信息
type TokenInfo struct {ID    stringRoles []string
}// AuthInterceptor 认证拦截器，对以authorization为头部，形式为`bearer token`的Token进行验证
func AuthInterceptor(ctx context.Context) (context.Context, error) {token, err := grpc_auth.AuthFromMD(ctx, "bearer")if err != nil {return nil, err}tokenInfo, err := parseToken(token)if err != nil {return nil, grpc.Errorf(codes.Unauthenticated, " %v", err)}//使用context.WithValue添加了值后，可以用Value(key)方法获取值newCtx := context.WithValue(ctx, tokenInfo.ID, tokenInfo)//log.Println(newCtx.Value(tokenInfo.ID))return newCtx, nil
}//解析token，并进行验证
func parseToken(token string) (TokenInfo, error) {var tokenInfo TokenInfoif token == "grpc.auth.token" {tokenInfo.ID = "1"tokenInfo.Roles = []string{"admin"}return tokenInfo, nil}return tokenInfo, errors.New("Token无效: bearer " + token)
}//从token中获取用户唯一标识
func userClaimFromToken(tokenInfo TokenInfo) string {return tokenInfo.ID
}

1.3.4 cred.go

package credimport ("google.golang.org/grpc""google.golang.org/grpc/credentials""log"
)// TLSInterceptor TLS证书认证
func TLSInterceptor() grpc.ServerOption {// 从输入证书文件和密钥文件为服务端构造TLS凭证creds, err := credentials.NewServerTLSFromFile("../cert/server/server.pem", "../cert/server/server.key")if err != nil {log.Fatalf("Failed to generate credentials %v", err)}return grpc.Creds(creds)
}

1.3.5 recovery.go

package recoveryimport (grpc_recovery "github.com/grpc-ecosystem/go-grpc-middleware/recovery""google.golang.org/grpc""google.golang.org/grpc/codes"
)// RecoveryInterceptor panic时返回Unknown错误吗
func RecoveryInterceptor() grpc_recovery.Option {return grpc_recovery.WithRecoveryHandler(func(p interface{}) (err error) {return grpc.Errorf(codes.Unknown, "panic triggered: %v", p)})
}

1.3.6 zap.go

package zapimport (grpc_zap "github.com/grpc-ecosystem/go-grpc-middleware/logging/zap""go.uber.org/zap""go.uber.org/zap/zapcore""gopkg.in/natefinch/lumberjack.v2"
)// ZapInterceptor返回zap.logger实例(把日志写到文件中)
func ZapInterceptor() *zap.Logger {w := zapcore.AddSync(&lumberjack.Logger{Filename:  "log/debug.log",MaxSize:   1024, //MBLocalTime: true,})config := zap.NewProductionEncoderConfig()config.EncodeTime = zapcore.ISO8601TimeEncodercore := zapcore.NewCore(zapcore.NewJSONEncoder(config),w,zap.NewAtomicLevel(),)logger := zap.New(core, zap.AddCaller(), zap.AddCallerSkip(1))grpc_zap.ReplaceGrpcLogger(logger)return logger
}// ZapInterceptor 返回zap.logger实例(把日志输出到控制台)
// func ZapInterceptor() *zap.Logger {
// 	logger, err := zap.NewDevelopment()
// 	if err != nil {
// 		log.Fatalf("failed to initialize zap logger: %v", err)
// 	}
// 	grpc_zap.ReplaceGrpcLogger(logger)
// 	return logger
// }

1.3.7 client.go

package mainimport ("context""demo/client/auth"pb "demo/proto/simple""google.golang.org/grpc""google.golang.org/grpc/credentials""log"
)// Address 连接地址
const Address string = ":8000"var grpcClient pb.SimpleClientfunc main() {//从输入的证书文件中为客户端构造TLS凭证creds, err := credentials.NewClientTLSFromFile("../cert/server/server.pem", "test.example.com")if err != nil {log.Fatalf("Failed to create TLS credentials %v", err)}//构建Tokentoken := auth.Token{Value: "bearer grpc.auth.token",}// 连接服务器conn, err := grpc.Dial(Address, grpc.WithTransportCredentials(creds), grpc.WithPerRPCCredentials(&token))if err != nil {log.Fatalf("net.Connect err: %v", err)}defer conn.Close()// 建立gRPC连接grpcClient = pb.NewSimpleClient(conn)route()
}// route 调用服务端Route方法
func route() {// 创建发送结构体req := pb.InnerMessage{SomeInteger: 99,SomeFloat:   1,}// 调用我们的服务(Route方法)// 同时传入了一个 context.Context ，在有需要时可以让我们改变RPC的行为，比如超时/取消一个正在运行的RPCres, err := grpcClient.Route(context.Background(), &req)if err != nil {log.Fatalf("Call Route err: %v", err)}// 打印返回值log.Println(res)
}

1.3.8 auth.go

package authimport ("context"
)// Token token认证
type Token struct {Value string
}const headerAuthorize string = "authorization"// GetRequestMetadata 获取当前请求认证所需的元数据
func (t *Token) GetRequestMetadata(ctx context.Context, uri ...string) (map[string]string, error) {return map[string]string{headerAuthorize: t.Value}, nil
}// RequireTransportSecurity 是否需要基于 TLS 认证进行安全传输
func (t *Token) RequireTransportSecurity() bool {return true
}

1.4 测试

[root@zsx server]# go run server.go
2023/02/12 13:31:42 127.0.0.1:8000 net.Listing with TLS and token...
2023/02/12 13:31:42 127.0.0.1:8000 HTTP.Listing with TLS and token...

[root@zsx client]# go run client.go
2023/02/12 13:31:50 important_string:"hello grpc validator" inner:{some_integer:99 some_float:1}

1.5 发送http请求

[root@zsx client]# curl -X POST -k https://127.0.0.1:8000/v1/example/route --header 'Authorization: Bearer grpc.auth.token' -d '{"some_integer": 10,"some_float": 1}'
{"important_string":"hello grpc validator","inner":{"some_integer":10,"some_float":1}}

会进行输入的校验：

[root@zsx client]# curl -X POST -k https://127.0.0.1:8000/v1/example/route --header 'Authorization: Bearer grpc.auth.token' -d '{"some_integer": 200,"some_float": 1}'
{"error":"invalid field SomeInteger: value '200' must be less than '100'","code":3,"message":"invalid field SomeInteger: value '200' must be less than '100'"}

会进行auth的验证：

[root@zsx client]# curl -X POST -k https://127.0.0.1:8000/v1/example/route -d '{"some_integer": 10,"some_float": 1}'
{"error":"Request unauthenticated with bearer","code":16,"message":"Request unauthenticated with bearer"}

在上面可以看到，我们的gRPC服务已经同时支持RPC和HTTP请求了，而且API接口支持bearer token验证和数

据验证。为了方便对接，我们把API接口生成swagger文档。

1.6 生成swagger文档

1.6.1 生成swagger文档(simple.swagger.json)

1、安装protoc-gen-swagger

$ go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger
$ go install github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger

2、编译生成simple.swagger.json

$ cd proto
$ protoc --swagger_out=logtostderr=true:. simple/simple.proto

1.6.2 把swagger-ui转成Go代码，备用

1、下载swagger-ui

从 https://github.com/swagger-api/swagger-ui下载包，解压把dist目录下的所有文件拷贝我们项目的

server/swagger/swagger-ui/目录下。

$ tree ./server/swagger/swagger-ui/
./server/swagger/swagger-ui/
├── favicon-16x16.png
├── favicon-32x32.png
├── index.css
├── index.html
├── oauth2-redirect.html
├── swagger-initializer.js
├── swagger-ui-bundle.js
├── swagger-ui-bundle.js.map
├── swagger-ui.css
├── swagger-ui.css.map
├── swagger-ui-es-bundle-core.js
├── swagger-ui-es-bundle-core.js.map
├── swagger-ui-es-bundle.js
├── swagger-ui-es-bundle.js.map
├── swagger-ui.js
├── swagger-ui.js.map
├── swagger-ui-standalone-preset.js
└── swagger-ui-standalone-preset.js.map0 directories, 18 files

2、把Swagger UI转换为Go代码

安装go-bindata：

$ go get -u github.com/jteeuwen/go-bindata/...
$ go install github.com/jteeuwen/go-bindata/...

回到server/所在目录，运行指令把Swagger UI转成Go代码：

$ go-bindata --nocompress -pkg swagger -o swagger/datafile.go swagger/swagger-ui/...

在swagger目录下回生成datafile.go文件。

1.6.3 对外提供swagger-ui

1、在swagger/目录下新建swagger.go文件

package swaggerimport (assetfs "github.com/elazarl/go-bindata-assetfs""log""net/http""path""strings"
)// ServeSwaggerFile把proto文件夹中的swagger.json文件暴露出去
func ServeSwaggerFile(w http.ResponseWriter, r *http.Request) {if !strings.HasSuffix(r.URL.Path, "swagger.json") {log.Printf("Not Found: %s", r.URL.Path)http.NotFound(w, r)return}p := strings.TrimPrefix(r.URL.Path, "/swagger/")// "../proto/"为.swagger.json所在目录p = path.Join("../proto/simple/", p)log.Printf("Serving swagger-file: %s", p)http.ServeFile(w, r, p)
}// ServeSwaggerUI 对外提供swagger-ui
func ServeSwaggerUI(mux *http.ServeMux) {fileServer := http.FileServer(&assetfs.AssetFS{Asset:    Asset,AssetDir: AssetDir,Prefix:   "swagger/swagger-ui", //swagger-ui文件夹所在目录})prefix := "/swagger-ui/"mux.Handle(prefix, http.StripPrefix(prefix, fileServer))
}

2、注册swagger

在gateway.go中添加如下代码：

//新建mux，它是http的请求复用器
mux := http.NewServeMux()
//注册gwmux
mux.Handle("/", gwmux)
//注册swagger
mux.HandleFunc("/swagger/", swagger.ServeSwaggerFile)
swagger.ServeSwaggerUI(mux)

3、到这里我们已经完成了swagger文档的添加工作了，重新运行server.go。

[root@zsx server]# go run server.go
2023/02/12 16:59:27 127.0.0.1:8000 net.Listing with TLS and token...
2023/02/12 16:59:27 127.0.0.1:8000 HTTP.Listing with TLS and token...

由于谷歌浏览器不能使用自己制作的TLS证书，所以我们用火狐浏览器进行测试。

用火狐浏览器打开：https://127.0.0.1:8000/swagger-ui/

在最上面地址栏输入：https://127.0.0.1:8000/swagger/simple.swagger.json

然后就可以看到swagger生成的API文档了：

还有个问题，我们使用了bearer token进行接口验证的，怎么把bearer token也添加到swagger中呢？

在grpc-gatewayGitHub上的这个Issues找到解决办法：

https://github.com/grpc-ecosystem/grpc-gateway/issues/1089

1.6.4 在swagger中配置bearer token

1、修改simple.proto文件

// 协议为proto3
syntax = "proto3";
package proto;
option go_package = "./simple;proto";import "validator.proto";
import "google/api/annotations.proto";
import "protoc-gen-swagger/options/annotations.proto";message InnerMessage {// some_integer can only be in range (1, 100).int32 some_integer = 1 [(validator.field) = {int_gt: 0, int_lt: 100}];// some_float can only be in range (0;1).double some_float = 2 [(validator.field) = {float_gte: 0, float_lte: 1}];
}message OuterMessage {// important_string must be a lowercase alpha-numeric of 5 to 30 characters (RE2 syntax).string important_string = 1 [(validator.field) = {regex: "^[a-z]{2,5}$"}];// proto3 doesn't have `required`, the `msg_exist` enforces presence of InnerMessage.InnerMessage inner = 2 [(validator.field) = {msg_exists : true}];
}option (grpc.gateway.protoc_gen_swagger.options.openapiv2_swagger) = {security_definitions: {security: {key: "bearer"value: {type: TYPE_API_KEYin: IN_HEADERname: "Authorization"description: "Authentication token, prefixed by Bearer: Bearer <token>"}}}security: {security_requirement: {key: "bearer"}}info: {title: "grpc gateway sample";version: "1.0";license: {name: "MIT";};}schemes: HTTPS
};service Simple{rpc Route (InnerMessage) returns (OuterMessage){option (google.api.http) ={post:"/v1/example/route"body:"*"};// 禁用bearer token// option (grpc.gateway.protoc_gen_swagger.options.openapiv2_operation) = {//   security: { } // Disable security key// };}
}

2、重新编译生成simple.swagger.json。

$ cd proto
$ protoc --swagger_out=logtostderr=true:. simple/simple.proto

1.6.5 验证测试

1、添加bearer token

填写token：

2、调用接口，正确返回数据

填写数据：

进行测试：

3、传递不合规则的数据，返回违反数据验证逻辑错误

填写数据：

进行测试：

# 项目结构
$ tree demo/
demo/
├── cert
│   ├── ca.crt
│   ├── ca.csr
│   ├── ca.key
│   ├── ca.srl
│   ├── client
│   │   ├── client.csr
│   │   ├── client.key
│   │   └── client.pem
│   ├── openssl.cnf
│   └── server
│       ├── server.csr
│       ├── server.key
│       └── server.pem
├── client
│   ├── auth
│   │   └── auth.go
│   └── client.go
├── go.mod
├── go.sum
├── proto
│   ├── google
│   │   └── api
│   │       ├── annotations.pb.go
│   │       ├── annotations.proto
│   │       ├── http.pb.go
│   │       └── http.proto
│   ├── protoc-gen-swagger
│   │   └── options
│   │       ├── annotations.proto
│   │       └── openapiv2.proto
│   ├── simple
│   │   ├── simple.pb.go
│   │   ├── simple.pb.gw.go
│   │   ├── simple.proto
│   │   ├── simple.swagger.json
│   │   └── simple.validator.pb.go
│   └── validator.proto
└── server├── gateway│   └── gateway.go├── log│   └── debug.log├── middleware│   ├── auth│   │   └── auth.go│   ├── cred│   │   └── cred.go│   ├── recovery│   │   └── recovery.go│   └── zap│       └── zap.go├── server.go└── swagger├── datafile.go├── swagger.go└── swagger-ui├── favicon-16x16.png├── favicon-32x32.png├── index.css├── index.html├── oauth2-redirect.html├── swagger-initializer.js├── swagger-ui-bundle.js├── swagger-ui-bundle.js.map├── swagger-ui.css├── swagger-ui.css.map├── swagger-ui-es-bundle-core.js├── swagger-ui-es-bundle-core.js.map├── swagger-ui-es-bundle.js├── swagger-ui-es-bundle.js.map├── swagger-ui.js├── swagger-ui.js.map├── swagger-ui-standalone-preset.js└── swagger-ui-standalone-preset.js.map21 directories, 54 files

1.7 总结

本篇介绍了如何使用grpc-gateway让gRPC同时支持HTTP，最终转成的Restful Api支持bearer token验

证、数据验证。同时生成swagger文档，方便API接口对接。