unity3d：GameFramework+xLua+Protobuf+lua-protobuf，与服务器交互收发协议

概述

1.cs收发协议，通过protobuf序列化
2.lua收发协议，通过lua-protobuf序列化

一条协议字节流组成

C#协议基类

CSPacketBase，SCPacketBaseC#用协议基类

proto生成的CS类，基于这两个基类。分别为CSPacketBase是客户端发送至服务器，SCPacketBase是服务器发送至客户端
Q：为什么要区分这2个
A：反射注册所有SCPacketBase类，为C#接收协议反序列化候选
一个类示例

  [global::System.Serializable, global::ProtoBuf.ProtoContract(Name=@"CSLogin")]public partial class CSLogin : CSPacketBase{public CSLogin() {}private string _account = "";[global::ProtoBuf.ProtoMember(1, IsRequired = false, Name=@"account", DataFormat = global::ProtoBuf.DataFormat.Default)][global::System.ComponentModel.DefaultValue("")]public string account{get { return _account; }set { _account = value; }}private string _password = "";[global::ProtoBuf.ProtoMember(2, IsRequired = false, Name=@"password", DataFormat = global::ProtoBuf.DataFormat.Default)][global::System.ComponentModel.DefaultValue("")]public string password{get { return _password; }set { _password = value; }} //网络协议Idpublic override int Id { get {return (int)Network.NetMsgID.CSLogin;} } //回到引用池，变量设置初始化。如果是引用型成员变量也要回到引用池public override void Clear(){_account = "" ;_password = "" ;}}

SCPacketLua

C#中接收包，传递给Lua处理
其中m_id为协议id，m_len为字节数组长度

public class SCPacketLua : PacketBase
{public int m_id;public int m_len;public PacketBuffer m_bytes; 

在网络事件使用完毕GameFramework.EventPool.HandleEvent 中

ReferencePool.Release(e);

触发回收

    public override void Clear(){ReferencePool.Release(m_bytes);m_bytes = null; //要去引用，不然引用池那释放不了 

CSPacketLua

C#中用于接收从lua传递过来的字节流，发送给服务器

    public  class CSPacketBaseLua : PacketBase{public PacketBuffer m_bytes; //发送字节流public int m_id = 0; //协议idpublic ushort Len //字节流长度

C#中发送协议

CSLogin csLogin = ReferencePool.Acquire<CSLogin>();
csLogin.account = "123";
csLogin.password = "456";
GameEntry.Network.Send(csLogin);

主线程遍历发送队列

每有一个发送，把packet放入到发送队列中，unity主线程中遍历发送队列，把当前帧的全部待发送包，序列化到一个流中

GameFramework.Network.NetworkManager.NetworkChannelBase.ProcessSend

if (m_SendState.Stream.Length > 0 || m_SendPacketPool.Count <= 0)
{//发送流中还有未发送完的或者没有待发送的包return false;
}
//所有在发送队列中，都序列化到流中
while (m_SendPacketPool.Count > 0)
{Packet packet = null;lock (m_SendPacketPool){//每次从发送队列中取一个packet = m_SendPacketPool.Dequeue();}bool serializeResult = false;try{serializeResult = m_NetworkChannelHelper.Serialize(packet, m_SendState.Stream);}
}
//流的操作，写完，要回到起点
m_SendState.Stream.Position = 0L;    

序列化消息包

StarForce.NetworkChannelHelper.Serialize

/// <summary>
/// 序列化消息包。
/// </summary>
/// <typeparam name="T">消息包类型。</typeparam>
/// <param name="packet">要序列化的消息包。</param>
/// <param name="destination">要序列化的目标流。</param>
/// <returns>是否序列化成功。</returns>
public bool Serialize<T>(T packet, Stream destination) where T : Packet
{//todo：频繁is as 会有性能消耗if (packet is CSPacketBase){PacketBase packetImpl = packet as CSPacketBase;//先写入bodym_CachedStream.SetLength(1024*8); // 此行防止 Array.Copy 的数据无法写入m_CachedStream.Position = PacketHeaderLength;Serializer.Serialize(m_CachedStream, packet);//获得body长度,再写入消息头，id，body长度ushort bodyLen = (ushort)((int)m_CachedStream.Position - PacketHeaderLength);arrID = BitConverter.GetBytes(packetImpl.Id);arrBodyLen = BitConverter.GetBytes(bodyLen);m_CachedStream.Position = 0;m_CachedStream.Write(arrID, 0, 4);m_CachedStream.Write(arrBodyLen, 0, 2);m_CachedStream.SetLength(PacketHeaderLength + bodyLen);byte[] arrBytes = m_CachedStream.ToArray();Log.Info("序列化字节流：{0}", BitConverter.ToString(arrBytes));m_CachedStream.WriteTo(destination);ReferencePool.Release(packet);return true;}//else if (packetImpl.PacketType == PacketType.ClientToServerLua)else if (packet is CSPacketLua){m_CachedStream.SetLength(1024 * 8); // 此行防止 Array.Copy 的数据无法写入CSPacketLua packetLua = packet as CSPacketLua;  arrID = BitConverter.GetBytes(packetLua.Id);arrBodyLen = BitConverter.GetBytes(packetLua.Len);m_CachedStream.Position = 0; //每次开始写流，流位置要设置为0，代表起始位置。每次写byte，pos会自动增加m_CachedStream.Write(arrID, 0, 4);m_CachedStream.Write(arrBodyLen, 0, 2);m_CachedStream.Write(packetLua.m_bytes.Buffer, 0, packetLua.Len);m_CachedStream.SetLength(PacketHeaderLength + packetLua.Len);//写完流，要设置下流的真实长度。因为流是复用的，不然不会截断byte[] arrBytes = m_CachedStream.ToArray();Log.Info("序列化字节流Lua：{0}", BitConverter.ToString(arrBytes));m_CachedStream.WriteTo(destination);//缓存流写入到发送流中ReferencePool.Release(packet);return true;}Log.Warning("Send packet invalid.");return false;
}

对于CSPacketBase类型
1.m_CachedStream是每个packet序列化的流，每次使用前需要设置position
2.先设置m_CachedStream.Position = PacketHeaderLength; 先跳过id，bodyLen位置，先写入body
3.protobuf序列化Serializer.Serialize(m_CachedStream, packet);后得到ushort bodyLen = (ushort)((int)m_CachedStream.Position - PacketHeaderLength);即为bodyLen长度
4.再设置m_CachedStream.Position = 0;，写入id字节流，bodyLen字节流
5.m_CachedStream.SetLength(PacketHeaderLength + bodyLen);写完后要设置长度截断，因为m_CachedStream是复用的，可能上次使用后面还有字节数据
6. m_CachedStream.WriteTo(destination);即为发送流，每次会添加到发送流的末尾
对于CSPacketLua类型
1.由于byte是在lua中序列化好的传递到C#的，只需要按照顺序写入到m_CachedStream中，其他流程与CSPacketBase一致

发送流

GameFramework.Network.NetworkManager.TcpWithSyncReceiveNetworkChannel.SendAsync

private void SendAsync()
{try{m_Socket.BeginSend(m_SendState.Stream.GetBuffer(), (int)m_SendState.Stream.Position, (int)(m_SendState.Stream.Length - m_SendState.Stream.Position), SocketFlags.None, m_SendCallback, m_Socket);}

每次从流中取position到length部分。每次发送一个完整流，从0开始。进入到发送回调
GameFramework.Network.NetworkManager.TcpWithSyncReceiveNetworkChannel.SendCallback

private void SendCallback(IAsyncResult ar)
{Socket socket = (Socket)ar.AsyncState;int bytesSent = 0;try{bytesSent = socket.EndSend(ar);}m_SendState.Stream.Position += bytesSent;if (m_SendState.Stream.Position < m_SendState.Stream.Length){SendAsync();return;}

发送了一段，设置流位置
如果位置<Length，接着调用发送，直到把流全部发送完毕

C#中接收协议

初始化时反射注册协议id对应type，协议id对应处理Handle

StarForce.NetworkChannelHelper.Initialize

// 反射注册包和包处理函数。
Type packetBaseType = typeof(SCPacketBase);
Type packetHandlerBaseType = typeof(PacketHandlerBase);
Assembly assembly = Assembly.GetExecutingAssembly();
Type[] types = assembly.GetTypes();
for (int i = 0; i < types.Length; i++)
{if (!types[i].IsClass || types[i].IsAbstract){continue;}if (types[i].BaseType == packetBaseType){//确定msgID反序列化的类结构PacketBase packetBase = (PacketBase)Activator.CreateInstance(types[i]);Type packetType = GetServerToClientPacketType(packetBase.Id);//防止监听的协议id重复if (packetType != null){Log.Warning("Already exist packet type '{0}', check '{1}' or '{2}'?.", packetBase.Id.ToString(), packetType.Name, packetBase.GetType().Name);continue;}m_ServerToClientPacketTypes.Add(packetBase.Id, types[i]);}else if (types[i].BaseType == packetHandlerBaseType){//网络事件handle处理IPacketHandler packetHandler = (IPacketHandler)Activator.CreateInstance(types[i]);m_NetworkChannel.RegisterHandler(packetHandler);}
}

异步接收流

连接成功，开始异步接收流
GameFramework.Network.NetworkManager.TcpNetworkChannel.ReceiveAsync

//每次获取到完整头/body。流会pos = 0，len为头长，或者body长
//每次拆包都是读取pos累积，剩余要读的为len-pos
m_Socket.BeginReceive(m_ReceiveState.Stream.GetBuffer(), (int)m_ReceiveState.Stream.Position, (int)(m_ReceiveState.Stream.Length - m_ReceiveState.Stream.Position), SocketFlags.None, m_ReceiveCallback, m_Socket);

有服务器下发协议，进入到接收回调中
GameFramework.Network.NetworkManager.TcpNetworkChannel.ReceiveCallback

private void ReceiveCallback(IAsyncResult ar)
{Socket socket = (Socket)ar.AsyncState;int bytesReceived = 0;try{bytesReceived = socket.EndReceive(ar);}//每次读取pos会累加m_ReceiveState.Stream.Position += bytesReceived;if (m_ReceiveState.Stream.Position < m_ReceiveState.Stream.Length){//未读满上次设定长度，接着读。上次设定长度为HeadLen，BodyLenReceiveAsync();return;}//开始解析，流要置为0位置开始解析m_ReceiveState.Stream.Position = 0L;bool processSuccess = false;if (m_ReceiveState.PacketHeader != null){//上次获取到头，这次反序列处理bodyprocessSuccess = ProcessPacket();m_ReceivedPacketCount++;}else{//未获取到头，反序列化头processSuccess = ProcessPacketHeader();}if (processSuccess){//处理成功，接着接收ReceiveAsync();return;}
}

1.初始化时，设置第一次流接收Length为6（协议id int+ bodyLen ushort）。即待接收包头
2.如果接收满了Length，进入到处理包头，解析出协议id，bodyLen
3.设置下一次接收为Length为bodyLen。即待接收包体
4.如果接收满Length，此时进入到处理包体，解析出body对应的对象。设置下次接收为包头Length6，循环到第一步
注意事项
如果有拆包黏包，在接收回调中处理，并且接满一个模式，再解析

    m_ReceiveState.Stream.Position += bytesReceived;if (m_ReceiveState.Stream.Position < m_ReceiveState.Stream.Length){//未读满上次设定长度，接着读。上次设定长度为HeadLen，BodyLenReceiveAsync();return;}

每次开始解析前，需要流postion = 0开始，因为随着接收，position到了末尾，无法解析

    //开始解析，流要置为0位置开始解析m_ReceiveState.Stream.Position = 0L;

解析包头

StarForce.NetworkChannelHelper.DeserializePacketHeader

source.Position = 0;
SCPacketHeader scHead = ReferencePool.Acquire<SCPacketHeader>();
source.Read(arrID, 0, 4);
scHead.Id = BitConverter.ToInt32(arrID,0);
source.Read(arrBodyLen, 0, 2);
scHead.PacketLength = BitConverter.ToUInt16(arrBodyLen, 0);

得到协议id，bodyLen

解析包体

StarForce.NetworkChannelHelper.DeserializePacket

Packet packet = null;
if (scPacketHeader.IsValid)
{Type packetType = GetServerToClientPacketType(scPacketHeader.Id);if (packetType != null){//source 为接收流，每次接收一整条消息前，设置了Lenpacket = (Packet)ReferencePool.Acquire(packetType);packet = (Packet)RuntimeTypeModel.Default.Deserialize(source, packet, packetType);}else{//如果id找不到字节流传递给Lua中反序列化为tableSCPacketLua luaPacket = ReferencePool.Acquire<SCPacketLua>();//引用池中使用，后续会在使用事件通知后，回到引用池luaPacket.m_id = scPacketHeader.Id;luaPacket.m_len = scPacketHeader.PacketLength;luaPacket.m_bytes = PacketBuffer.GetBuffer(scPacketHeader.PacketLength);source.Read(luaPacket.m_bytes.Buffer, 0, scPacketHeader.PacketLength);packet = luaPacket;

1.根据包体id（协议id），找到初始化反射注册的协议id，type
2.如果有，说明是C#用协议，protobuf反序列化为对象，加入到事件队列中，等待分发，这样做事为了从其他线程中转回主线程处理
3.如果不存在type，说明是Lua用协议，把字节流保存到SCPacketLua，传递到Lua处理字节流转为Lua中table

网络消息分发

GameFramework.EventPool.HandleEvent

if (m_EventHandlers.TryGetValue(e.Id, out range))
{LinkedListNode<EventHandler<T>> current = range.First;while (current != null && current != range.Terminal){m_CachedNodes[e] = current.Next != range.Terminal ? current.Next : null;if (m_NoSenderDict.ContainsKey(e.Id) && m_NoSenderDict[e.Id] == current.Value){current.Value(null, e);}else{//这里会传递网络handlecurrent.Value(sender, e);}current = m_CachedNodes[e];}m_CachedNodes.Remove(e);
}
else if (m_DefaultHandler != null)
{m_DefaultHandler(sender, e);
}
else if ((m_EventPoolMode & EventPoolMode.AllowNoHandler) == 0)
{noHandlerException = true;
}ReferencePool.Release(e);

1.C#中初始化时反射注册协议id对应handle，例如handle中内容

public class SCLoginHandler : PacketHandlerBase
{public override int Id{get{return (int)Network.NetMsgID.SCLogin;}}public override void Handle(object sender, Packet packet){SCLogin packetImpl = (SCLogin)packet;Log.Info("SCLoginHandler name:{0}-passeword{1}", packetImpl.account, packetImpl.password);}
}

会在current.Value(sender, e);中调用到 Handle，这里可以把协议转换好的对象，进一步处理
2.未找到协议id对应handle，执行m_DefaultHandler(sender, e);，这里可以在初始化设置委托在lua中执行，把SCPacketLua传递到Lua进一步处理

Lua中发送协议

lua中

function TestSendPlayerInfo()--每次协议都是全手写table，未看到可生成协议格式.lua文件，这样不可复用，每次都需要看proto描述，写全部字段。--需要在业务module中手写一遍全部协议send函数，确定发送的参数，组装成一个table再发送。这样相当于手动写了一遍local data = {name = "789",level = 123}LuaEntry.NetworkModule:Send(NetMsgID.CSPlayerInfo,data)
endfunction NetworkModule:Send(msgID,data)local sProto = MsgID2Proto[msgID]if sProto == nil thenLog.Info("消息ID：{0}找不到需要序列化的proto对象",msgID)returnendlocal bytes = assert(pb.encode(sProto, data))--返回值虽然为string，但是这是字节数组在lua中表达，可以直接传递给C#的byte[]local sHex = pb.tohex(bytes)Log.Info("Send Hex：{0}",sHex)GameEntry.Network:SendByLua(msgID,bytes) --有时候调用不到，生成一次wrap，再清除掉
end

1.lua中声明一个table为packet，里面字段为proto中描述
2.assert(pb.encode(sProto, data)，table序列化二进制数组，返回值虽然为string，但这是字节数组在lua中的表达，可以直接传递到c#的byte[]中https://www.jianshu.com/p/63987134c1ba
C#处理

public static void SendByLua(this NetworkComponent networkComponent, int msgID, byte[] bytes)
{CSPacketLua packet = ReferencePool.Acquire<CSPacketLua>();packet.m_id = msgID;packet.m_bytes = PacketBuffer.GetBufferAndCopyBytes(bytes,bytes.Length);packet.Len = (ushort)bytes.Length;networkComponent.Send<CSPacketLua>(packet);
}

Lua中接收协议

C#中注册网络委托

public static Action<SCPacketLua> CreateFunc = null;
m_NetworkChannel.SetDefaultHandler(LuaPacketHandler);

在网络消息分发时，未找打id对应handle，再进入到网络委托中处理
GameFramework.EventPool.HandleEvent

 m_DefaultHandler(sender, e);

Lua中执行网络委托内容

SF.NetworkChannelHelper.CreateFunc = handler(self,self.CreatePacket)function NetworkModule:CreatePacket(packet)local luaPacket = packetlocal sProto = MsgID2Proto[luaPacket.Id]if sProto == nil thenLog.Info("消息ID：{0}找不到需要序列化的proto对象",luaPacket.Id)returnendlocal data = assert(pb.decode(sProto, luaPacket.m_bytes.Buffer))PrintTable(data,false,true,"CreatePacket")
end

把C#中传递过来的SCPacketLua中的字节流使用lua-protobuf反序列化为table

流程图

GFxLuaProto发送协议流程图

GFxLuaProto接收协议流程图

遇到错误

字节流长度不对

ProtoBuf.ProtoException: Invalid wire-type; this usually means you have over-written a file without truncating or setting the length
使用字节流反序列化错误，检查长度之类
复用流，每次使用完要进行截断SetLength，否则会带入上次长度
流每次写入，都会改变position位置

lua-protobuf中反序列化，默认值问题

如果protobuf的成员值为默认值，序列化后会缺省这部分字节流。
lua中反序列化不出这个member。需要设置lua-protobuf中使用默认值

pb.option "use_default_values" --将默认值表复制到解码目标表中来

安卓测试

从C#发送，C#接收处打印

从Lua发送，Lua接收处打印