当前位置: 首页 > news >正文

WebRTC 系列(四、多人通话,H5、Android、iOS)

WebRTC 系列(三、点对点通话,H5、Android、iOS)

 上一篇博客中,我们已经实现了点对点通话,即一对一通话,这一次就接着实现多人通话。多人通话的实现方式呢也有好几种方案,这里我简单介绍两种方案。

一、多人通话方案

1.Mesh

多个客户端之间建立多个 PeerConnection,即如果有三个客户端 A、B、C,A 有两个 PeerConnection 分别与 B、C 通信,B 也是有两个 PeerConnection,分别与 A、C 通信,C 也是有两个 PeerConnection,分别与 A、B 通信,如图:

​​​​​​​​​​​​​​优点:服务端压力小,不需要对音视频数据做处理。
缺点:客户端编解码压力较大,传输的数据与通话人数成正比,兼容性较差。

2.Mixer

客户端只与服务器有一个 PeerConnection,有多个客户端时,服务端增加多个媒体流,由服务端来做媒体数据转发,如图:

优点:客户端只有一个连接,传输数据减少,服务端可对音视频数据预处理,兼容性好。
缺点:服务器压力大,通话人数过多时,服务器如果对音视频数据有预处理,可能导致通话延迟。

3.demo 方案选择

两种方案各有利弊,感觉在实际业务中,第二种方案更合适,毕竟把更多逻辑放在服务端更可控一点,我为了演示简单,就选用了第一种方案,下面就说说第一种方案的话,第一个人、第二个人、第三个人加入房间的流程是什么样的。

第一个人 A 加入房间:

  1. A 发送 join;
  2. 服务器向房间内其他所有人发送 otherJoin;
  3. 房间内没有其他人,结束。

第二个人 B 加入房间:

  1. B 发送 join;
  2. 服务器向房间内其他所有人发送 otherJoin;
  3. A 收到 otherJoin(带有 B 的 userId);
  4. A 检查远端视频渲染控件集合中是否存在 B 的 userId 对应的远端视频渲染控件,没有则创建并初始化,然后保存;
  5. A 检查 PeerConnection 集合中是否存在 B 的 userId 对应的 PeerConnection,没有则创建并添加音轨、视轨,然后保存;
  6. A 通过 PeerConnection 创建 offer,获取 sdp;
  7. A 将 offer sdp 作为参数 setLocalDescription;
  8. A 发送 offer sdp(带有 A 的 userId);
  9. B 收到 offer(带有 A 的 userId);
  10. B 检查远端视频渲染控件集合中是否存在 A 的 userId 对应远端视频渲染控件,没有则创建并初始化,然后保存;
  11. B 检查 PeerConnection 集合中是否存在 A 的 userId 对应的 PeerConnection,没有则创建并添加音轨、视轨,然后保存;
  12. B 将 offer sdp 作为参数 setRemoteDescription;
  13. B 通过 PeerConnection 创建 answer,获取 sdp;
  14. B 将 answer sdp 作为参数 setLocalDescription;
  15. B 发送 answer sdp(带有 B 的 userId);
  16. A 收到 answer sdp(带有 B 的 userId);
  17. A 通过 userId 找到对应 PeerConnection,将 answer sdp 作为参数 setRemoteDescription。

第三个人 C 加入房间:

  1. C 发送 join;
  2. 服务器向房间内其他所有人发送 otherJoin;
  3. A 收到 otherJoin(带有 C 的 userId);
  4. A 检查远端视频渲染控件集合中是否存在 C 的 userId 对应的远端视频渲染控件,没有则创建并初始化,然后保存;
  5. A 检查 PeerConnection 集合中是否存在 C 的 userId 对应的 PeerConnection,没有则创建并添加音轨、视轨,然后保存;
  6. A 通过 PeerConnection 创建 offer,获取 sdp;
  7. A 将 offer sdp 作为参数 setLocalDescription;
  8. A 发送 offer sdp(带有 A 的 userId);
  9. C 收到 offer(带有 A 的 userId);
  10. C 检查远端视频渲染控件集合中是否存在 A 的 userId 对应远端视频渲染控件,没有则创建并初始化,然后保存;
  11. C 检查 PeerConnection 集合中是否存在 A 的 userId 对应的 PeerConnection,没有则创建并添加音轨、视轨,然后保存;
  12. C 将 offer sdp 作为参数 setRemoteDescription;
  13. C 通过 PeerConnection 创建 answer,获取 sdp;
  14. C 将 answer sdp 作为参数 setLocalDescription;
  15. C 发送 answer sdp(带有 C 的 userId);
  16. A 收到 answer sdp(带有 C 的 userId);
  17. A 通过 userId 找到对应 PeerConnection,将 answer sdp 作为参数 setRemoteDescription。
  18. B 收到 otherJoin(带有 C 的 userId);
  19. B 检查远端视频渲染控件集合中是否存在 C 的 userId 对应的远端视频渲染控件,没有则创建并初始化,然后保存;
  20. B 检查 PeerConnection 集合中是否存在 C 的 userId 对应的 PeerConnection,没有则创建并添加音轨、视轨,然后保存;
  21. B 通过 PeerConnection 创建 offer,获取 sdp;
  22. B 将 offer sdp 作为参数 setLocalDescription;
  23. B 发送 offer sdp(带有 B 的 userId);
  24. C 收到 offer(带有 B 的 userId);
  25. C 检查远端视频渲染控件集合中是否存在 B 的 userId 对应远端视频渲染控件,没有则创建并初始化,然后保存;
  26. C 检查 PeerConnection 集合中是否存在 B 的 userId 对应的 PeerConnection,没有则创建并添加音轨、视轨,然后保存;
  27. C 将 offer sdp 作为参数 setRemoteDescription;
  28. C 通过 PeerConnection 创建 answer,获取 sdp;
  29. C 将 answer sdp 作为参数 setLocalDescription;
  30. C 发送 answer sdp(带有 C 的 userId);
  31. B 收到 answer sdp(带有 C 的 userId);
  32. B 通过 userId 找到对应 PeerConnection,将 answer sdp 作为参数 setRemoteDescription。

依此类推,如果还有第四个用户 D 再加入房间的话,D 也会发送 join,然后 A、B、C 也会类似上述 3~17 步处理。

这期间的 onIceCandidate 回调的处理和之前类似,只是将生成的 IceCandidate 对象传递给对方时需要带上发送方(自己)的 userId,便于对方找到对应的 PeerConnection,以及接收方的 userId,便于服务器找到接收方的长连接。

这期间的 onAddStream 回调的处理也和之前类似,只是需要通过对方的 userId 找到对应的远端控件渲染控件。

二、信令服务器

信令服务器的依赖就不重复了,根据上述流程,我们需要引入用户的概念,但暂时我没有引入房间的概念,所以在测试的时候我认为只有一个房间,所有人都加入的同一个房间。

多人通话 WebSocket 服务端代码:

package com.qinshou.webrtcdemo_server;import com.google.gson.Gson;
import com.google.gson.JsonObject;
import com.google.gson.reflect.TypeToken;import org.java_websocket.WebSocket;
import org.java_websocket.handshake.ClientHandshake;
import org.java_websocket.server.WebSocketServer;import java.net.InetSocketAddress;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;/*** Author: MrQinshou* Email: cqflqinhao@126.com* Date: 2023/2/8 9:33* Description: 多人通话 WebSocketServer*/
public class MultipleWebSocketServerHelper {public static class WebSocketBean {private String mUserId;private WebSocket mWebSocket;public WebSocketBean() {}public WebSocketBean(WebSocket webSocket) {mWebSocket = webSocket;}public String getUserId() {return mUserId;}public void setUserId(String userId) {mUserId = userId;}public WebSocket getWebSocket() {return mWebSocket;}public void setWebSocket(WebSocket webSocket) {mWebSocket = webSocket;}}private WebSocketServer mWebSocketServer;private final List<WebSocketBean> mWebSocketBeans = new LinkedList<>();//    private static final String HOST_NAME = "192.168.1.104";private static final String HOST_NAME = "172.16.2.172";private static final int PORT = 8888;private WebSocketBean getWebSocketBeanByWebSocket(WebSocket webSocket) {for (WebSocketBean webSocketBean : mWebSocketBeans) {if (webSocket == webSocketBean.getWebSocket()) {return webSocketBean;}}return null;}private WebSocketBean getWebSocketBeanByUserId(String userId) {for (WebSocketBean webSocketBean : mWebSocketBeans) {if (userId.equals(webSocketBean.getUserId())) {return webSocketBean;}}return null;}private WebSocketBean removeWebSocketBeanByWebSocket(WebSocket webSocket) {for (WebSocketBean webSocketBean : mWebSocketBeans) {if (webSocket == webSocketBean.getWebSocket()) {mWebSocketBeans.remove(webSocketBean);return webSocketBean;}}return null;}public void start() {InetSocketAddress inetSocketAddress = new InetSocketAddress(HOST_NAME, PORT);mWebSocketServer = new WebSocketServer(inetSocketAddress) {@Overridepublic void onOpen(WebSocket conn, ClientHandshake handshake) {System.out.println("onOpen--->" + conn);// 有客户端连接,创建 WebSocketBean,此时仅保存了 WebSocket 连接,但还没有和 userId 绑定mWebSocketBeans.add(new WebSocketBean(conn));}@Overridepublic void onClose(WebSocket conn, int code, String reason, boolean remote) {System.out.println("onClose--->" + conn);WebSocketBean webSocketBean = removeWebSocketBeanByWebSocket(conn);if (webSocketBean == null) {return;}// 通知其他用户有人退出房间JsonObject jsonObject = new JsonObject();jsonObject.addProperty("msgType", "otherQuit");jsonObject.addProperty("userId", webSocketBean.mUserId);for (WebSocketBean w : mWebSocketBeans) {if (w != webSocketBean) {w.mWebSocket.send(jsonObject.toString());}}}@Overridepublic void onMessage(WebSocket conn, String message) {System.out.println("onMessage--->" + message);Map<String, String> map = new Gson().fromJson(message, new TypeToken<Map<String, String>>() {}.getType());String msgType = map.get("msgType");if ("join".equals(msgType)) {// 收到加入房间指令String userId = map.get("userId");WebSocketBean webSocketBean = getWebSocketBeanByWebSocket(conn);// WebSocket 连接绑定 userIdif (webSocketBean != null) {webSocketBean.setUserId(userId);}// 通知其他用户有其他人加入房间JsonObject jsonObject = new JsonObject();jsonObject.addProperty("msgType", "otherJoin");jsonObject.addProperty("userId", userId);for (WebSocketBean w : mWebSocketBeans) {if (w != webSocketBean && w.getUserId() != null) {w.mWebSocket.send(jsonObject.toString());}}return;}if ("quit".equals(msgType)) {// 收到退出房间指令String userId = map.get("userId");WebSocketBean webSocketBean = getWebSocketBeanByWebSocket(conn);// WebSocket 连接解绑 userIdif (webSocketBean != null) {webSocketBean.setUserId(null);}// 通知其他用户有其他人退出房间JsonObject jsonObject = new JsonObject();jsonObject.addProperty("msgType", "otherQuit");jsonObject.addProperty("userId", userId);for (WebSocketBean w : mWebSocketBeans) {if (w != webSocketBean && w.getUserId() != null) {w.mWebSocket.send(jsonObject.toString());}}return;}// 其他消息透传// 接收方String toUserId = map.get("toUserId");// 找到接收方对应 WebSocket 连接WebSocketBean webSocketBean = getWebSocketBeanByUserId(toUserId);if (webSocketBean != null) {webSocketBean.getWebSocket().send(message);}}@Overridepublic void onError(WebSocket conn, Exception ex) {ex.printStackTrace();System.out.println("onError");}@Overridepublic void onStart() {System.out.println("onStart");}};mWebSocketServer.start();}public void stop() {if (mWebSocketServer == null) {return;}for (WebSocket webSocket : mWebSocketServer.getConnections()) {webSocket.close();}try {mWebSocketServer.stop();} catch (InterruptedException e) {throw new RuntimeException(e);}mWebSocketServer = null;}public static void main(String[] args) {new MultipleWebSocketServerHelper().start();}
}

三、消息格式

传递的消息的话,相较于点对点通话,sdp 和 iceCandidate 中需要添加 fromUserId 和 toUserId 字段,另外还需要增加 join、otherJoin、quit、ohterQuit 消息:

// sdp
{"msgType": "sdp","fromUserId": userId,"toUserId": toUserId,"type": sessionDescription.type,"sdp": sessionDescription.sdp
}// iceCandidate
{"msgType": "iceCandidate","fromUserId": userId,"toUserId": toUserId,"id": iceCandidate.sdpMid,"label": iceCandidate.sdpMLineIndex,"candidate": iceCandidate.candidate
}// join
{"msgType": "join""userId": userId
}// otherJoin
{"msgType": "otherJoin""userId": userId
}// quit
{"msgType": "quit""userId": userId
}// otherQuit
{"msgType": "otherQuit""userId": userId
}

四、H5

代码与 p2p_demo 其实差不了太多,但是我们创建 PeerConnection 的时机需要根据上面梳理流程进行修改,发送的信令也需要根据上面定义的格式进行修改,布局中将远端视频渲染控件去掉,改成一个远端视频渲染控件的容器,每当有新的连接时创建新的远端视频渲染控件放到容器中,另外,WebSocket 需要额外处理 otherJoin 和 otherQuit 信令。

1.添加依赖

这个跟前两篇的一样,不需要额外引入。

2.multiple_demo.html

<html><head><title>Multiple Demo</title><style>body {overflow: hidden;margin: 0px;padding: 0px;}#local_view {width: 100%;height: 100%;}#remote_views {width: 9%;height: 80%;position: absolute;top: 10%;right: 10%;bottom: 10%;overflow-y: auto;}.remote_view {width: 100%;aspect-ratio: 9/16;}#left {width: 10%;height: 5%;position: absolute;left: 10%;top: 10%;}#p_websocket_state,#input_server_url,.my_button {width: 100%;height: 100%;display: block;margin-bottom: 10%;}</style>
</head><body><video id="local_view" width="480" height="270" autoplay controls muted></video><div id="remote_views"></div><div id="left"><p id="p_websocket_state">WebSocket 已断开</p><input id="input_server_url" type="text" placeholder="请输入服务器地址" value="ws://192.168.1.104:8888"></input><button id="btn_connect" class="my_button" onclick="connect()">连接 WebSocket</button><button id="btn_disconnect" class="my_button" onclick="disconnect()">断开 WebSocket</button><button id="btn_join" class="my_button" onclick="join()">加入房间</button><button id="btn_quit" class="my_button" onclick="quit()">退出房间</button></div>
</body><script type="text/javascript">/*** Author: MrQinshou* Email: cqflqinhao@126.com* Date: 2023/4/15 11:24* Description: 生成 uuid*/function uuid() {return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function (c) {var r = Math.random() * 16 | 0;var v = c == 'x' ? r : (r & 0x3 | 0x8);return v.toString(16);});}
</script><script type="text/javascript">var localView = document.getElementById("local_view");var remoteViews = document.getElementById("remote_views");var localStream;// let userId = uuid();let userId = "h5";let peerConnectionDict = {};let remoteViewDict = {};function createPeerConnection(fromUserId) {let peerConnection = new RTCPeerConnection();peerConnection.oniceconnectionstatechange = function (event) {if ("disconnected" == event.target.iceConnectionState) {let peerConnection = peerConnectionDict[fromUserId];if (peerConnection != null) {peerConnection.close();delete peerConnectionDict[fromUserId];}let remoteView = remoteViewDict[fromUserId];if (remoteView != null) {remoteView.removeAttribute('src');remoteView.load();remoteView.remove();delete remoteViewDict[fromUserId];}}}peerConnection.onicecandidate = function (event) {console.log("onicecandidate--->" + event.candidate);sendIceCandidate(event.candidate, fromUserId);}peerConnection.ontrack = function (event) {console.log("remote ontrack--->" + event.streams);let remoteView = remoteViewDict[fromUserId];if (remoteView == null) {return;}let streams = event.streams;if (streams && streams.length > 0) {remoteView.srcObject = streams[0];}}return peerConnection;}function createOffer(peerConnection, fromUserId) {peerConnection.createOffer().then(function (sessionDescription) {console.log(fromUserId + " create offer success.");peerConnection.setLocalDescription(sessionDescription).then(function () {console.log(fromUserId + " set local sdp success.");var jsonObject = {"msgType": "sdp","fromUserId": userId,"toUserId": fromUserId,"type": "offer","sdp": sessionDescription.sdp};send(JSON.stringify(jsonObject));}).catch(function (error) {console.log("error--->" + error);})}).catch(function (error) {console.log("error--->" + error);})}function createAnswer(peerConnection, fromUserId) {peerConnection.createAnswer().then(function (sessionDescription) {console.log(fromUserId + " create answer success.");peerConnection.setLocalDescription(sessionDescription).then(function () {console.log(fromUserId + " set local sdp success.");var jsonObject = {"msgType": "sdp","fromUserId": userId,"toUserId": fromUserId,"type": "answer","sdp": sessionDescription.sdp};send(JSON.stringify(jsonObject));}).catch(function (error) {console.log("error--->" + error);})}).catch(function (error) {console.log("error--->" + error);})}function join() {var jsonObject = {"msgType": "join","userId": userId,};send(JSON.stringify(jsonObject));}function quit() {var jsonObject = {"msgType": "quit","userId": userId,};send(JSON.stringify(jsonObject));for (var key in peerConnectionDict) {let peerConnection = peerConnectionDict[key];peerConnection.close();delete peerConnectionDict[key];}for (var key in remoteViewDict) {let remoteView = remoteViewDict[key];remoteView.removeAttribute('src');remoteView.load();remoteView.remove();delete remoteViewDict[key];}}function sendOffer(offer, toUserId) {var jsonObject = {"msgType": "sdp","fromUserId": userId,"toUserId": toUserId,"type": "offer","sdp": offer.sdp};send(JSON.stringify(jsonObject));}function receivedOffer(jsonObject) {let fromUserId = jsonObject["fromUserId"];var peerConnection = peerConnectionDict[fromUserId];if (peerConnection == null) {// 创建 PeerConnectionpeerConnection = createPeerConnection(fromUserId);// 为 PeerConnection 添加音轨、视轨for (let i = 0; localStream != null && i < localStream.getTracks().length; i++) {const track = localStream.getTracks()[i];peerConnection.addTrack(track, localStream);}peerConnectionDict[fromUserId] = peerConnection;}var remoteView = remoteViewDict[fromUserId];if (remoteView == null) {remoteView = document.createElement("video");remoteView.className = "remote_view";remoteView.autoplay = true;remoteView.control = true;remoteView.muted = true;remoteViews.appendChild(remoteView);remoteViewDict[fromUserId] = remoteView;}let options = {"type": jsonObject["type"],"sdp": jsonObject["sdp"]}// 将 offer sdp 作为参数 setRemoteDescriptionlet sessionDescription = new RTCSessionDescription(options);peerConnection.setRemoteDescription(sessionDescription).then(function () {console.log(fromUserId + " set remote sdp success.");// 通过 PeerConnection 创建 answer,获取 sdppeerConnection.createAnswer().then(function (sessionDescription) {console.log(fromUserId + " create answer success.");// 将 answer sdp 作为参数 setLocalDescriptionpeerConnection.setLocalDescription(sessionDescription).then(function () {console.log(fromUserId + " set local sdp success.");// 发送 answer sdpsendAnswer(sessionDescription, fromUserId);})})}).catch(function (error) {console.log("error--->" + error);});}function sendAnswer(answer, toUserId) {var jsonObject = {"msgType": "sdp","fromUserId": userId,"toUserId": toUserId,"type": "answer","sdp": answer.sdp};send(JSON.stringify(jsonObject));}function receivedAnswer(jsonObject) {let fromUserId = jsonObject["fromUserId"];var peerConnection = peerConnectionDict[fromUserId];if (peerConnection == null) {// 创建 PeerConnectionpeerConnection = createPeerConnection(fromUserId);// 为 PeerConnection 添加音轨、视轨for (let i = 0; localStream != null && i < localStream.getTracks().length; i++) {const track = localStream.getTracks()[i];peerConnection.addTrack(track, localStream);}peerConnectionDict[fromUserId] = peerConnection;}var remoteView = remoteViewDict[fromUserId];if (remoteView == null) {remoteView = document.createElement("video");remoteView.className = "remote_view";remoteView.autoplay = true;remoteView.control = true;remoteView.muted = true;remoteViews.appendChild(remoteView);remoteViewDict[fromUserId] = remoteView;}let options = {"type": jsonObject["type"],"sdp": jsonObject["sdp"]}let sessionDescription = new RTCSessionDescription(options);let type = jsonObject["type"];peerConnection.setRemoteDescription(sessionDescription).then(function () {console.log(fromUserId + " set remote sdp success.");}).catch(function (error) {console.log("error--->" + error);});}function sendIceCandidate(iceCandidate, toUserId) {if (iceCandidate == null) {return;}var jsonObject = {"msgType": "iceCandidate","fromUserId": userId,"toUserId": toUserId,"id": iceCandidate.sdpMid,"label": iceCandidate.sdpMLineIndex,"candidate": iceCandidate.candidate};send(JSON.stringify(jsonObject));}function receivedCandidate(jsonObject) {let fromUserId = jsonObject["fromUserId"];let peerConnection = peerConnectionDict[fromUserId];if (peerConnection == null) {return}let options = {"sdpMLineIndex": jsonObject["label"],"sdpMid": jsonObject["id"],"candidate": jsonObject["candidate"]}let iceCandidate = new RTCIceCandidate(options);peerConnection.addIceCandidate(iceCandidate);}function receivedOtherJoin(jsonObject) {// 创建 PeerConnectionlet userId = jsonObject["userId"];var peerConnection = peerConnectionDict[userId];if (peerConnection == null) {peerConnection = createPeerConnection(userId);for (let i = 0; localStream != null && i < localStream.getTracks().length; i++) {const track = localStream.getTracks()[i];peerConnection.addTrack(track, localStream);}peerConnectionDict[userId] = peerConnection;}var remoteView = remoteViewDict[userId];if (remoteView == null) {remoteView = document.createElement("video");remoteView.className = "remote_view";remoteView.autoplay = true;remoteView.control = true;remoteView.muted = true;remoteViews.appendChild(remoteView);remoteViewDict[userId] = remoteView;}// 通过 PeerConnection 创建 offer,获取 sdppeerConnection.createOffer().then(function (sessionDescription) {console.log(userId + " create offer success.");// 将 offer sdp 作为参数 setLocalDescriptionpeerConnection.setLocalDescription(sessionDescription).then(function () {console.log(userId + " set local sdp success.");// 发送 offer sdpsendOffer(sessionDescription, userId);}).catch(function (error) {console.log("error--->" + error);})}).catch(function (error) {console.log("error--->" + error);});}function receivedOtherQuit(jsonObject) {let userId = jsonObject["userId"];let peerConnection = peerConnectionDict[userId];if (peerConnection != null) {peerConnection.close();delete peerConnectionDict[userId];}let remoteView = remoteViewDict[userId];if (remoteView != null) {remoteView.removeAttribute('src');remoteView.load();remoteView.remove();delete remoteViewDict[userId];}}navigator.mediaDevices.getUserMedia({ audio: true, video: true }).then(function (mediaStream) {// 初始化 PeerConnectionFactory;// 创建 EglBase;// 创建 PeerConnectionFactory;// 创建音轨;// 创建视轨;localStream = mediaStream;// 初始化本地视频渲染控件;// 初始化远端视频渲染控件;// 开始本地渲染。localView.srcObject = mediaStream;}).catch(function (error) {console.log("error--->" + error);})
</script><script type="text/javascript">var websocket;function connect() {let inputServerUrl = document.getElementById("input_server_url");let pWebsocketState = document.getElementById("p_websocket_state");let url = inputServerUrl.value;websocket = new WebSocket(url);websocket.onopen = function () {console.log("onOpen");pWebsocketState.innerText = "WebSocket 已连接";}websocket.onmessage = function (message) {console.log("onmessage--->" + message.data);let jsonObject = JSON.parse(message.data);let msgType = jsonObject["msgType"];if ("sdp" == msgType) {let type = jsonObject["type"];if ("offer" == type) {receivedOffer(jsonObject);} else if ("answer" == type) {receivedAnswer(jsonObject);}} else if ("iceCandidate" == msgType) {receivedCandidate(jsonObject);} else if ("otherJoin" == msgType) {receivedOtherJoin(jsonObject);} else if ("otherQuit" == msgType) {receivedOtherQuit(jsonObject);}}websocket.onclose = function (error) {console.log("onclose--->" + error);pWebsocketState.innerText = "WebSocket 已断开";}websocket.onerror = function (error) {console.log("onerror--->" + error);}}function disconnect() {websocket.close();}function send(message) {if (!websocket) {return;}websocket.send(message);}</script></html>

多人通话至少需要三个端,我们就等所有端都实现了再最后来看效果。

五、Android

1.添加依赖

这个跟前两篇的一样,不需要额外引入。

2.布局

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"xmlns:app="http://schemas.android.com/apk/res-auto"xmlns:tools="http://schemas.android.com/tools"android:layout_width="match_parent"android:layout_height="match_parent"android:background="#FF000000"android:keepScreenOn="true"tools:context=".P2PDemoActivity"><org.webrtc.SurfaceViewRendererandroid:id="@+id/svr_local"android:layout_width="match_parent"android:layout_height="0dp"app:layout_constraintBottom_toBottomOf="parent"app:layout_constraintDimensionRatio="9:16"app:layout_constraintEnd_toEndOf="parent"app:layout_constraintStart_toStartOf="parent"app:layout_constraintTop_toTopOf="parent" /><androidx.core.widget.NestedScrollViewandroid:layout_width="90dp"android:layout_height="wrap_content"android:layout_marginTop="30dp"android:layout_marginEnd="30dp"android:layout_marginBottom="30dp"app:layout_constraintEnd_toEndOf="parent"app:layout_constraintTop_toTopOf="parent"><androidx.appcompat.widget.LinearLayoutCompatandroid:id="@+id/ll_remotes"android:layout_width="match_parent"android:layout_height="wrap_content"android:orientation="vertical"></androidx.appcompat.widget.LinearLayoutCompat></androidx.core.widget.NestedScrollView><androidx.appcompat.widget.LinearLayoutCompatandroid:layout_width="match_parent"android:layout_height="wrap_content"android:layout_marginStart="30dp"android:layout_marginTop="30dp"android:layout_marginEnd="30dp"android:orientation="vertical"app:layout_constraintStart_toStartOf="parent"app:layout_constraintTop_toTopOf="parent"><androidx.appcompat.widget.AppCompatTextViewandroid:id="@+id/tv_websocket_state"android:layout_width="match_parent"android:layout_height="wrap_content"android:text="WebSocket 已断开"android:textColor="#FFFFFFFF" /><androidx.appcompat.widget.AppCompatEditTextandroid:id="@+id/et_server_url"android:layout_width="match_parent"android:layout_height="wrap_content"android:hint="请输入服务器地址"android:textColor="#FFFFFFFF"android:textColorHint="#FFFFFFFF" /><androidx.appcompat.widget.AppCompatButtonandroid:id="@+id/btn_connect"android:layout_width="wrap_content"android:layout_height="wrap_content"android:text="连接 WebSocket"android:textAllCaps="false" /><androidx.appcompat.widget.AppCompatButtonandroid:id="@+id/btn_disconnect"android:layout_width="wrap_content"android:layout_height="wrap_content"android:text="断开 WebSocket"android:textAllCaps="false" /><androidx.appcompat.widget.AppCompatButtonandroid:id="@+id/btn_join"android:layout_width="wrap_content"android:layout_height="wrap_content"android:text="加入房间"android:textSize="12sp" /><androidx.appcompat.widget.AppCompatButtonandroid:id="@+id/btn_quit"android:layout_width="wrap_content"android:layout_height="wrap_content"android:text="退出房间"android:textSize="12sp" /></androidx.appcompat.widget.LinearLayoutCompat>
</androidx.constraintlayout.widget.ConstraintLayout>

布局中将远端视频渲染控件去掉,改成一个远端视频渲染控件的容器,每当有新的连接时创建新的远端视频渲染控件放到容器中。

3.MultipleDemoActivity.java

package com.qinshou.webrtcdemo_android;import android.content.Context;
import android.os.Bundle;
import android.text.TextUtils;
import android.view.View;
import android.view.ViewGroup;
import android.widget.EditText;
import android.widget.LinearLayout;
import android.widget.TextView;import androidx.appcompat.app.AppCompatActivity;
import androidx.appcompat.widget.LinearLayoutCompat;import org.json.JSONException;
import org.json.JSONObject;
import org.webrtc.AudioSource;
import org.webrtc.AudioTrack;
import org.webrtc.Camera2Capturer;
import org.webrtc.Camera2Enumerator;
import org.webrtc.CameraEnumerator;
import org.webrtc.DataChannel;
import org.webrtc.DefaultVideoDecoderFactory;
import org.webrtc.DefaultVideoEncoderFactory;
import org.webrtc.EglBase;
import org.webrtc.IceCandidate;
import org.webrtc.MediaConstraints;
import org.webrtc.MediaStream;
import org.webrtc.PeerConnection;
import org.webrtc.PeerConnectionFactory;
import org.webrtc.RtpReceiver;
import org.webrtc.SessionDescription;
import org.webrtc.SurfaceTextureHelper;
import org.webrtc.SurfaceViewRenderer;
import org.webrtc.VideoCapturer;
import org.webrtc.VideoDecoderFactory;
import org.webrtc.VideoEncoderFactory;
import org.webrtc.VideoSource;
import org.webrtc.VideoTrack;import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;/*** Author: MrQinshou* Email: cqflqinhao@126.com* Date: 2023/3/21 17:22* Description: P2P demo*/
public class MultipleDemoActivity extends AppCompatActivity {private static final String TAG = MultipleDemoActivity.class.getSimpleName();private static final String AUDIO_TRACK_ID = "ARDAMSa0";private static final String VIDEO_TRACK_ID = "ARDAMSv0";private static final List<String> STREAM_IDS = new ArrayList<String>() {{add("ARDAMS");}};private static final String SURFACE_TEXTURE_HELPER_THREAD_NAME = "SurfaceTextureHelperThread";private static final int WIDTH = 1280;private static final int HEIGHT = 720;private static final int FPS = 30;private EglBase mEglBase;private PeerConnectionFactory mPeerConnectionFactory;private VideoCapturer mVideoCapturer;private AudioTrack mAudioTrack;private VideoTrack mVideoTrack;private WebSocketClientHelper mWebSocketClientHelper = new WebSocketClientHelper();
//    private String mUserId = UUID.randomUUID().toString();private String mUserId = "Android";private final Map<String, PeerConnection> mPeerConnectionMap = new ConcurrentHashMap<>();private final Map<String, SurfaceViewRenderer> mRemoteViewMap = new ConcurrentHashMap<>();@Overrideprotected void onCreate(Bundle savedInstanceState) {super.onCreate(savedInstanceState);setContentView(R.layout.activity_multiple_demo);((EditText) findViewById(R.id.et_server_url)).setText("ws://192.168.1.104:8888");findViewById(R.id.btn_connect).setOnClickListener(new View.OnClickListener() {@Overridepublic void onClick(View view) {String url = ((EditText) findViewById(R.id.et_server_url)).getText().toString().trim();mWebSocketClientHelper.connect(url);}});findViewById(R.id.btn_disconnect).setOnClickListener(new View.OnClickListener() {@Overridepublic void onClick(View view) {mWebSocketClientHelper.disconnect();}});findViewById(R.id.btn_join).setOnClickListener(new View.OnClickListener() {@Overridepublic void onClick(View view) {join();}});findViewById(R.id.btn_quit).setOnClickListener(new View.OnClickListener() {@Overridepublic void onClick(View view) {quit();}});mWebSocketClientHelper.setOnWebSocketListener(new WebSocketClientHelper.OnWebSocketClientListener() {@Overridepublic void onOpen() {runOnUiThread(new Runnable() {@Overridepublic void run() {((TextView) findViewById(R.id.tv_websocket_state)).setText("WebSocket 已连接");}});}@Overridepublic void onClose() {runOnUiThread(new Runnable() {@Overridepublic void run() {((TextView) findViewById(R.id.tv_websocket_state)).setText("WebSocket 已断开");}});}@Overridepublic void onMessage(String message) {ShowLogUtil.debug("message--->" + message);try {JSONObject jsonObject = new JSONObject(message);String msgType = jsonObject.optString("msgType");if (TextUtils.equals("sdp", msgType)) {String type = jsonObject.optString("type");if (TextUtils.equals("offer", type)) {receivedOffer(jsonObject);} else if (TextUtils.equals("answer", type)) {receivedAnswer(jsonObject);}} else if (TextUtils.equals("iceCandidate", msgType)) {receivedCandidate(jsonObject);} else if (TextUtils.equals("otherJoin", msgType)) {receivedOtherJoin(jsonObject);} else if (TextUtils.equals("otherQuit", msgType)) {receivedOtherQuit(jsonObject);}} catch (JSONException e) {e.printStackTrace();}}});// 初始化 PeerConnectionFactoryinitPeerConnectionFactory(MultipleDemoActivity.this);// 创建 EglBasemEglBase = EglBase.create();// 创建 PeerConnectionFactorymPeerConnectionFactory = createPeerConnectionFactory(mEglBase);// 创建音轨mAudioTrack = createAudioTrack(mPeerConnectionFactory);// 创建视轨mVideoCapturer = createVideoCapturer();VideoSource videoSource = createVideoSource(mPeerConnectionFactory, mVideoCapturer);mVideoTrack = createVideoTrack(mPeerConnectionFactory, videoSource);// 初始化本地视频渲染控件,这个方法非常重要,不初始化会黑屏SurfaceViewRenderer svrLocal = findViewById(R.id.svr_local);svrLocal.init(mEglBase.getEglBaseContext(), null);mVideoTrack.addSink(svrLocal);// 开始本地渲染// 创建 SurfaceTextureHelper,用来表示 camera 初始化的线程SurfaceTextureHelper surfaceTextureHelper = SurfaceTextureHelper.create(SURFACE_TEXTURE_HELPER_THREAD_NAME, mEglBase.getEglBaseContext());// 初始化视频采集器mVideoCapturer.initialize(surfaceTextureHelper, MultipleDemoActivity.this, videoSource.getCapturerObserver());mVideoCapturer.startCapture(WIDTH, HEIGHT, FPS);}@Overrideprotected void onDestroy() {super.onDestroy();if (mEglBase != null) {mEglBase.release();mEglBase = null;}if (mVideoCapturer != null) {try {mVideoCapturer.stopCapture();} catch (InterruptedException e) {e.printStackTrace();}mVideoCapturer.dispose();mVideoCapturer = null;}if (mAudioTrack != null) {mAudioTrack.dispose();mAudioTrack = null;}if (mVideoTrack != null) {mVideoTrack.dispose();mVideoTrack = null;}for (PeerConnection peerConnection : mPeerConnectionMap.values()) {peerConnection.close();peerConnection.dispose();}mPeerConnectionMap.clear();SurfaceViewRenderer svrLocal = findViewById(R.id.svr_local);svrLocal.release();for (SurfaceViewRenderer surfaceViewRenderer : mRemoteViewMap.values()) {surfaceViewRenderer.release();}mRemoteViewMap.clear();mWebSocketClientHelper.disconnect();}private void initPeerConnectionFactory(Context context) {PeerConnectionFactory.initialize(PeerConnectionFactory.InitializationOptions.builder(context).createInitializationOptions());}private PeerConnectionFactory createPeerConnectionFactory(EglBase eglBase) {VideoEncoderFactory videoEncoderFactory = new DefaultVideoEncoderFactory(eglBase.getEglBaseContext(), true, true);VideoDecoderFactory videoDecoderFactory = new DefaultVideoDecoderFactory(eglBase.getEglBaseContext());return PeerConnectionFactory.builder().setVideoEncoderFactory(videoEncoderFactory).setVideoDecoderFactory(videoDecoderFactory).createPeerConnectionFactory();}private AudioTrack createAudioTrack(PeerConnectionFactory peerConnectionFactory) {AudioSource audioSource = peerConnectionFactory.createAudioSource(new MediaConstraints());AudioTrack audioTrack = peerConnectionFactory.createAudioTrack(AUDIO_TRACK_ID, audioSource);audioTrack.setEnabled(true);return audioTrack;}private VideoCapturer createVideoCapturer() {VideoCapturer videoCapturer = null;CameraEnumerator cameraEnumerator = new Camera2Enumerator(MultipleDemoActivity.this);for (String deviceName : cameraEnumerator.getDeviceNames()) {// 前摄像头if (cameraEnumerator.isFrontFacing(deviceName)) {videoCapturer = new Camera2Capturer(MultipleDemoActivity.this, deviceName, null);}}return videoCapturer;}private VideoSource createVideoSource(PeerConnectionFactory peerConnectionFactory, VideoCapturer videoCapturer) {// 创建视频源VideoSource videoSource = peerConnectionFactory.createVideoSource(videoCapturer.isScreencast());return videoSource;}private VideoTrack createVideoTrack(PeerConnectionFactory peerConnectionFactory, VideoSource videoSource) {// 创建视轨VideoTrack videoTrack = peerConnectionFactory.createVideoTrack(VIDEO_TRACK_ID, videoSource);videoTrack.setEnabled(true);return videoTrack;}private PeerConnection createPeerConnection(PeerConnectionFactory peerConnectionFactory, String fromUserId) {// 内部会转成 RTCConfigurationList<PeerConnection.IceServer> iceServers = new ArrayList<>();PeerConnection peerConnection = peerConnectionFactory.createPeerConnection(iceServers, new PeerConnection.Observer() {@Overridepublic void onSignalingChange(PeerConnection.SignalingState signalingState) {}@Overridepublic void onIceConnectionChange(PeerConnection.IceConnectionState iceConnectionState) {ShowLogUtil.debug("onIceConnectionChange--->" + iceConnectionState);if (iceConnectionState == PeerConnection.IceConnectionState.DISCONNECTED) {PeerConnection peerConnection = mPeerConnectionMap.get(fromUserId);ShowLogUtil.debug("peerConnection--->" + peerConnection);if (peerConnection != null) {peerConnection.close();mPeerConnectionMap.remove(fromUserId);}runOnUiThread(new Runnable() {@Overridepublic void run() {SurfaceViewRenderer surfaceViewRenderer = mRemoteViewMap.get(fromUserId);if (surfaceViewRenderer != null) {((ViewGroup) surfaceViewRenderer.getParent()).removeView(surfaceViewRenderer);mRemoteViewMap.remove(fromUserId);}}});}}@Overridepublic void onIceConnectionReceivingChange(boolean b) {}@Overridepublic void onIceGatheringChange(PeerConnection.IceGatheringState iceGatheringState) {}@Overridepublic void onIceCandidate(IceCandidate iceCandidate) {ShowLogUtil.verbose("onIceCandidate--->" + iceCandidate);sendIceCandidate(iceCandidate, fromUserId);}@Overridepublic void onIceCandidatesRemoved(IceCandidate[] iceCandidates) {}@Overridepublic void onAddStream(MediaStream mediaStream) {ShowLogUtil.verbose("onAddStream--->" + mediaStream);if (mediaStream == null || mediaStream.videoTracks == null || mediaStream.videoTracks.isEmpty()) {return;}runOnUiThread(new Runnable() {@Overridepublic void run() {SurfaceViewRenderer surfaceViewRenderer = mRemoteViewMap.get(fromUserId);if (surfaceViewRenderer != null) {mediaStream.videoTracks.get(0).addSink(surfaceViewRenderer);}}});}@Overridepublic void onRemoveStream(MediaStream mediaStream) {}@Overridepublic void onDataChannel(DataChannel dataChannel) {}@Overridepublic void onRenegotiationNeeded() {}@Overridepublic void onAddTrack(RtpReceiver rtpReceiver, MediaStream[] mediaStreams) {}});return peerConnection;}private void join() {try {JSONObject jsonObject = new JSONObject();jsonObject.put("msgType", "join");jsonObject.put("userId", mUserId);mWebSocketClientHelper.send(jsonObject.toString());} catch (JSONException e) {e.printStackTrace();}}private void quit() {try {JSONObject jsonObject = new JSONObject();jsonObject.put("msgType", "quit");jsonObject.put("userId", mUserId);mWebSocketClientHelper.send(jsonObject.toString());} catch (JSONException e) {e.printStackTrace();}new Thread(new Runnable() {@Overridepublic void run() {for (PeerConnection peerConnection : mPeerConnectionMap.values()) {peerConnection.close();}mPeerConnectionMap.clear();}}).start();for (SurfaceViewRenderer surfaceViewRenderer : mRemoteViewMap.values()) {((ViewGroup) surfaceViewRenderer.getParent()).removeView(surfaceViewRenderer);}mRemoteViewMap.clear();}private void sendOffer(SessionDescription offer, String toUserId) {try {JSONObject jsonObject = new JSONObject();jsonObject.put("msgType", "sdp");jsonObject.put("fromUserId", mUserId);jsonObject.put("toUserId", toUserId);jsonObject.put("type", "offer");jsonObject.put("sdp", offer.description);mWebSocketClientHelper.send(jsonObject.toString());} catch (JSONException e) {e.printStackTrace();}}private void receivedOffer(JSONObject jsonObject) {String fromUserId = jsonObject.optString("fromUserId");PeerConnection peerConnection = mPeerConnectionMap.get(fromUserId);if (peerConnection == null) {// 创建 PeerConnectionpeerConnection = createPeerConnection(mPeerConnectionFactory, fromUserId);// 为 PeerConnection 添加音轨、视轨peerConnection.addTrack(mAudioTrack, STREAM_IDS);peerConnection.addTrack(mVideoTrack, STREAM_IDS);mPeerConnectionMap.put(fromUserId, peerConnection);}runOnUiThread(new Runnable() {@Overridepublic void run() {SurfaceViewRenderer surfaceViewRenderer = mRemoteViewMap.get(fromUserId);if (surfaceViewRenderer == null) {// 初始化 SurfaceViewRender ,这个方法非常重要,不初始化黑屏surfaceViewRenderer = new SurfaceViewRenderer(MultipleDemoActivity.this);surfaceViewRenderer.init(mEglBase.getEglBaseContext(), null);surfaceViewRenderer.setLayoutParams(new LinearLayout.LayoutParams(dp2px(MultipleDemoActivity.this, 90), dp2px(MultipleDemoActivity.this, 160)));LinearLayoutCompat llRemotes = findViewById(R.id.ll_remotes);llRemotes.addView(surfaceViewRenderer);mRemoteViewMap.put(fromUserId, surfaceViewRenderer);}}});String type = jsonObject.optString("type");String sdp = jsonObject.optString("sdp");PeerConnection finalPeerConnection = peerConnection;// 将 offer sdp 作为参数 setRemoteDescriptionSessionDescription sessionDescription = new SessionDescription(SessionDescription.Type.fromCanonicalForm(type), sdp);peerConnection.setRemoteDescription(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}@Overridepublic void onSetSuccess() {ShowLogUtil.debug(fromUserId + " set remote sdp success.");// 通过 PeerConnection 创建 answer,获取 sdpMediaConstraints mediaConstraints = new MediaConstraints();finalPeerConnection.createAnswer(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {ShowLogUtil.verbose(fromUserId + "create answer success.");// 将 answer sdp 作为参数 setLocalDescriptionfinalPeerConnection.setLocalDescription(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}@Overridepublic void onSetSuccess() {ShowLogUtil.verbose(fromUserId + " set local sdp success.");// 发送 answer sdpsendAnswer(sessionDescription, fromUserId);}}, sessionDescription);}@Overridepublic void onSetSuccess() {}}, mediaConstraints);}}, sessionDescription);}private void sendAnswer(SessionDescription answer, String toUserId) {try {JSONObject jsonObject = new JSONObject();jsonObject.put("msgType", "sdp");jsonObject.put("fromUserId", mUserId);jsonObject.put("toUserId", toUserId);jsonObject.put("type", "answer");jsonObject.put("sdp", answer.description);mWebSocketClientHelper.send(jsonObject.toString());} catch (JSONException e) {e.printStackTrace();}}private void receivedAnswer(JSONObject jsonObject) {String fromUserId = jsonObject.optString("fromUserId");PeerConnection peerConnection = mPeerConnectionMap.get(fromUserId);if (peerConnection == null) {peerConnection = createPeerConnection(mPeerConnectionFactory, fromUserId);peerConnection.addTrack(mAudioTrack, STREAM_IDS);peerConnection.addTrack(mVideoTrack, STREAM_IDS);mPeerConnectionMap.put(fromUserId, peerConnection);}runOnUiThread(new Runnable() {@Overridepublic void run() {SurfaceViewRenderer surfaceViewRenderer = mRemoteViewMap.get(fromUserId);if (surfaceViewRenderer == null) {// 初始化 SurfaceViewRender ,这个方法非常重要,不初始化黑屏surfaceViewRenderer = new SurfaceViewRenderer(MultipleDemoActivity.this);surfaceViewRenderer.init(mEglBase.getEglBaseContext(), null);surfaceViewRenderer.setLayoutParams(new LinearLayout.LayoutParams(dp2px(MultipleDemoActivity.this, 90), dp2px(MultipleDemoActivity.this, 160)));LinearLayoutCompat llRemotes = findViewById(R.id.ll_remotes);llRemotes.addView(surfaceViewRenderer);mRemoteViewMap.put(fromUserId, surfaceViewRenderer);}}});String type = jsonObject.optString("type");String sdp = jsonObject.optString("sdp");// 收到 answer sdp,将 answer sdp 作为参数 setRemoteDescriptionSessionDescription sessionDescription = new SessionDescription(SessionDescription.Type.fromCanonicalForm(type), sdp);peerConnection.setRemoteDescription(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}@Overridepublic void onSetSuccess() {ShowLogUtil.debug(fromUserId + " set remote sdp success.");}}, sessionDescription);}private void sendIceCandidate(IceCandidate iceCandidate, String toUserId) {try {JSONObject jsonObject = new JSONObject();jsonObject.put("msgType", "iceCandidate");jsonObject.put("fromUserId", mUserId);jsonObject.put("toUserId", toUserId);jsonObject.put("id", iceCandidate.sdpMid);jsonObject.put("label", iceCandidate.sdpMLineIndex);jsonObject.put("candidate", iceCandidate.sdp);mWebSocketClientHelper.send(jsonObject.toString());} catch (JSONException e) {e.printStackTrace();}}private void receivedCandidate(JSONObject jsonObject) {String fromUserId = jsonObject.optString("fromUserId");PeerConnection peerConnection = mPeerConnectionMap.get(fromUserId);if (peerConnection == null) {return;}String id = jsonObject.optString("id");int label = jsonObject.optInt("label");String candidate = jsonObject.optString("candidate");IceCandidate iceCandidate = new IceCandidate(id, label, candidate);peerConnection.addIceCandidate(iceCandidate);}private void receivedOtherJoin(JSONObject jsonObject) throws JSONException {String userId = jsonObject.optString("userId");PeerConnection peerConnection = mPeerConnectionMap.get(userId);if (peerConnection == null) {// 创建 PeerConnectionpeerConnection = createPeerConnection(mPeerConnectionFactory, userId);// 为 PeerConnection 添加音轨、视轨peerConnection.addTrack(mAudioTrack, STREAM_IDS);peerConnection.addTrack(mVideoTrack, STREAM_IDS);mPeerConnectionMap.put(userId, peerConnection);}runOnUiThread(new Runnable() {@Overridepublic void run() {SurfaceViewRenderer surfaceViewRenderer = mRemoteViewMap.get(userId);if (surfaceViewRenderer == null) {// 初始化 SurfaceViewRender ,这个方法非常重要,不初始化黑屏surfaceViewRenderer = new SurfaceViewRenderer(MultipleDemoActivity.this);surfaceViewRenderer.init(mEglBase.getEglBaseContext(), null);surfaceViewRenderer.setLayoutParams(new LinearLayout.LayoutParams(dp2px(MultipleDemoActivity.this, 90), dp2px(MultipleDemoActivity.this, 160)));LinearLayoutCompat llRemotes = findViewById(R.id.ll_remotes);llRemotes.addView(surfaceViewRenderer);mRemoteViewMap.put(userId, surfaceViewRenderer);}}});PeerConnection finalPeerConnection = peerConnection;// 通过 PeerConnection 创建 offer,获取 sdpMediaConstraints mediaConstraints = new MediaConstraints();peerConnection.createOffer(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {ShowLogUtil.verbose(userId + " create offer success.");// 将 offer sdp 作为参数 setLocalDescriptionfinalPeerConnection.setLocalDescription(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}@Overridepublic void onSetSuccess() {ShowLogUtil.verbose(userId + " set local sdp success.");// 发送 offer sdpsendOffer(sessionDescription, userId);}}, sessionDescription);}@Overridepublic void onSetSuccess() {}}, mediaConstraints);}private void receivedOtherQuit(JSONObject jsonObject) throws JSONException {String userId = jsonObject.optString("userId");PeerConnection peerConnection = mPeerConnectionMap.get(userId);if (peerConnection != null) {peerConnection.close();mPeerConnectionMap.remove(userId);}runOnUiThread(new Runnable() {@Overridepublic void run() {SurfaceViewRenderer surfaceViewRenderer = mRemoteViewMap.get(userId);if (surfaceViewRenderer != null) {((ViewGroup) surfaceViewRenderer.getParent()).removeView(surfaceViewRenderer);mRemoteViewMap.remove(userId);}}});}public static int dp2px(Context context, float dp) {float density = context.getResources().getDisplayMetrics().density;return (int) (dp * density + 0.5f);}
}

其中 WebSocketClientHelper 跟之前一样的,其余逻辑跟 H5 是一样的。多人通话至少需要三个端,我们就等所有端都实现了再最后来看效果。

六、iOS

1.添加依赖

这个跟前两篇的一样,不需要额外引入。

2.MultipleDemoViewController.swift

//
//  LocalDemoViewController.swift
//  WebRTCDemo-iOS
//
//  Created by 覃浩 on 2023/3/21.
//import UIKit
import WebRTC
import SnapKitclass MultipleDemoViewController: UIViewController {private static let AUDIO_TRACK_ID = "ARDAMSa0"private static let VIDEO_TRACK_ID = "ARDAMSv0"private static let STREAM_IDS = ["ARDAMS"]private static let WIDTH = 1280private static let HEIGHT = 720private static let FPS = 30private var localView: RTCEAGLVideoView!private var remoteViews: UIScrollView!private var peerConnectionFactory: RTCPeerConnectionFactory!private var audioTrack: RTCAudioTrack?private var videoTrack: RTCVideoTrack?/**iOS 需要将 Capturer 保存为全局变量,否则无法渲染本地画面*/private var videoCapturer: RTCVideoCapturer?/**iOS 需要将远端流保存为全局变量,否则无法渲染远端画面*/private var remoteStreamDict: [String : RTCMediaStream] = [:]
//    private let userId = UUID().uuidStringprivate let userId = "iOS"private var peerConnectionDict: [String : RTCPeerConnection] = [:]private var remoteViewDict: [String : RTCEAGLVideoView] = [:]private var lbWebSocketState: UILabel? = nilprivate var tfServerUrl: UITextField? = nilprivate let webSocketHelper = WebSocketClientHelper()override func viewDidLoad() {super.viewDidLoad()// 表明 View 不要扩展到整个屏幕,而是在 NavigationBar 下的区域edgesForExtendedLayout = UIRectEdge()self.view.backgroundColor = UIColor.black// WebSocket 状态文本框lbWebSocketState = UILabel()lbWebSocketState!.textColor = UIColor.whitelbWebSocketState!.text = "WebSocket 已断开"self.view.addSubview(lbWebSocketState!)lbWebSocketState!.snp.makeConstraints({ make inmake.left.equalToSuperview().offset(30)make.right.equalToSuperview().offset(-30)make.height.equalTo(40)})// 服务器地址输入框tfServerUrl = UITextField()tfServerUrl!.textColor = UIColor.whitetfServerUrl!.text = "ws://192.168.1.104:8888"tfServerUrl!.placeholder = "请输入服务器地址"tfServerUrl!.delegate = selfself.view.addSubview(tfServerUrl!)tfServerUrl!.snp.makeConstraints({ make inmake.left.equalToSuperview().offset(30)make.right.equalToSuperview().offset(-30)make.height.equalTo(20)make.top.equalTo(lbWebSocketState!.snp.bottom).offset(10)})// 连接 WebSocket 按钮let btnConnect = UIButton()btnConnect.backgroundColor = UIColor.lightGraybtnConnect.setTitle("连接 WebSocket", for: .normal)btnConnect.setTitleColor(UIColor.black, for: .normal)btnConnect.addTarget(self, action: #selector(connect), for: .touchUpInside)self.view.addSubview(btnConnect)btnConnect.snp.makeConstraints({ make inmake.left.equalToSuperview().offset(30)make.width.equalTo(140)make.height.equalTo(40)make.top.equalTo(tfServerUrl!.snp.bottom).offset(10)})// 断开 WebSocket 按钮let btnDisconnect = UIButton()btnDisconnect.backgroundColor = UIColor.lightGraybtnDisconnect.setTitle("断开 WebSocket", for: .normal)btnDisconnect.setTitleColor(UIColor.black, for: .normal)btnDisconnect.addTarget(self, action: #selector(disconnect), for: .touchUpInside)self.view.addSubview(btnDisconnect)btnDisconnect.snp.makeConstraints({ make inmake.left.equalToSuperview().offset(30)make.width.equalTo(140)make.height.equalTo(40)make.top.equalTo(btnConnect.snp.bottom).offset(10)})// 呼叫按钮let btnCall = UIButton()btnCall.backgroundColor = UIColor.lightGraybtnCall.setTitle("加入房间", for: .normal)btnCall.setTitleColor(UIColor.black, for: .normal)btnCall.addTarget(self, action: #selector(join), for: .touchUpInside)self.view.addSubview(btnCall)btnCall.snp.makeConstraints({ make inmake.left.equalToSuperview().offset(30)make.width.equalTo(160)make.height.equalTo(40)make.top.equalTo(btnDisconnect.snp.bottom).offset(10)})// 挂断按钮let btnHangUp = UIButton()btnHangUp.backgroundColor = UIColor.lightGraybtnHangUp.setTitle("退出房间", for: .normal)btnHangUp.setTitleColor(UIColor.black, for: .normal)btnHangUp.addTarget(self, action: #selector(quit), for: .touchUpInside)self.view.addSubview(btnHangUp)btnHangUp.snp.makeConstraints({ make inmake.left.equalToSuperview().offset(30)make.width.equalTo(160)make.height.equalTo(40)make.top.equalTo(btnCall.snp.bottom).offset(10)})webSocketHelper.setDelegate(delegate: self)// 初始化 PeerConnectionFactoryinitPeerConnectionFactory()// 创建 EglBase// 创建 PeerConnectionFactorypeerConnectionFactory = createPeerConnectionFactory()// 创建音轨audioTrack = createAudioTrack(peerConnectionFactory: peerConnectionFactory)// 创建视轨videoTrack = createVideoTrack(peerConnectionFactory: peerConnectionFactory)let tuple = createVideoCapturer(videoSource: videoTrack!.source)let captureDevice = tuple.captureDevicevideoCapturer = tuple.videoCapture// 初始化本地视频渲染控件localView = RTCEAGLVideoView()localView.delegate = selfself.view.insertSubview(localView,at: 0)localView.snp.makeConstraints({ make inmake.width.equalToSuperview()make.height.equalTo(localView.snp.width).multipliedBy(16.0/9.0)make.centerY.equalToSuperview()})videoTrack?.add(localView!)// 开始本地渲染(videoCapturer as? RTCCameraVideoCapturer)?.startCapture(with: captureDevice!, format: captureDevice!.activeFormat, fps: MultipleDemoViewController.FPS)// 初始化远端视频渲染控件容器remoteViews = UIScrollView()self.view.insertSubview(remoteViews, aboveSubview: localView)remoteViews.snp.makeConstraints { maker inmaker.width.equalTo(90)maker.top.equalToSuperview().offset(30)maker.right.equalToSuperview().offset(-30)maker.bottom.equalToSuperview().offset(-30)}}override func viewDidDisappear(_ animated: Bool) {(videoCapturer as? RTCCameraVideoCapturer)?.stopCapture()videoCapturer = nilfor peerConnection in peerConnectionDict.values {peerConnection.close()}peerConnectionDict.removeAll(keepingCapacity: false)remoteViewDict.removeAll(keepingCapacity: false)remoteStreamDict.removeAll(keepingCapacity: false)webSocketHelper.disconnect()}private func initPeerConnectionFactory() {RTCPeerConnectionFactory.initialize()}private func createPeerConnectionFactory() -> RTCPeerConnectionFactory {var videoEncoderFactory = RTCDefaultVideoEncoderFactory()var videoDecoderFactory = RTCDefaultVideoDecoderFactory()if TARGET_OS_SIMULATOR != 0 {videoEncoderFactory = RTCSimluatorVideoEncoderFactory()videoDecoderFactory = RTCSimulatorVideoDecoderFactory()}return RTCPeerConnectionFactory(encoderFactory: videoEncoderFactory, decoderFactory: videoDecoderFactory)}private func createAudioTrack(peerConnectionFactory: RTCPeerConnectionFactory) -> RTCAudioTrack {let mandatoryConstraints : [String : String] = [:]let optionalConstraints : [String : String] = [:]let audioSource = peerConnectionFactory.audioSource(with: RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints))let audioTrack = peerConnectionFactory.audioTrack(with: audioSource, trackId: MultipleDemoViewController.AUDIO_TRACK_ID)audioTrack.isEnabled = truereturn audioTrack}private func createVideoTrack(peerConnectionFactory: RTCPeerConnectionFactory) -> RTCVideoTrack? {let videoSource = peerConnectionFactory.videoSource()let videoTrack = peerConnectionFactory.videoTrack(with: videoSource, trackId: MultipleDemoViewController.VIDEO_TRACK_ID)videoTrack.isEnabled = truereturn videoTrack}private func createVideoCapturer(videoSource: RTCVideoSource) -> (captureDevice: AVCaptureDevice?, videoCapture: RTCVideoCapturer?) {let videoCapturer = RTCCameraVideoCapturer(delegate: videoSource)let captureDevices = RTCCameraVideoCapturer.captureDevices()if (captureDevices.count == 0) {return (nil, nil)}var captureDevice: AVCaptureDevice?for c in captureDevices {// 前摄像头if (c.position == .front) {captureDevice = cbreak}}if (captureDevice == nil) {return (nil, nil)}return (captureDevice, videoCapturer)}private func createPeerConnection(peerConnectionFactory: RTCPeerConnectionFactory, fromUserId: String) -> RTCPeerConnection {let configuration = RTCConfiguration()//        configuration.sdpSemantics = .unifiedPlan//        configuration.continualGatheringPolicy = .gatherContinually//        configuration.iceServers = [RTCIceServer(urlStrings: ["stun:stun.l.google.com:19302"])]let mandatoryConstraints : [String : String] = [:]//      let mandatoryConstraints = [kRTCMediaConstraintsOfferToReceiveAudio: kRTCMediaConstraintsValueTrue,//                                  kRTCMediaConstraintsOfferToReceiveVideo: kRTCMediaConstraintsValueTrue]let optionalConstraints : [String : String] = [:]//        let optionalConstraints = ["DtlsSrtpKeyAgreement" : kRTCMediaConstraintsValueTrue]let mediaConstraints = RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)return peerConnectionFactory.peerConnection(with: configuration, constraints: mediaConstraints, delegate: self)}@objc private func connect() {webSocketHelper.connect(url: tfServerUrl!.text!.trimmingCharacters(in: .whitespacesAndNewlines))}@objc private func disconnect() {webSocketHelper.disconnect()}@objc private func join() {var jsonObject = [String : String]()jsonObject["msgType"] = "join"jsonObject["userId"] = userIddo {let data = try JSONSerialization.data(withJSONObject: jsonObject)webSocketHelper.send(message: String(data: data, encoding: .utf8)!)} catch {ShowLogUtil.verbose("error--->\(error)")}}@objc private func quit() {var jsonObject = [String : String]()jsonObject["msgType"] = "quit"jsonObject["userId"] = userIddo {let data = try JSONSerialization.data(withJSONObject: jsonObject)webSocketHelper.send(message: String(data: data, encoding: .utf8)!)} catch {ShowLogUtil.verbose("error--->\(error)")}for peerConnection in peerConnectionDict.values {peerConnection.close()}peerConnectionDict.removeAll(keepingCapacity: false)for (key, value) in remoteViewDict {remoteViews.removeSubview(view: value)}remoteViewDict.removeAll(keepingCapacity: false)}private func sendOffer(offer: RTCSessionDescription, toUserId: String) {var jsonObject = [String : String]()jsonObject["msgType"] = "sdp"jsonObject["fromUserId"] = userIdjsonObject["toUserId"] = toUserIdjsonObject["type"] = "offer"jsonObject["sdp"] = offer.sdpdo {let data = try JSONSerialization.data(withJSONObject: jsonObject)webSocketHelper.send(message: String(data: data, encoding: .utf8)!)} catch {ShowLogUtil.verbose("error--->\(error)")}}private func receivedOffer(jsonObject: [String : Any]) {let fromUserId = jsonObject["fromUserId"] as? String ?? ""var peerConnection = peerConnectionDict[fromUserId]if (peerConnection == nil) {// 创建 PeerConnectionpeerConnection = createPeerConnection(peerConnectionFactory: peerConnectionFactory, fromUserId: fromUserId)// 为 PeerConnection 添加音轨、视轨peerConnection!.add(audioTrack!, streamIds: MultipleDemoViewController.STREAM_IDS)peerConnection!.add(videoTrack!, streamIds: MultipleDemoViewController.STREAM_IDS)peerConnectionDict[fromUserId] = peerConnection}var remoteView = remoteViewDict[fromUserId]if (remoteView == nil) {let x = 0var y = 0if (remoteViews.subviews.count == 0) {y = 0} else {for i in 0..<remoteViews.subviews.count {y += Int(remoteViews.subviews[i].frame.height)}}let width = 90let height = width / 9 * 16remoteView = RTCEAGLVideoView(frame: CGRect(x: x, y: y, width: width, height: height))remoteViews.appendSubView(view: remoteView!)remoteViewDict[fromUserId] = remoteView}// 将 offer sdp 作为参数 setRemoteDescriptionlet type = jsonObject["type"] as? Stringlet sdp = jsonObject["sdp"] as? Stringlet sessionDescription = RTCSessionDescription(type: .offer, sdp: sdp!)peerConnection?.setRemoteDescription(sessionDescription, completionHandler: { _ inShowLogUtil.verbose("\(fromUserId) set remote sdp success.")// 通过 PeerConnection 创建 answer,获取 sdplet mandatoryConstraints : [String : String] = [:]let optionalConstraints : [String : String] = [:]let mediaConstraints = RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)peerConnection?.answer(for: mediaConstraints, completionHandler: { sessionDescription, error inShowLogUtil.verbose("\(fromUserId) create answer success.")// 将 answer sdp 作为参数 setLocalDescriptionpeerConnection?.setLocalDescription(sessionDescription!, completionHandler: { _ inShowLogUtil.verbose("\(fromUserId) set local sdp success.")// 发送 answer sdpself.sendAnswer(answer: sessionDescription!, toUserId: fromUserId)})})})}private func sendAnswer(answer: RTCSessionDescription, toUserId: String) {var jsonObject = [String : String]()jsonObject["msgType"] = "sdp"jsonObject["fromUserId"] = userIdjsonObject["toUserId"] = toUserIdjsonObject["type"] = "answer"jsonObject["sdp"] = answer.sdpdo {let data = try JSONSerialization.data(withJSONObject: jsonObject)webSocketHelper.send(message: String(data: data, encoding: .utf8)!)} catch {ShowLogUtil.verbose("error--->\(error)")}}private func receivedAnswer(jsonObject: [String : Any]) {let fromUserId = jsonObject["fromUserId"] as? String ?? ""var peerConnection = peerConnectionDict[fromUserId]if (peerConnection == nil) {peerConnection = createPeerConnection(peerConnectionFactory: peerConnectionFactory, fromUserId: fromUserId)peerConnection!.add(audioTrack!, streamIds: MultipleDemoViewController.STREAM_IDS)peerConnection!.add(videoTrack!, streamIds: MultipleDemoViewController.STREAM_IDS)peerConnectionDict[fromUserId] = peerConnection}DispatchQueue.main.async {var remoteView = self.remoteViewDict[fromUserId]if (remoteView == nil) {let x = 0var y = 0if (self.remoteViews.subviews.count == 0) {y = 0} else {for i in 0..<self.remoteViews.subviews.count {y += Int(self.remoteViews.subviews[i].frame.height)}}let width = 90let height = width / 9 * 16remoteView = RTCEAGLVideoView(frame: CGRect(x: x, y: y, width: width, height: height))self.remoteViews.appendSubView(view: remoteView!)self.remoteViewDict[fromUserId] = remoteView}}// 收到 answer sdp,将 answer sdp 作为参数 setRemoteDescriptionlet type = jsonObject["type"] as? Stringlet sdp = jsonObject["sdp"] as? Stringlet sessionDescription = RTCSessionDescription(type: .answer, sdp: sdp!)peerConnection!.setRemoteDescription(sessionDescription, completionHandler: { _ inShowLogUtil.verbose(fromUserId + " set remote sdp success.");})}private func sendIceCandidate(iceCandidate: RTCIceCandidate, toUserId: String)  {var jsonObject = [String : Any]()jsonObject["msgType"] = "iceCandidate"jsonObject["fromUserId"] = userIdjsonObject["toUserId"] = toUserIdjsonObject["id"] = iceCandidate.sdpMidjsonObject["label"] = iceCandidate.sdpMLineIndexjsonObject["candidate"] = iceCandidate.sdpdo {let data = try JSONSerialization.data(withJSONObject: jsonObject)webSocketHelper.send(message: String(data: data, encoding: .utf8)!)} catch {ShowLogUtil.verbose("error--->\(error)")}}private func receivedCandidate(jsonObject: [String : Any]) {let fromUserId = jsonObject["fromUserId"] as? String ?? ""let peerConnection = peerConnectionDict[fromUserId]if (peerConnection == nil) {return}let id = jsonObject["id"] as? Stringlet label = jsonObject["label"] as? Int32let candidate = jsonObject["candidate"] as? Stringlet iceCandidate = RTCIceCandidate(sdp: candidate!, sdpMLineIndex: label!, sdpMid: id)peerConnection!.add(iceCandidate)}private func receiveOtherJoin(jsonObject: [String : Any]) {let userId = jsonObject["userId"] as? String ?? ""var peerConnection = peerConnectionDict[userId]if (peerConnection == nil) {// 创建 PeerConnectionpeerConnection = createPeerConnection(peerConnectionFactory: peerConnectionFactory, fromUserId: userId)// 为 PeerConnection 添加音轨、视轨peerConnection!.add(audioTrack!, streamIds: MultipleDemoViewController.STREAM_IDS)peerConnection!.add(videoTrack!, streamIds: MultipleDemoViewController.STREAM_IDS)peerConnectionDict[userId] = peerConnection}DispatchQueue.main.async {var remoteView = self.remoteViewDict[userId]if (remoteView == nil) {let x = 0var y = 0if (self.remoteViews.subviews.count == 0) {y = 0} else {for i in 0..<self.remoteViews.subviews.count {y += Int(self.remoteViews.subviews[i].frame.height)}}let width = 90let height = width / 9 * 16remoteView = RTCEAGLVideoView(frame: CGRect(x: x, y: y, width: width, height: height))self.remoteViews.appendSubView(view: remoteView!)self.remoteViewDict[userId] = remoteView}}// 通过 PeerConnection 创建 offer,获取 sdplet mandatoryConstraints : [String : String] = [:]let optionalConstraints : [String : String] = [:]let mediaConstraints = RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)peerConnection?.offer(for: mediaConstraints, completionHandler: { sessionDescription, error inShowLogUtil.verbose("\(userId) create offer success.")if (error != nil) {return}// 将 offer sdp 作为参数 setLocalDescriptionpeerConnection?.setLocalDescription(sessionDescription!, completionHandler: { _ inShowLogUtil.verbose("\(userId) set local sdp success.")// 发送 offer sdpself.sendOffer(offer: sessionDescription!, toUserId: userId)})})}private func receiveOtherQuit(jsonObject: [String : Any]) {let userId = jsonObject["userId"] as? String ?? ""Thread(block: {let peerConnection = self.peerConnectionDict[userId]if (peerConnection != nil) {peerConnection?.close()self.peerConnectionDict.removeValue(forKey: userId)}}).start()let remoteView = remoteViewDict[userId]if (remoteView != nil) {remoteViews.removeSubview(view: remoteView!)remoteViewDict.removeValue(forKey: userId)}remoteStreamDict.removeValue(forKey: userId)}
}// MARK: - RTCVideoViewDelegate
extension MultipleDemoViewController: RTCVideoViewDelegate {func videoView(_ videoView: RTCVideoRenderer, didChangeVideoSize size: CGSize) {}
}// MARK: - RTCPeerConnectionDelegate
extension MultipleDemoViewController: RTCPeerConnectionDelegate {func peerConnection(_ peerConnection: RTCPeerConnection, didChange stateChanged: RTCSignalingState) {}func peerConnection(_ peerConnection: RTCPeerConnection, didAdd stream: RTCMediaStream) {ShowLogUtil.verbose("peerConnection didAdd stream--->\(stream)")var userId: String?for (key, value) in peerConnectionDict {if (value == peerConnection) {userId = key}}if (userId == nil) {return}remoteStreamDict[userId!] = streamlet remoteView = remoteViewDict[userId!]if (remoteView == nil) {return}if let videoTrack = stream.videoTracks.first {ShowLogUtil.verbose("video track found.")videoTrack.add(remoteView!)}if let audioTrack = stream.audioTracks.first{ShowLogUtil.verbose("audio track found.")audioTrack.source.volume = 8}}func peerConnection(_ peerConnection: RTCPeerConnection, didRemove stream: RTCMediaStream) {}func peerConnectionShouldNegotiate(_ peerConnection: RTCPeerConnection) {}func peerConnection(_ peerConnection: RTCPeerConnection, didChange newState: RTCIceConnectionState) {if (newState == .disconnected) {DispatchQueue.main.async {var userId: String?for (key, value) in self.peerConnectionDict {if (value == peerConnection) {userId = key}}if (userId == nil) {return}Thread(block: {let peerConnection = self.peerConnectionDict[userId!]if (peerConnection != nil) {peerConnection?.close()self.peerConnectionDict.removeValue(forKey: userId!)}}).start()let remoteView = self.remoteViewDict[userId!]if (remoteView != nil) {self.remoteViews.removeSubview(view: remoteView!)self.remoteViewDict.removeValue(forKey: userId!)}self.remoteStreamDict.removeValue(forKey: userId!)}}}func peerConnection(_ peerConnection: RTCPeerConnection, didChange newState: RTCIceGatheringState) {}func peerConnection(_ peerConnection: RTCPeerConnection, didGenerate candidate: RTCIceCandidate) {
//        ShowLogUtil.verbose("didGenerate candidate--->\(candidate)")var userId: String?for (key, value) in self.peerConnectionDict {if (value == peerConnection) {userId = key}}if (userId == nil) {return}self.sendIceCandidate(iceCandidate: candidate, toUserId: userId!)}func peerConnection(_ peerConnection: RTCPeerConnection, didRemove candidates: [RTCIceCandidate]) {}func peerConnection(_ peerConnection: RTCPeerConnection, didOpen dataChannel: RTCDataChannel) {}
}// MARK: - UITextFieldDelegate
extension MultipleDemoViewController: UITextFieldDelegate {func textFieldShouldReturn(_ textField: UITextField) -> Bool {textField.resignFirstResponder()return true}
}// MARK: - WebSocketDelegate
extension MultipleDemoViewController: WebSocketDelegate {func onOpen() {lbWebSocketState?.text = "WebSocket 已连接"}func onClose() {lbWebSocketState?.text = "WebSocket 已断开"}func onMessage(message: String) {do {let data = message.data(using: .utf8)let jsonObject: [String : Any] = try JSONSerialization.jsonObject(with: data!) as! [String : Any]let msgType = jsonObject["msgType"] as? Stringif ("sdp" == msgType) {let type = jsonObject["type"] as? String;if ("offer" == type) {receivedOffer(jsonObject: jsonObject);} else if ("answer" == type) {receivedAnswer(jsonObject: jsonObject);}} else if ("iceCandidate" == msgType) {receivedCandidate(jsonObject: jsonObject);} else if ("otherJoin" == msgType) {receiveOtherJoin(jsonObject: jsonObject)} else if ("otherQuit" == msgType) {receiveOtherQuit(jsonObject: jsonObject)}} catch {}}
}

其中 UIScrollView 的 appendSubView 和 removeSubView 是我为 UIScrollView 添加的两个扩展方法,方便纵向添加和删除控件:

import UIKitextension UIScrollView {func appendSubView(view: UIView) {let oldShowsHorizontalScrollIndicator = showsHorizontalScrollIndicatorlet oldShowsVerticalScrollIndicator = showsVerticalScrollIndicatorshowsHorizontalScrollIndicator = falseshowsVerticalScrollIndicator = falsevar y = 0.0if (subviews.count == 0) {y = 0} else {for i in 0..<subviews.count {if ("_UIScrollViewScrollIndicator" == String(reflecting: type(of: subviews[i]))){continue}y += subviews[i].frame.height}}view.frame.origin.y = yaddSubview(view)let contentSizeWidth = contentSize.width// 重新计算 UIScrollView 内容高度var contentSizeHeight = 0.0for i in 0..<subviews.count {if ("_UIScrollViewScrollIndicator" == String(reflecting: type(of: subviews[i]))){continue}contentSizeHeight += subviews[i].frame.height}contentSize = CGSize(width: contentSizeWidth, height: contentSizeHeight)showsHorizontalScrollIndicator = oldShowsHorizontalScrollIndicatorshowsVerticalScrollIndicator = oldShowsVerticalScrollIndicator}func removeSubview(view: UIView) {let oldShowsHorizontalScrollIndicator = showsHorizontalScrollIndicatorlet oldShowsVerticalScrollIndicator = showsVerticalScrollIndicatorshowsHorizontalScrollIndicator = falseshowsVerticalScrollIndicator = falsevar index = -1for i in 0..<subviews.count {if (subviews[i] == view) {index = ibreak}}if (index == -1) {return}for i in index+1..<subviews.count {subviews[i].frame.origin.y = subviews[i].frame.origin.y-view.frame.height}view.removeFromSuperview()let contentSizeWidth = contentSize.width// 重新计算 UIScrollView 内容高度var contentSizeHeight = 0.0for i in 0..<subviews.count {if ("_UIScrollViewScrollIndicator" == String(reflecting: type(of: subviews[i]))){continue}contentSizeHeight += subviews[i].frame.height}contentSize = CGSize(width: contentSizeWidth, height: contentSizeHeight)showsHorizontalScrollIndicator = oldShowsHorizontalScrollIndicatorshowsVerticalScrollIndicator = oldShowsVerticalScrollIndicator}
}

好了,现在三端都实现了,我们可以来看看效果了。

七、效果展示

运行 MultipleWebSocketServerHelper 的 main() 方法,我们可以看到服务端已经开启,然后我们依次将 H5、Android、iOS 连接 WebSocket,再依次加入房间:

其中 iOS 在录屏的时候可能是系统限制,画面静止了,但其实跟另外两端是一样的,从另外两端的远端画面可以看到 iOS 是有在采集摄像头画面的。

八、总结

实现完成后可以感觉到多人呼叫其实也没有多难,跟点对点 Demo 的流程大致一样,只是我们需要重新定义创建 PeerConnection 的时机,但是流程仍然是不变的。以及信令有些许不同,信令这就是业务层面的,自己按需来设计,上面我定义的消息格式只是一个最简单的实现。

至此,WebRTC 单人和多人通话的 Demo 全部完成,这就说明它能满足我们基本的视频通话、视频会议等需求,至于丢包处理、美颜滤镜就是网络优化、图像处理相关的了,后续还会记录网络穿透如何去做,以及使用 WebRTC 时的一些小功能,比如屏幕录制、图片投屏、白板等这些视频会议常用的功能。

九、Demo

Demo 传送门

相关文章:

WebRTC 系列(四、多人通话,H5、Android、iOS)

WebRTC 系列&#xff08;三、点对点通话&#xff0c;H5、Android、iOS&#xff09; 上一篇博客中&#xff0c;我们已经实现了点对点通话&#xff0c;即一对一通话&#xff0c;这一次就接着实现多人通话。多人通话的实现方式呢也有好几种方案&#xff0c;这里我简单介绍两种方案…...

uniapp 点击 富文本元素 图片 可以预览(非nvue)

我使用的是uniapp 官方推荐的组件 rich-text&#xff0c;一般我能用官方级用官方&#xff0c;更有保障一些。 一、整体逻辑 1. 定义一段html标签字符串&#xff0c;里面包含图片 2. 将字符串放入rich-text组件中&#xff0c;绑定点击事件itemclick 3. 通过点击事件获取到图片ur…...

【2023年11月第四版教材】第24章《法律法规与标准规范》(合集篇)

第24章《法律法规与标准规范》(合集篇&#xff09; 1 民法典&#xff08;合同编&#xff09;2 招标投标法2.1 关于时间的总结2.2 内容 3 政府采购法4 专利法5 著作权法6 商标法7 网络安全法8 数据安全法 1 民法典&#xff08;合同编&#xff09; 1、要约是希望和他人订立合同的…...

提升战斗力!吃鸡行家分享顶级游戏干货,助你轻松拿下绝地求生

作为吃鸡行家&#xff0c;我们都知道&#xff0c;在绝地求生中提高战斗力至关重要。今天我来分享一些独特的干货&#xff0c;帮助你成为顶级的吃鸡玩家&#xff0c;并分享一些方便吃鸡作图、装备皮肤库存展示和查询的技巧。 首先&#xff0c;让我们来谈谈绝地求生作图工具推荐。…...

C语言练习百题之宏#define命令

宏&#xff08;Macro&#xff09;是C语言中的一种预处理指令&#xff0c;它使用#define命令定义符号常量、宏函数和代码片段。下面列举了各种宏的应用场景以及相关注意事项&#xff1a; 定义常量&#xff1a; #define PI 3.14159265注意事项&#xff1a;使用宏定义常量可以提高…...

阿里云存储I/O性能、IOPS和吞吐量是什么意思?

云盘的存储I/O性能是什么&#xff1f;存储I/O性能又称存储读写性能&#xff0c;指不同阿里云服务器ECS实例规格挂载云盘时&#xff0c;可以达到的性能表现&#xff0c;包括IOPS和吞吐量。阿里云百科网aliyunbaike.com分享阿里云服务器云盘&#xff08;系统盘或数据盘&#xff0…...

Linux知识点 -- 网络基础 -- 数据链路层

Linux知识点 – 网络基础 – 数据链路层 文章目录 Linux知识点 -- 网络基础 -- 数据链路层一、数据链路层1.以太网2.以太网帧格式3.重谈局域网原理4.MAC地址5.MTU6.查看硬件地址和MTU的命令7.ARP协议 二、其他重要协议或技术1.DNS&#xff08;Domain Name System&#xff09;2.…...

git服务器宕机后,怎么用本地仓库重新建立gitlab服务器(包括所有历史版本)

一、重新建立 当您的 GitLab 服务器因为某种原因宕机后&#xff0c;您可以使用本地仓库中的备份数据来恢复 GitLab 服务器。以下是一般的步骤&#xff0c;用于重新建立 GitLab 服务器&#xff1a; 注意&#xff1a; 这些步骤假定您已经定期备份了 GitLab 数据&#xff0c;包括…...

华为云云耀云服务器L实例评测 | 实例使用教学之综合导览

华为云云耀云服务器L实例评测 &#xff5c; 实例使用教学之综合导览 实例使用教学实例场景体验实例性能评测实例评测使用介绍华为云云耀云服务器 华为云云耀云服务器 &#xff08;目前已经全新升级为 华为云云耀云服务器L实例&#xff09; 华为云云耀云服务器是什么华为云云耀云…...

Elasticsearch 高级查询用法

ES&#xff08;Elasticsearch&#xff09;查询语法是用于搜索和检索文档的强大工具&#xff0c;它支持多种查询类型和选项。以下是一些常见的查询语法示例&#xff1a; 1. **Match查询**&#xff1a;使用match查询可以执行全文本搜索。 { "query": { …...

网络架构介绍

1 网络 7 层架构 7 层模型主要包括&#xff1a; 1. 物理层&#xff1a;主要定义物理设备标准&#xff0c;如网线的接口类型、光纤的接口类型、各种传输介质的传输速率等。它的主要作用是传输比特流&#xff08;就是由 1、0 转化为电流强弱来进行传输,到达目的地后在转化为1、0…...

第53节——Redux Toolkit初识

一、什么是Redux Toolkit 1、概念 Redux Toolkit是一个官方支持的、用于简化Redux开发的工具集。它提供了一些简单易用的API和工具&#xff0c;可以帮助开发者更快速、更高效地编写Redux应用。 2、主要功能 简化Redux的配置 Redux Toolkit提供了一个createSlice函数&#…...

AndroidStudio报错:Plugin with id ‘kotlin-android‘ not found.

第一步 要在自己的项目的build.gradle的buildscript中添加ext.kotlin_version 1.3.72 第二步 然后在dependencies里添加classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version" 大体如下&#xff1a; buildscript {ext.kotlin_version 1.3.72r…...

【ADB】借助ADB模拟滑动屏幕,并进行循环

使用adb shell input 的swipe函数&#xff08;应该是个函数&#xff09; swipe x1 y1 x2 y2 time&#xff08;以毫秒为单位&#xff09; adb shell input swipe 1070 2200 1070 200 10000 进行循环 adb shell "for i in $(seq 1 10); do input swipe 1070 2200 1070 2…...

BN体系理解——类封装复现

from pathlib import Path from typing import Optionalimport torch import torch.nn as nn from torch import Tensorclass BN(nn.Module):def __init__(self,num_features,momentum0.1,eps1e-8):##num_features是通道数"""初始化方法:param num_features:特征…...

请求和响应的概述

请求&#xff1a;在浏览器地址栏输入地址&#xff0c;点击回车请求服务器&#xff0c;这个过程就是一个请求过程。 响应&#xff1a;服务器根据浏览器发送的请求&#xff0c;返回数据到浏览器在网页上进行显示&#xff0c;这个过程就称之为响应。 针对Servlet的每次请求&…...

(深度学习快速入门)A Gentle Introduction to Graph Neural Networks 笔记

博客链接沐神讲解视频文章目录 一:什么是图二:常见数据如何表示为图(1)图像(2)文本(3)Real WorldA:分子B:社交网络C:论文引用D:其他三:图涉及的任务(1)图级别任务(2)节点级别任务(3)边级别任务四:使用图数据的挑战...

VIM指令

vim的工作模式 vim一般有6种工作模式。 普通模式&#xff1a;使用vim打开一个文件时默认模式&#xff0c;也叫命令模式&#xff0c;允许用户通过各种命令浏览代码、滚屏等操作。 插入模式&#xff1a;也可以叫做编辑模式&#xff0c;在普通模式下敲击 i 、a 或 o 就进入插入模…...

Android 10.0 framework层实现app默认全屏显示

1.前言 在10.0的系统开发中,在对于第三方app全屏显示的功能需求开发中,需要默认app全屏显示,针对这一个要求,就需要在系统启动app 的过程中,在绘制app阶段就设置全屏属性,接下来就实现这个功能 效果图如下: 2.framework层实现app默认全屏显示的核心类 frameworks\base\…...

【计算机网络黑皮书】传输层

【事先声明】 这是对于中科大的计算机网络的网课的学习笔记&#xff0c;感谢郑烇老师的无偿分享 书籍是《计算机网络&#xff08;自顶向下方法 第6版&#xff09;》 需要的可以私信我&#xff0c;无偿分享&#xff0c;课程简介下也有 课程链接 目录 传输服务与协议网络层与传输…...

在鸿蒙HarmonyOS 5中实现抖音风格的点赞功能

下面我将详细介绍如何使用HarmonyOS SDK在HarmonyOS 5中实现类似抖音的点赞功能&#xff0c;包括动画效果、数据同步和交互优化。 1. 基础点赞功能实现 1.1 创建数据模型 // VideoModel.ets export class VideoModel {id: string "";title: string ""…...

云启出海,智联未来|阿里云网络「企业出海」系列客户沙龙上海站圆满落地

借阿里云中企出海大会的东风&#xff0c;以**「云启出海&#xff0c;智联未来&#xff5c;打造安全可靠的出海云网络引擎」为主题的阿里云企业出海客户沙龙云网络&安全专场于5.28日下午在上海顺利举办&#xff0c;现场吸引了来自携程、小红书、米哈游、哔哩哔哩、波克城市、…...

汽车生产虚拟实训中的技能提升与生产优化​

在制造业蓬勃发展的大背景下&#xff0c;虚拟教学实训宛如一颗璀璨的新星&#xff0c;正发挥着不可或缺且日益凸显的关键作用&#xff0c;源源不断地为企业的稳健前行与创新发展注入磅礴强大的动力。就以汽车制造企业这一极具代表性的行业主体为例&#xff0c;汽车生产线上各类…...

Nginx server_name 配置说明

Nginx 是一个高性能的反向代理和负载均衡服务器&#xff0c;其核心配置之一是 server 块中的 server_name 指令。server_name 决定了 Nginx 如何根据客户端请求的 Host 头匹配对应的虚拟主机&#xff08;Virtual Host&#xff09;。 1. 简介 Nginx 使用 server_name 指令来确定…...

2025盘古石杯决赛【手机取证】

前言 第三届盘古石杯国际电子数据取证大赛决赛 最后一题没有解出来&#xff0c;实在找不到&#xff0c;希望有大佬教一下我。 还有就会议时间&#xff0c;我感觉不是图片时间&#xff0c;因为在电脑看到是其他时间用老会议系统开的会。 手机取证 1、分析鸿蒙手机检材&#x…...

Maven 概述、安装、配置、仓库、私服详解

目录 1、Maven 概述 1.1 Maven 的定义 1.2 Maven 解决的问题 1.3 Maven 的核心特性与优势 2、Maven 安装 2.1 下载 Maven 2.2 安装配置 Maven 2.3 测试安装 2.4 修改 Maven 本地仓库的默认路径 3、Maven 配置 3.1 配置本地仓库 3.2 配置 JDK 3.3 IDEA 配置本地 Ma…...

Reasoning over Uncertain Text by Generative Large Language Models

https://ojs.aaai.org/index.php/AAAI/article/view/34674/36829https://ojs.aaai.org/index.php/AAAI/article/view/34674/36829 1. 概述 文本中的不确定性在许多语境中传达,从日常对话到特定领域的文档(例如医学文档)(Heritage 2013;Landmark、Gulbrandsen 和 Svenevei…...

iOS性能调优实战:借助克魔(KeyMob)与常用工具深度洞察App瓶颈

在日常iOS开发过程中&#xff0c;性能问题往往是最令人头疼的一类Bug。尤其是在App上线前的压测阶段或是处理用户反馈的高发期&#xff0c;开发者往往需要面对卡顿、崩溃、能耗异常、日志混乱等一系列问题。这些问题表面上看似偶发&#xff0c;但背后往往隐藏着系统资源调度不当…...

springboot整合VUE之在线教育管理系统简介

可以学习到的技能 学会常用技术栈的使用 独立开发项目 学会前端的开发流程 学会后端的开发流程 学会数据库的设计 学会前后端接口调用方式 学会多模块之间的关联 学会数据的处理 适用人群 在校学生&#xff0c;小白用户&#xff0c;想学习知识的 有点基础&#xff0c;想要通过项…...

Caliper 配置文件解析:fisco-bcos.json

config.yaml 文件 config.yaml 是 Caliper 的主配置文件,通常包含以下内容: test:name: fisco-bcos-test # 测试名称description: Performance test of FISCO-BCOS # 测试描述workers:type: local # 工作进程类型number: 5 # 工作进程数量monitor:type: - docker- pro…...