wpf下RTSP|RTMP播放器两种渲染模式实现

技术背景

在这篇blog之前，我提到了wpf下播放RTMP和RTSP渲染的两种方式，一种是通过控件模式，另外一种是直接原生RTSP、RTMP播放模块，回调rgb，然后在wpf下渲染，本文就两种方式做个说明。

技术实现

以大牛直播SDK的Windows平台SmartPlayer为例，我们先说第一种通过控件模式，控件模式，非常简单：可以用picturebox，在MainWindow.xaml 做以下设置：

        <WindowsFormsHost HorizontalAlignment="Left" Height="338" Margin="10,10,0,0" VerticalAlignment="Top" Width="480" Background="Black"><wf:PictureBox x:Name="RealPlayWnd"></wf:PictureBox></WindowsFormsHost>

StartPlayer的时候，调NT_SP_SetRenderWindow，把handler设置下去即可，如果需要硬解码，可以先做硬解码检测，检测支持的话，设置硬解码模式。

       /** nt_player_wrapper.cs* Author: daniusdk.com*/public bool StartPlay(String url, bool is_rtsp_tcp_mode, bool is_mute, bool is_hardware_decorder){if ( is_playing_ )return false;if (!OpenPlayerHandle(url, is_rtsp_tcp_mode, is_mute, is_hardware_decorder))return false;//video resolution callbackvideo_size_call_back_ = new SP_SDKVideoSizeCallBack(SP_SDKVideoSizeHandle);NTSmartPlayerSDK.NT_SP_SetVideoSizeCallBack(player_handle_, IntPtr.Zero, video_size_call_back_);if (render_wnd_ != null){NTSmartPlayerSDK.NT_SP_SetRenderWindow(player_handle_, render_wnd_.Handle);NTSmartPlayerSDK.NT_SP_SetRenderScaleMode(player_handle_, 1);}else if(image_wnd_ != null){//video frame callback (YUV/RGB)//format请参见 NT_SP_E_VIDEO_FRAME_FORMAT，如需回调YUV，请设置为 NT_SP_E_VIDEO_FRAME_FROMAT_I420video_frame_call_back_ = new SP_SDKVideoFrameCallBack(SetVideoFrameCallBack);NTSmartPlayerSDK.NT_SP_SetVideoFrameCallBack(player_handle_, (Int32)NT.NTSmartPlayerDefine.NT_SP_E_VIDEO_FRAME_FORMAT.NT_SP_E_VIDEO_FRAME_FORMAT_RGB32, IntPtr.Zero, video_frame_call_back_);}uint ret = NTSmartPlayerSDK.NT_SP_StartPlay(player_handle_);if ( NTBaseCodeDefine.NT_ERC_OK != ret ){NTSmartPlayerSDK.NT_SP_Close(player_handle_);player_handle_ = IntPtr.Zero;return false;}is_playing_ = true;return true;}

另外一种模式，是通过回调rgb，然后在image上渲染，回调rgb，在StartPlay()已有说明。=，设置回调，选择NT_SP_E_VIDEO_FRAME_FORMAT_RGB32格式，然后处理回调数据即可。

video_frame_call_back_ = new SP_SDKVideoFrameCallBack(SetVideoFrameCallBack);NTSmartPlayerSDK.NT_SP_SetVideoFrameCallBack(player_handle_, (Int32)NT.NTSmartPlayerDefine.NT_SP_E_VIDEO_FRAME_FORMAT.NT_SP_E_VIDEO_FRAME_FORMAT_RGB32, IntPtr.Zero, video_frame_call_back_);

处理rgb数据回调的地方，拿到bitmap_source数据，设给image.Source即可：

        public void SDKVideoFrameCallBack(IntPtr handle, UInt32 status, BitmapSource bitmap_source){if (image_wnd_ == null)return;if (player_handle_ == IntPtr.Zero || !is_playing_ || bitmap_source == null)return;image_wnd_.Source = bitmap_source;}

为了便于比较，我们做了个四窗口的demo展示（一路2560*1440，一路1920*1080），上面是通过picturebox控件直接设置handle到原生模块播放，第三第四个窗口知通过image自己绘制：

具体实现如下：

       /** MainWindow.xaml.cs* Author: daniusdk.com*/public MainWindow(){InitializeComponent();if (!InitSDK())return;UIDispatcher = Dispatcher.CurrentDispatcher;player1_ = new nt_player_wrapper(RealPlayWnd, null, UIDispatcher);player1_.EventGetPlayerEventMsg += new DelGetPlayerEventMsg(GetPlayerEventMsgInfo);player1_.EventGetVideoSize += new DelGetVideoSize(GetVideoSize);player2_ = new nt_player_wrapper(RealPlayWnd1, null, UIDispatcher);player2_.EventGetPlayerEventMsg += new DelGetPlayerEventMsg(GetPlayerEventMsgInfo);player2_.EventGetVideoSize += new DelGetVideoSize(GetVideoSize);player3_ = new nt_player_wrapper(null, image_render, UIDispatcher);player3_.EventGetPlayerEventMsg += new DelGetPlayerEventMsg(GetPlayerEventMsgInfo);player3_.EventGetVideoSize += new DelGetVideoSize(GetVideoSize);player4_ = new nt_player_wrapper(null, image_render1, UIDispatcher);player4_.EventGetPlayerEventMsg += new DelGetPlayerEventMsg(GetPlayerEventMsgInfo);player4_.EventGetVideoSize += new DelGetVideoSize(GetVideoSize);}private bool InitSDK(){if (!is_player_sdk_init_){UInt32 isPlayerInited = NT.NTSmartPlayerSDK.NT_SP_Init(0, IntPtr.Zero);if (isPlayerInited != 0){MessageBox.Show("调用NT_SP_Init失败..");return false;}is_player_sdk_init_ = true;}return true;}private void Button_Click_1(object sender, RoutedEventArgs e){if (!player1_.IsPlaying()){player1_.SetBuffer(0);bool is_mute = true;bool is_hardware_decoder = true;if (!player1_.StartPlay("rtsp://admin:daniulive12345@192.168.0.120:554/h264/ch1/main/av_stream", false, is_mute, is_hardware_decoder))return;btn_playback1.Content = "停止播放";}else{player1_.StopPlay();btn_playback1.Content = "开始播放";}}private void Button_Click_2(object sender, RoutedEventArgs e){if (!player2_.IsPlaying()){player2_.SetBuffer(0);bool is_mute = true;bool is_hardware_decoder = true;if (!player2_.StartPlay("rtsp://admin:admin123456@192.168.0.121:554/cam/realmonitor?channel=1&subtype=0", false, is_mute, is_hardware_decoder))return;btn_playback2.Content = "停止播放";}else{player2_.StopPlay();btn_playback2.Content = "开始播放";}}private void Button_Click_3(object sender, RoutedEventArgs e){if (!player3_.IsPlaying()){player3_.SetBuffer(0);bool is_mute = true;bool is_hardware_decoder = true;if (!player3_.StartPlay("rtsp://admin:daniulive12345@192.168.0.120:554/h264/ch1/main/av_stream", false, is_mute, is_hardware_decoder))return;btn_playback3.Content = "停止播放";}else{player3_.StopPlay();btn_playback3.Content = "开始播放";}}private void Button_Click_4(object sender, RoutedEventArgs e){if (!player4_.IsPlaying()){player4_.SetBuffer(0);bool is_mute = true;bool is_hardware_decoder = true;if (!player4_.StartPlay("rtsp://admin:admin123456@192.168.0.121:554/cam/realmonitor?channel=1&subtype=0", false, is_mute, is_hardware_decoder))return;btn_playback4.Content = "停止播放";}else{player4_.StopPlay();btn_playback4.Content = "开始播放";}}

关闭窗口的时候，记得调用停止播放逻辑，所有实例关闭后，调用NT_SP_UnInit()：

        protected override void OnClosing(System.ComponentModel.CancelEventArgs e){if (MessageBox.Show("确定要关闭窗口吗？", "确认", MessageBoxButton.YesNo) != MessageBoxResult.Yes){// 如果用户选择“否”，取消关闭e.Cancel = true;}if (player1_.IsPlaying()){player1_.StopPlay();}player1_.Dispose();if (player2_.IsPlaying()){player2_.StopPlay();}player2_.Dispose();if (player3_.IsPlaying()){player3_.StopPlay();}player3_.Dispose();if (player4_.IsPlaying()){player4_.StopPlay();}player4_.Dispose();if (is_player_sdk_init_){NTSmartPlayerSDK.NT_SP_UnInit();is_player_sdk_init_ = false;}    base.OnClosing(e);}

总结

wpf下实现低延迟的RTSP或RTMP播放，以上两种模式都可以尝试看，都不麻烦，如果想更灵活，可以采用回调rgb然后自己绘制的模式，如果想更省事，那么直接picturebox控件handle设置下去，底层自己绘制，以上是大概的实现逻辑，感兴趣的开发者，或有这方面技术诉求的，有问题可以单独跟我沟通。