当前位置：首页 > news >正文

QT下使用ffmpeg+SDL实现音视频播放器，支持录像截图功能，提供源码分享与下载

news 文章来源：https://blog.csdn.net/linyibin_123/article/details/132531652 2025/5/15 21:58:05

前言：

SDL是音视频播放和渲染的一个开源库，主要利用它进行视频渲染和音频播放。
SDL库下载路径：https://github.com/libsdl-org/SDL/releases/tag/release-2.26.3，我使用的是2.26.3版本，大家可以自行选择该版本或其他版本的库。

在这里插入图片描述

一、SDL库介绍：

SDL2.lib、SDL2main.lib和SDL2test.lib是SDL库的不同部分和功能。

SDL2.lib：这是SDL库的主要部分，包含了所有常用的SDL功能和函数。它提供了与窗口、渲染、音频、事件处理等相关的功能。

SDL2main.lib：这是用于Windows平台上的SDL2的可执行文件的入口点的库文件。它包含了与Windows系统相关的代码，用于初始化SDL2和设置应用程序的入口点。

SDL2test.lib：这是SDL测试库，包含了一些用于测试和验证SDL功能的测试代码和工具。

通常情况下，您只需要链接SDL2.lib和SDL2main.lib就可以使用SDL库的大部分功能。SDL2test.lib主要是用于SDL开发的测试和验证，一般情况下不需要链接到您的应用程序中。

请注意，如果你在Qt项目中使用了Qt的消息循环（例如使用QApplication::exec()），则应该使用SDL2main.lib而不是SDL2.lib。这是因为SDL2main.lib包含了一个定义了WinMain函数的模块，可以与Qt的消息循环兼容。如果你不使用Qt的消息循环，可以使用SDL2.lib。

音频方面：
SDL提供两种使音频设备取得音频数据方法：
a. push，SDL以特定的频率调用回调函数，在回调函数中取得音频数据（本文采用）；
b. pull，用户程序以特定的频率调用SDL_QueueAudio()，向音频设备提供数据。

二、SDL常用函数介绍：

2.1. SDL初始化，初始化音频和视频模块以及时间模块

if (SDL_Init(SDL_INIT_VIDEO | SDL_INIT_AUDIO | SDL_INIT_TIMER)) {
TestNotNull(NULL, SDL_GetError());
}

2.2. 创建一个显示窗口给SDL winid是qt窗口的id，这样SDL窗口就会嵌入到Qt窗口里面

window = SDL_CreateWindowFrom((void *)winId());

SDL_Window* SDL_CreateWindow(const char* A,int B,int C,int D,int E,Uint32 F)
函数说明：创建窗口，成功返回指向SDL_Window的指针，失败返回NULL

2.3.为指定窗口创建渲染器上下文

SDL_Renderer* SDL_CreateRenderer(SDL_Window* window, int index, Uint32 flags)

2.4.将制定区域（srcrect）的纹理数据，拷贝到渲染目标尺寸为（dstrect）的渲染器上下文（renderer）中，为下一步的渲染做准备

int SDL_RenderCopy(SDL_Renderer* renderer, SDL_Texture* texture, const SDL_Rect* srcrect, const SDL_Rect* dstrect)

参数的含义如下。
renderer：渲染目标。
texture：输入纹理。
srcrect：选择输入纹理的一块矩形区域作为输入。设置为NULL的时候整个纹理作为输入。
dstrect：选择渲染目标的一块矩形区域作为输出。设置为NULL的时候整个渲染目标作为输出。

2.5.将渲染器上下文中的数据，渲染到关联窗体上去

void SDL_RenderPresent(SDL_Renderer* renderer)

2.6.此函数可为渲染器上下文创建纹理

SDL_Texture* SDL_CreateTexture(SDL_Renderer* renderer,
Uint32 format,
int access,
int w,
int h)

2.7.使用新的像素数据更新给定的纹理矩形。也就是说，可以固定刷新纹理的某一分部区域

图像数据写入显存中
int SDL_UpdateTexture(SDL_Texture* texture,
const SDL_Rect* rect,
const void* pixels,
int pitch)

2.8.清理屏幕

int SDLCALL SDL_RenderClear(SDL_Renderer * renderer);

2.9.线程、锁、条件变量

SDL线程创建：SDL_CreateThread
SDL线程等待：SDL_WaitThead
SDL互斥锁：SDL_CreateMutex/SDL_DestroyMutex
SDL锁定互斥：SDL_LockMutex/SDL_UnlockMutex
SDL条件变量(信号量)：SDL_CreateCond/SDL_DestoryCond
SDL条件变量(信号量)等待/通知：SDL_CondWait/SDL_CondSingal

10.读取事件

SDL_PollEvent
从事件队列中，读取事件的常用函数
如果时间队列中有待处理事件，返回1;如果没有可处理事件，则返回0

11.发送事件

int SDL_PushEvent(SDL_Event * event);

12.等待事件

SDL_WaitEvent

13.音频相关

SDL_PauseAudio(0) //恢复音频播放
SDL_PauseAudio(1) //暂停音频播放
SDL_CloseAudio() //关闭音频

三、ffmpeg+SDL实现音视频播放器：

3.1 支持实时流、支持视频文件的音视频播放

在这里插入图片描述

3.2 支持录像、支持截图功能

录像：
在这里插入图片描述

截图：
在这里插入图片描述

3.3 核心代码

cvideoplayer.h

#ifndef CVIDEOPLAYER_H
#define CVIDEOPLAYER_H#include <QObject>
#include <QWidget>#include <QtWidgets>
#include <SDL.h>
#include "commondef.h"
#include <QQueue>
#include "mp4recorder.h"#define MAX_AUDIO_OUT_SIZE 8*1152typedef struct audio_data
{char data[MAX_AUDIO_OUT_SIZE];int size;
}audio_data_t;class cVideoPlayer : public QWidget
{Q_OBJECT
public:cVideoPlayer(QWidget *parent = nullptr, int nWidth = 704, int nHeight = 576);~cVideoPlayer();public:bool loadVideo(const QString &sUrl);void playVideo();void OpenAudio(bool bOpen);void Snapshot();void startRecord(bool bStart);protected:void paintEvent(QPaintEvent *event) override;public:QQueue<audio_data_t*> m_adq;private:bool m_bRun = false;int m_nPlayWay = MEDIA_PLAY_STREAM;SDL_Window *m_pWindow = nullptr;SDL_Renderer *m_pRenderer = nullptr;SDL_Texture *m_pTexture = nullptr;AVFormatContext *m_pFormatCtx = nullptr;AVCodecContext* m_pVideoCodecCxt = nullptr;          //视频解码器上下文AVCodecContext* m_pAudioCodecCxt = nullptr;          //音频解码器上下文SwrContext *m_pAudioSwrContext = nullptr;            //音频重采样上下文AVFrame *m_pFrame = nullptr;AVFrame *m_pYuvFrame = nullptr;AVFrame *m_pPcmFrame = nullptr;struct SwsContext *m_pSwsCtx = nullptr;int m_nBufferSize = 0;uint8_t *m_pDstBuffer = nullptr;int m_nVideoIndex = -1;int m_nAudioIndex = -1;int64_t m_nStartTime;AVPacket m_packet;enum AVCodecID m_CodecId;enum AVCodecID m_AudioCodecId;int m_nSDLWidth;int m_nSDLHeight;//音频SDL_AudioSpec m_audioSpec;bool m_bSupportAudio = false;bool m_bPauseAudio = false;int m_nAudioSampleRate = 8000;                       //音频采样率int m_nAudioPlaySampleRate = 44100;                  //音频播放采样率int m_nAudioPlayChannelNum = 1;                      //音频播放通道数//mp4录像mp4Recorder m_mp4Recorder;bool m_bRecord = false;//截图bool m_bSnapshot = false;SDL_Surface *m_pSurface = nullptr;
};#endif // CVIDEOPLAYER_H

cvideoplayer.cpp

#include "cvideoplayer.h"QMutex g_lock;// 音频处理回调函数。读队列获取音频包，解码，播放
void sdl_audio_callback(void *userdata, uint8_t *stream, int len)
{cVideoPlayer *pPlayer = (cVideoPlayer*)userdata;SDL_memset(stream, 0, len);if(pPlayer && pPlayer->m_adq.size() > 0){QMutexLocker guard(&g_lock);audio_data_t* pAudioData = pPlayer->m_adq.dequeue();if(pAudioData->size == 0)return;len = (len>pAudioData->size?pAudioData->size:len);SDL_MixAudio(stream, (uint8_t *)pAudioData->data, len, SDL_MIX_MAXVOLUME);delete pAudioData;}
}cVideoPlayer::cVideoPlayer(QWidget *parent, int nWidth, int nHeight) : QWidget(parent),m_nSDLWidth(nWidth), m_nSDLHeight(nHeight)
{this->resize(nWidth, nHeight);
}cVideoPlayer::~cVideoPlayer()
{m_bRun = false;MY_DEBUG << "~cVideoPlayer 000";if(nullptr != m_pSurface){SDL_FreeSurface(m_pSurface);m_pSurface = nullptr;}if(nullptr != m_pTexture){SDL_DestroyTexture(m_pTexture);m_pTexture = nullptr;}MY_DEBUG << "~cVideoPlayer 111";if(nullptr != m_pRenderer){SDL_DestroyRenderer(m_pRenderer);m_pRenderer = nullptr;}MY_DEBUG << "~cVideoPlayer 222";if(nullptr != m_pWindow){SDL_DestroyWindow(m_pWindow);m_pWindow = nullptr;}MY_DEBUG << "~cVideoPlayer 333";SDL_CloseAudio();SDL_Quit();MY_DEBUG << "~cVideoPlayer 444";if(nullptr != m_pFrame){av_frame_free(&m_pFrame);m_pFrame = nullptr;}if(nullptr != m_pYuvFrame){av_frame_free(&m_pYuvFrame);m_pYuvFrame = nullptr;}MY_DEBUG << "~cVideoPlayer 555";if(nullptr != m_pVideoCodecCxt){avcodec_close(m_pVideoCodecCxt);//m_pVideoCodecCxt = nullptr; //设置为nullptr会导致崩溃}MY_DEBUG << "~cVideoPlayer 666";if(m_pFormatCtx){avformat_close_input(&m_pFormatCtx);avformat_free_context(m_pFormatCtx);m_pFormatCtx = nullptr;}MY_DEBUG << "~cVideoPlayer 777";if(nullptr != m_pSwsCtx){sws_freeContext(m_pSwsCtx);m_pSwsCtx = nullptr;}MY_DEBUG << "~cVideoPlayer 888";if(nullptr != m_pDstBuffer){av_free(m_pDstBuffer);}MY_DEBUG << "~cVideoPlayer end";
}bool cVideoPlayer::loadVideo(const QString &sUrl)
{avformat_network_init();if(sUrl.contains("rtsp://") || sUrl.contains("http://")) //实时流m_nPlayWay = MEDIA_PLAY_STREAM;elsem_nPlayWay = MEDIA_PLAY_FILE;m_pFormatCtx = avformat_alloc_context();if (avformat_open_input(&m_pFormatCtx, sUrl.toStdString().c_str(), nullptr, nullptr) != 0){MY_DEBUG << "Failed to open video file";return false;}if (avformat_find_stream_info(m_pFormatCtx, nullptr) < 0){MY_DEBUG << "Failed to find stream information";return false;}m_nVideoIndex = av_find_best_stream(m_pFormatCtx, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);m_nAudioIndex = av_find_best_stream(m_pFormatCtx, AVMEDIA_TYPE_AUDIO, -1, -1, NULL, 0);if (m_nVideoIndex == -1){MY_DEBUG << "Failed to find video stream";return false;}//查找并打开视频解码器m_pVideoCodecCxt = avcodec_alloc_context3(nullptr);if (avcodec_parameters_to_context(m_pVideoCodecCxt, m_pFormatCtx->streams[m_nVideoIndex]->codecpar) != 0){MY_DEBUG << "Failed to copy codec parameters to codec context";return false;}const AVCodec *codec = avcodec_find_decoder(m_pVideoCodecCxt->codec_id);if (codec == nullptr){MY_DEBUG << "Failed to find video decoder";return false;}if (avcodec_open2(m_pVideoCodecCxt, codec, nullptr) < 0){MY_DEBUG << "Failed to open video decoder";return false;}m_CodecId = codec->id;//查找并打开音频解码器if(m_nAudioIndex > 0){//查找音频解码器const AVCodec *pAVCodec = avcodec_find_decoder(m_pFormatCtx->streams[m_nAudioIndex]->codecpar->codec_id);if(!pAVCodec){MY_DEBUG << "audio decoder not found";return false;}m_AudioCodecId = pAVCodec->id;//音频解码器参数配置if (!m_pAudioCodecCxt)m_pAudioCodecCxt = avcodec_alloc_context3(nullptr);if(nullptr == m_pAudioCodecCxt){MY_DEBUG << "avcodec_alloc_context3 error m_pAudioCodecCxt=nullptr";return false;}avcodec_parameters_to_context(m_pAudioCodecCxt, m_pFormatCtx->streams[m_nAudioIndex]->codecpar);//打开音频解码器int nRet = avcodec_open2(m_pAudioCodecCxt, pAVCodec, nullptr);if(nRet < 0){avcodec_close(m_pAudioCodecCxt);MY_DEBUG << "avcodec_open2 error m_pAudioCodecCxt";return false;}//音频重采样初始化if (nullptr == m_pAudioSwrContext){if(m_pAudioCodecCxt->channel_layout <= 0 || m_pAudioCodecCxt->channel_layout > 3)m_pAudioCodecCxt->channel_layout = 1;MY_DEBUG << "m_audioCodecContext->channel_layout:" << m_pAudioCodecCxt->channel_layout;MY_DEBUG << "m_audioCodecContext->channels:" << m_pAudioCodecCxt->channels;m_pAudioSwrContext = swr_alloc_set_opts(0,m_pAudioCodecCxt->channel_layout,AV_SAMPLE_FMT_S16,m_pAudioCodecCxt->sample_rate,av_get_default_channel_layout(m_pAudioCodecCxt->channels),m_pAudioCodecCxt->sample_fmt,m_pAudioCodecCxt->sample_rate,0,0);auto nRet = swr_init(m_pAudioSwrContext);if(nRet < 0){MY_DEBUG << "swr_init error";return false;}}m_nAudioSampleRate = m_pAudioCodecCxt->sample_rate;m_audioSpec.freq = m_nAudioSampleRate;                  // 采样率m_audioSpec.format = AUDIO_S16SYS;                      // S表带符号，16是采样深度，SYS表采用系统字节序m_audioSpec.channels = m_pAudioCodecCxt->channels;      // 声道数m_audioSpec.silence = 0;                                // 静音值m_audioSpec.samples = m_pAudioCodecCxt->frame_size;     // SDL声音缓冲区尺寸，单位是单声道采样点尺寸x通道数m_audioSpec.callback = sdl_audio_callback;              // 回调函数，若为NULL，则应使用SDL_QueueAudio()机制m_audioSpec.userdata = this;                            // 提供给回调函数的参数//打开音频设备并创建音频处理线程if(SDL_OpenAudio(&m_audioSpec, NULL) < 0){MY_DEBUG << "SDL_OpenAudio failed.";return false;}if(nullptr == m_pPcmFrame)m_pPcmFrame = av_frame_alloc();m_bSupportAudio = true;}av_dump_format(m_pFormatCtx, 0, NULL, 0);m_pFrame = av_frame_alloc();m_pSwsCtx = sws_getContext(m_pVideoCodecCxt->width, m_pVideoCodecCxt->height, m_pVideoCodecCxt->pix_fmt,m_pVideoCodecCxt->width, m_pVideoCodecCxt->height, AV_PIX_FMT_YUV420P,SWS_BICUBIC, nullptr, nullptr, nullptr);m_nBufferSize = av_image_get_buffer_size(AV_PIX_FMT_YUV420P, m_pVideoCodecCxt->width, m_pVideoCodecCxt->height, 1);m_pDstBuffer = (unsigned char*)av_malloc(m_nBufferSize);if (!m_pDstBuffer){MY_DEBUG << "av_malloc error";return false;}m_pYuvFrame = av_frame_alloc();int ret = av_image_fill_arrays(m_pYuvFrame->data, m_pYuvFrame->linesize, m_pDstBuffer, AV_PIX_FMT_YUV420P, m_pVideoCodecCxt->width, m_pVideoCodecCxt->height, 1);if(ret <= 0){return false;}//SDLif(SDL_Init(SDL_INIT_VIDEO | SDL_INIT_AUDIO | SDL_INIT_TIMER)){MY_DEBUG << "SDL_Init failed.";return false;}m_pWindow = SDL_CreateWindowFrom((void *)winId());if(!m_pWindow){MY_DEBUG << "SDL_CreateWindowFrom failed.";return false;}SDL_ShowWindow(m_pWindow);m_pRenderer = SDL_CreateRenderer(m_pWindow, -1, 0);if(!m_pRenderer){MY_DEBUG << "SDL_CreateRenderer failed.";return false;}m_pTexture = SDL_CreateTexture(m_pRenderer, SDL_PIXELFORMAT_IYUV, SDL_TEXTUREACCESS_STREAMING, m_pVideoCodecCxt->width, m_pVideoCodecCxt->height);if(!m_pTexture){MY_DEBUG << "SDL_CreateTexture failed.";return false;}m_pSurface = SDL_CreateRGBSurface(0, m_nSDLWidth, m_nSDLHeight, 24, 0x000000FF, 0x0000FF00, 0x00FF0000, 0);m_bRun = true;m_nStartTime = av_gettime();return true;
}void cVideoPlayer::playVideo()
{SDL_Event event;//启用音频子线程回调处理if(m_bSupportAudio)SDL_PauseAudio(0);while(m_bRun){if(av_read_frame(m_pFormatCtx, &m_packet) >= 0){if(m_bRecord){//MY_DEBUG << "record...";AVPacket* pPkt = av_packet_clone(&m_packet);m_mp4Recorder.saveOneFrame(*pPkt, m_CodecId, m_AudioCodecId);av_packet_free(&pPkt);}if (m_packet.stream_index == m_nVideoIndex){if(m_nPlayWay == MEDIA_PLAY_FILE)//本地文件播放延时处理{AVRational time_base = m_pFormatCtx->streams[m_nVideoIndex]->time_base;AVRational time_base_q = {1, AV_TIME_BASE}; // AV_TIME_BASE_Q;int64_t pts_time = av_rescale_q(m_packet.dts, time_base, time_base_q);//MY_DEBUG << "pts_time:" << pts_time;int64_t now_time = av_gettime() - m_nStartTime;if (pts_time > now_time)av_usleep((pts_time - now_time));}avcodec_send_packet(m_pVideoCodecCxt, &m_packet);while (avcodec_receive_frame(m_pVideoCodecCxt, m_pFrame) == 0){sws_scale(m_pSwsCtx, m_pFrame->data, m_pFrame->linesize, 0, m_pVideoCodecCxt->height,m_pYuvFrame->data, m_pYuvFrame->linesize);SDL_UpdateTexture(m_pTexture, nullptr, m_pYuvFrame->data[0], m_pYuvFrame->linesize[0]);SDL_RenderClear(m_pRenderer);SDL_RenderCopy(m_pRenderer, m_pTexture, nullptr, nullptr);SDL_RenderPresent(m_pRenderer);if(m_bSnapshot){SDL_RenderReadPixels(m_pRenderer, NULL, SDL_PIXELFORMAT_RGB24, m_pSurface->pixels, m_pSurface->pitch);QString sImgePath = QString("%1screenshot.bmp").arg(SNAPSHOT_DEFAULT_PATH);SDL_SaveBMP(m_pSurface, sImgePath.toUtf8().data());m_bSnapshot = false;}SDL_PollEvent(&event);if (event.type == SDL_QUIT){MY_DEBUG << "event.type == SDL_QUIT";m_bRun = false;break;}}}else if(m_packet.stream_index == m_nAudioIndex){int nRet = avcodec_send_packet(m_pAudioCodecCxt, &m_packet);if(nRet != 0){av_packet_unref(&m_packet);continue;}nRet = 0;while(!nRet){nRet = avcodec_receive_frame(m_pAudioCodecCxt, m_pPcmFrame);if(nRet == 0){audio_data_t* pAudioData = new audio_data_t;uint8_t  *pData[1];pData[0] = (uint8_t *)pAudioData->data;//获取目标样本数auto nDstNbSamples = av_rescale_rnd(m_pPcmFrame->nb_samples,m_nAudioPlaySampleRate,m_nAudioSampleRate,AV_ROUND_ZERO);//重采样int nLen = swr_convert(m_pAudioSwrContext, pData, nDstNbSamples,(const uint8_t **)m_pPcmFrame->data,m_pPcmFrame->nb_samples);if(nLen <= 0){MY_DEBUG << "swr_convert error";delete pAudioData;continue;}//获取样本保存的缓存大小int nOutsize = av_samples_get_buffer_size(nullptr, m_pAudioCodecCxt->channels,m_pPcmFrame->nb_samples,AV_SAMPLE_FMT_S16,0);pAudioData->size = nOutsize;QMutexLocker guard(&g_lock);if(!m_bPauseAudio)m_adq.enqueue(pAudioData);}}}av_packet_unref(&m_packet);}}
}void cVideoPlayer::OpenAudio(bool bOpen)
{if(bOpen){m_bPauseAudio = false;SDL_PauseAudio(0);}else{m_bPauseAudio = true;SDL_PauseAudio(1);}
}void cVideoPlayer::Snapshot()
{m_bSnapshot = true;
}void cVideoPlayer::startRecord(bool bStart)
{if(bStart){QString sPath = RECORD_DEFAULT_PATH;QDate date = QDate::currentDate();QTime time = QTime::currentTime();QString sRecordPath = QString("%1%2-%3-%4-%5%6%7.mp4").arg(sPath).arg(date.year()). \arg(date.month()).arg(date.day()).arg(time.hour()).arg(time.minute()). \arg(time.second());MY_DEBUG << "sRecordPath:" << sRecordPath;if(nullptr != m_pFormatCtx && m_bRun){m_bRecord = m_mp4Recorder.Init(m_pFormatCtx, m_CodecId, m_AudioCodecId, sRecordPath);}}else{if(m_bRecord){MY_DEBUG << "stopRecord...";m_mp4Recorder.DeInit();m_bRecord = false;}}
}void cVideoPlayer::paintEvent(QPaintEvent *event)
{Q_UNUSED(event);QPainter painter(this);painter.fillRect(rect(), Qt::black);
}

3.4 开发过程问题处理

1）qtmain.lib 无法解析外部符号_main _winmain
这个错误通常是由于Qt和SDL使用了不同的入口函数引起的。Qt使用的是main函数作为入口，而SDL使用的是WinMain函数作为入口。当你在Qt项目中调用SDL函数时，链接器会找不到main或WinMain函数，从而导致链接错误。
解决方法：在main函数上增加#undef main，如下

#undef main
int main(int argc, char *argv[])
{QApplication a(argc, argv);MainWindow w;w.show();return a.exec();
}

2）视频没渲染问题
创建好窗口以后需要调用 SDL_ShowWindow(window);显示SDL窗口

3.5 播放器工程下载

https://download.csdn.net/download/linyibin_123/88262235

前言：