当前位置：首页 > news >正文

opencv+ffmpeg+QOpenGLWidget开发的音视频播放器demo

news 2025/9/16 16:49:10

前言

本篇文档的demo包含了 1.使用OpenCV对图像进行处理，对图像进行置灰，旋转，抠图，高斯模糊，中值滤波，部分区域清除置黑，背景移除，边缘检测等操作；2.单纯使用opencv播放显示视频；3.使用opencv和openGL播放显示视频；4.在ffmpeg解码后，使用opencv显示视频，并支持对视频的旋转翻转、裁剪、添加文字、添加logo、亮度调节、置灰、录像截图，音频开关等功能。视频播放器同时支持本地文件与网络码流地址的播放。本篇博客的最后有提供工程代码的下载。

一、OpenCV简单介绍：

OpenCV是一个基于Apache2.0许可（开源）发行的跨平台计算机视觉和机器学习软件库，可以运行在Linux、Windows、Android和Mac OS操作系统上。它由一系列 C 函数和少量 C++ 类构成，同时提供了Python、Ruby、MATLAB等语言的接口，实现了图像处理和计算机视觉方面的很多通用算法。应用领域包含：人机互动、物体识别、图像分割、人脸识别、动作识别、运动跟踪、机器人、运动分析、机器视觉、结构分析、汽车安全驾驶等。

二、demo调用OpenCV：

本篇demo使用的是OpenCV-4.5.1, 是直接下载windows版本，没有额外去编译OpenCV的库，安装exe后，提取头文件和相关的库文件，用于工程调用即可。OpenCV下载路径：https://sourceforge.net/projects/opencvlibrary/

    OpenCV常用函数介绍：
    1）Mat Image = imread(filename); //从指定文件加载图像并返回
    2）namedWindow(“IMG-WIN”, cv::WINDOW_NORMAL);//创建图像显示窗口
    3）destroyWindow(“IMG-WIN”);//销毁图像显示窗口
    4）imshow(“IMG-WIN”, Image);//将图片显示在窗口中
    5）imwrite将图像保存到指定文件
    6）其他

三、demo功能介绍：

3.1.图像操作

在这里插入图片描述
图像操作包含了显示原图、置灰、旋转、抠图、高斯模糊、中值滤波，部分区域清除置黑，背景移除，边缘检测等操作。点击显示原图后，通过点击其他按钮，可以实现图像的不同效果，例如置灰：
在这里插入图片描述
其他功能可下载demo进行操作，点击显示原图，可恢复到之前图片显示，点击保存图片按钮可将处理后的图片进行保存。

3.2.图像操作相关代码

//图像操作
/
void MainWindow::on_pushButton_showImage_clicked()
{// load image using opencvm_srcImage = imread("./Image/test1.jpg");std::string winName = "Display window";namedWindow(winName, WINDOW_AUTOSIZE);imshow(winName, m_srcImage);if(m_bShowImage == false){HWND hwnd = static_cast<HWND>(cvGetWindowHandle(winName.c_str()));HWND parent = GetParent(hwnd);//得到nameWindow窗口的父句柄SetParent(hwnd, (HWND)ui->widget_image->winId());//设置ui控件的句柄是父句柄ShowWindow(parent, SW_HIDE);//隐藏掉nameWindow窗口resizeWindow(winName, cv::Size(ui->widget_image->width(), ui->widget_image->height()));}m_destImage = m_srcImage;m_bShowImage = true;
}void MainWindow::on_pushButton_saveImage_clicked()
{if(m_bShowImage){std::string saveFile = "./Image/test-new.jpg";if(imwrite(saveFile, m_destImage)){QMessageBox::information(this, "myOpencv", "Save Image Success.");}else{QMessageBox::critical(this, "myOpencv", "Save Image Failed.");}}elseQMessageBox::critical(this, "myOpencv", "Please click on the Show Original Image button first.");
}void MainWindow::on_pushButton_gray_clicked()
{if(m_bShowImage){cvtColor(m_srcImage, m_destImage, COLOR_BGR2GRAY);std::string winName = "Display window";namedWindow(winName, WINDOW_AUTOSIZE);imshow(winName, m_destImage);}elseQMessageBox::critical(this, "myOpencv", "Please click on the Show Original Image button first.");
}void MainWindow::on_pushButton_rotate_clicked()
{if(m_bShowImage){warpAffine(m_srcImage, m_destImage, getRotationMatrix2D({m_srcImage.cols/2.0f, m_srcImage.rows/2.0f},45,0.5),m_srcImage.size(),cv::INTER_LINEAR, cv::BORDER_CONSTANT, {255,0,0});std::string winName = "Display window";namedWindow(winName, WINDOW_AUTOSIZE);imshow(winName, m_destImage);}elseQMessageBox::critical(this, "myOpencv", "Please click on the Show Original Image button first.");
}void MainWindow::on_pushButton_matting_clicked()
{if(m_bShowImage){m_destImage = m_srcImage({20, 20, 60, 60}).clone();std::string winName = "Display window";namedWindow(winName, WINDOW_AUTOSIZE);imshow(winName, m_destImage);}elseQMessageBox::critical(this, "myOpencv", "Please click on the Show Original Image button first.");
}void MainWindow::on_pushButton_gauss_clicked()
{if(m_bShowImage){GaussianBlur(m_srcImage, m_destImage, cv::Size(5,5), 0);std::string winName = "Display window";namedWindow(winName, WINDOW_AUTOSIZE);imshow(winName, m_destImage);}elseQMessageBox::critical(this, "myOpencv", "Please click on the Show Original Image button first.");
}void MainWindow::on_pushButton_median_clicked()
{if(m_bShowImage){medianBlur(m_srcImage, m_destImage, 5);std::string winName = "Display window";namedWindow(winName, WINDOW_AUTOSIZE);imshow(winName, m_destImage);}elseQMessageBox::critical(this, "myOpencv", "Please click on the Show Original Image button first.");
}void MainWindow::on_pushButton_clear_clicked()
{if(m_bShowImage){cv::Rect roi(40, 40, 80, 80);Mat roiMat = m_destImage(roi);roiMat.setTo(cv::Scalar(0,0,0));std::string winName = "Display window";namedWindow(winName, WINDOW_AUTOSIZE);imshow(winName, m_destImage);}elseQMessageBox::critical(this, "myOpencv", "Please click on the Show Original Image button first.");
}void MainWindow::on_pushButton_clearBackground_clicked()
{//原图Mat image = imread("./Image/test1.jpg");// 创建一个掩码（mask），用于指定哪些区域是前景，哪些区域是背景cv::Mat mask(image.size(), CV_8UC1, cv::Scalar(cv::GC_BGD));cv::Rect rectangle(50, 50, image.cols - 100, image.rows - 100);cv::grabCut(image, mask, rectangle, cv::Mat(), cv::Mat(), 5, cv::GC_INIT_WITH_RECT);// 将掩码中被标记为前景的像素设为白色，背景设为黑色cv::Mat foregroundMask = (mask == cv::GC_PR_FGD) | (mask == cv::GC_FGD);cv::Mat foregroundImage(image.size(), CV_8UC3, cv::Scalar(0, 0, 0));image.copyTo(foregroundImage, foregroundMask);// 显示结果图像cv::imshow("Foreground", foregroundImage);cv::waitKey(0);
}void MainWindow::on_pushButton_edgeDetection_clicked()
{// 读取图片cv::Mat image = cv::imread("./Image/test1.jpg", cv::IMREAD_GRAYSCALE);// 边缘检测cv::Mat edges;cv::Canny(image, edges, 100, 200);// 显示结果cv::imshow("Edges", edges);cv::waitKey(0);
}

3.3.视频操作

在这里插入图片描述

代码中通过宏定义可以旋转不同的视频处理方式（默认使用ffmpeg解码后，opencv播放显示视频），如下：

#define OPENCV_PLAY_VIDEO                   0   //单纯使用opencv播放显示视频
#define OPENCV_OPENGL_PLAY_VIDEO            0   //使用opencv和openGL播放显示视频
#define OPENCV_FFMPEG_PLAY_VIDEO            1   //使用ffmpeg解码后，opencv显示视频

播放器支持对视频的旋转翻转、裁剪、添加文字、添加logo、亮度调节、置灰、录像截图，音频开关等功能，同时支持本地文件与网络码流地址的播放。

简单演示如下：
在这里插入图片描述

对原视频进行相关处理后，可通过点击录像截图按钮进行视频与图片的保存。通过开关音频按钮，支持对音频的播放处理。

demo工程中有提供测试视频和测试logo，可以进行添加测试。
在这里插入图片描述

3.4.视频操作部分代码

void MainWindow::on_pushButton_Open_clicked()
{QString sFileName = QFileDialog::getOpenFileName(this, QString::fromLocal8Bit("Select Video File"));if(sFileName.isEmpty()){QMessageBox::critical(this, "myOpencv", QObject::tr("错误:文件不能为空."));return;}ui->lineEdit_url->setText(sFileName);
}void MainWindow::on_pushButton_Play_clicked()
{QString sStreamAddr = ui->lineEdit_url->text();if(sStreamAddr.isEmpty()){QMessageBox::critical(this, "myOpencv", QObject::tr("错误:码流地址不能为空."));return;}#if OPENCV_FFMPEG_PLAY_VIDEOif(nullptr == m_pMediaThread){MY_DEBUG << "new MediaThread";m_pMediaThread = new MediaThread(ui->openGLWidget);}else{if(m_pMediaThread->isRunning()){QMessageBox::critical(this, "myOpencv", "错误:请先点击停止按钮关闭视频.");return;}}bool bMediaInit = m_pMediaThread->Init(sStreamAddr);if(bMediaInit){m_pMediaThread->startThread();}#elif OPENCV_PLAY_VIDEOif(m_pOpencvPlayThread1 == nullptr){m_pOpencvPlayThread1 = new COpencvPlayThread1(ui->openGLWidget, sStreamAddr);}m_pOpencvPlayThread1->startThread();#elif OPENCV_OPENGL_PLAY_VIDEOif(m_pOpencvPlayThread2 == nullptr){m_pOpencvPlayThread2 = new COpencvPlayThread2(ui->openGLWidget, sStreamAddr);}m_pOpencvPlayThread2->startThread();#endif}void MainWindow::on_pushButton_Stop_clicked()
{
#if OPENCV_PLAY_VIDEOif(m_pOpencvPlayThread1)m_pOpencvPlayThread1->stopThread();
#endif#if OPENCV_OPENGL_PLAY_VIDEOif(m_pOpencvPlayThread2)m_pOpencvPlayThread2->stopThread();
#endif#if OPENCV_FFMPEG_PLAY_VIDEOif(m_pMediaThread){disconnect(m_pMediaThread, SIGNAL(sig_emitImage(const QImage&)),ui->openGLWidget, SLOT(slot_showImage(const QImage&)));m_pMediaThread->stopThread();m_pMediaThread->quit();m_pMediaThread->wait();m_pMediaThread->DeInit();}
#endif}void MainWindow::on_pushButton_startRecord_clicked()
{if(m_pMediaThread)m_pMediaThread->startRecord();
}void MainWindow::on_pushButton_stopRecord_clicked()
{if(m_pMediaThread){m_pMediaThread->stopRecord();QDesktopServices::openUrl(QUrl::fromLocalFile(RECORD_DEFAULT_PATH));}
}void MainWindow::on_pushButton_snapshot_clicked()
{if(m_pMediaThread){m_pMediaThread->Snapshot();QDesktopServices::openUrl(QUrl::fromLocalFile(SNAPSHOT_DEFAULT_PATH));}
}void MainWindow::on_pushButton_video_rotate_clicked()
{if(m_pMediaThread){int nRotate = ui->comboBox_rotate->currentIndex() - 1;if(nRotate < 0)m_pMediaThread->setRotate(false, nRotate);elsem_pMediaThread->setRotate(true, nRotate);}
}void MainWindow::on_pushButton_video_overturn_clicked()
{if(m_pMediaThread){int nOverturn = ui->comboBox_overturn->currentIndex() - 1;if(nOverturn < 0)m_pMediaThread->setFlip(false, nOverturn);elsem_pMediaThread->setFlip(true, nOverturn);}
}bool bGrey = false;
void MainWindow::on_pushButton_video_grey_clicked()
{if(m_pMediaThread){if(bGrey){m_pMediaThread->setGray(false);bGrey = false;}else{m_pMediaThread->setGray(true);bGrey = true;}}
}void MainWindow::on_pushButton_video_cropping_clicked()
{if(m_pMediaThread){int x = ui->lineEdit_crop_X->text().toInt();int y = ui->lineEdit_crop_Y->text().toInt();int w = ui->lineEdit_crop_W->text().toInt();int h = ui->lineEdit_crop_H->text().toInt();m_pMediaThread->setCrop(true, x, y, w, h);}
}void MainWindow::on_pushButton_video_addText_clicked()
{if(m_pMediaThread){int x = ui->lineEdit_text_X->text().toInt();int y = ui->lineEdit_text_Y->text().toInt();int nColor = ui->comboBox_text_rgb->currentIndex();double size = ui->comboBox_text_fonsize->currentText().toDouble();QString sText = ui->lineEdit_videoText->text();MY_DEBUG << "size:" << size;m_pMediaThread->setAddText(true, sText, x, y, nColor, size);}
}void MainWindow::on_pushButton_logoPath_clicked()
{QString sFileName = QFileDialog::getOpenFileName(this, QString::fromLocal8Bit("Select Logo File"));if(sFileName.isEmpty()){QMessageBox::critical(this, "myOpenCV", QObject::tr("错误:文件不能为空."));return;}ui->lineEdit_logo_path->setText(sFileName);
}void MainWindow::on_pushButton_addLogo_clicked()
{if(m_pMediaThread){int x = ui->lineEdit_logo_X->text().toInt();int y = ui->lineEdit_logo_Y->text().toInt();QString sAddLogoPath = ui->lineEdit_logo_path->text();m_pMediaThread->setAddLogo(true, x, y, sAddLogoPath);}
}void MainWindow::on_pushButton_setBrightness_clicked()
{if(m_pMediaThread){double brightness = ui->comboBox_brightnessVal->currentText().toDouble();m_pMediaThread->setImagePara(brightness);}
}void MainWindow::on_pushButton_restore_clicked()
{if(m_pMediaThread){int x = ui->lineEdit_crop_X->text().toInt();int y = ui->lineEdit_crop_Y->text().toInt();int w = ui->lineEdit_crop_W->text().toInt();int h = ui->lineEdit_crop_H->text().toInt();m_pMediaThread->setCrop(false, x, y, w, h);}
}void MainWindow::on_pushButto_video_deleteText_clicked()
{if(m_pMediaThread){int x = ui->lineEdit_text_X->text().toInt();int y = ui->lineEdit_text_Y->text().toInt();int nColor = ui->comboBox_text_rgb->currentIndex();double size = ui->comboBox_text_rgb->currentText().toDouble();QString sText = ui->lineEdit_videoText->text();m_pMediaThread->setAddText(false, sText, x, y, nColor, size);}
}void MainWindow::on_pushButton_deleteLogo_clicked()
{if(m_pMediaThread){int x = ui->lineEdit_logo_X->text().toInt();int y = ui->lineEdit_logo_Y->text().toInt();QString sAddLogoPath = ui->lineEdit_logo_path->text();m_pMediaThread->setAddLogo(false, x, y, sAddLogoPath);}
}void MainWindow::on_pushButton_audioOpen_clicked()
{ctAudioPlayer::getInstance().isPlay(true);
}void MainWindow::on_pushButton_audioClose_clicked()
{ctAudioPlayer::getInstance().isPlay(false);
}

void ctFFmpeg::dealWithOpenCV(AVFrame *pFrame/*, int time*/)
{if(!m_bQuit){cv::Mat destCvMat(cv::Size(m_nVideoW, m_nVideoH), CV_8UC3, cv::Scalar(1, 2, 3));destCvMat.data = (uchar*)pFrame->data[0];//旋转if(m_bRotate){MY_DEBUG << "Set Rotate m_nRotate:" << m_nRotate;rotate(destCvMat, destCvMat, m_nRotate);}//翻转if(m_bFlip){MY_DEBUG << "Set Flip";flip(destCvMat, destCvMat, m_nFlip);}//裁剪if(m_bCrop){destCvMat = destCvMat(Rect(m_nCropX, m_nCropY, m_nCropW, m_nCropH));}//灰度if(m_bGray){MY_DEBUG << "Set Gray";cvtColor(destCvMat, destCvMat, COLOR_BGR2GRAY);}//添加文字if(m_bAddText){MY_DEBUG << "m_nAddTextX:" << m_nAddTextX << ", m_nAddTextY：" << m_nAddTextY << ",m_nAddTextSize:" << m_nAddTextSize;cv::Point ptPos(m_nAddTextX, m_nAddTextY);int fontFace = cv::FONT_HERSHEY_COMPLEX;double fontScale = m_nAddTextSize;int thickness = 2;cv::Scalar color;switch(m_nAddTextColor){case ADD_TEXT_COLOR_TYPE_BLUE:color[0]=255; color[1]=0; color[2]=0;cv::putText(destCvMat, m_sAddText.toStdString(), ptPos, fontFace, fontScale, color, thickness);break;case ADD_TEXT_COLOR_TYPE_GREEN:color[0]=0; color[1]=255; color[2]=0;cv::putText(destCvMat, m_sAddText.toStdString(), ptPos, fontFace, fontScale, color, thickness);break;case ADD_TEXT_COLOR_TYPE_RED:default:color[0]=0; color[1]=0; color[2]=255;//红色cv::putText(destCvMat, m_sAddText.toStdString(), ptPos, fontFace, fontScale, color, thickness);break;}}//添加logoif(m_bAddLogo){cv::Mat logoImage = cv::imread(m_sAddLogoPath.toStdString());cv::Mat roi = destCvMat(cv::Rect(m_nAddLogoX, m_nAddLogoY, logoImage.cols, logoImage.rows));cv::addWeighted(roi, 0.5, logoImage, 0.5, 0, roi);}//亮度\暗度cv::addWeighted(destCvMat, m_nBrightness, cv::Scalar(0, 0, 0), 0, 0, destCvMat);#if OPENCV_NAME_WINDOW_SHOWcv::imshow(m_sWinName.toStdString(), destCvMat);
#endif//效率太慢, 容易导致崩溃, 不适合转QImage再用openGL播放//...//emit sig_getImage(image);//截图if(m_bSnapshot){cv::cvtColor(destCvMat, destCvMat, cv::COLOR_BGR2RGB);QImage image(destCvMat.data, destCvMat.cols, destCvMat.rows, static_cast<int>(destCvMat.step), QImage::Format_RGB888);QPixmap pixPicture = QPixmap::fromImage(image);QString sPath = "./snapshot/";QDate date = QDate::currentDate();QTime time = QTime::currentTime();m_sSnapPath = QString("%1%2-%3-%4-%5%6%7.jpg").arg(sPath).arg(date.year()). \arg(date.month()).arg(date.day()).arg(time.hour()).arg(time.minute()). \arg(time.second());MY_DEBUG << "Snapshot... m_sSnapPath:" << m_sSnapPath;pixPicture.save(m_sSnapPath, "jpg");m_bSnapshot = false;}}
}

四、工程源码下载：

demo播放器下载：https://download.csdn.net/download/linyibin_123/88217460

demo播放器工程源码下载：https://download.csdn.net/download/linyibin_123/88217472

前言