Java工程师实现视频文件上传minio文件系统存储及网页实现分批加载视频播放
Java工程师实现minio存储大型视频文件网页实现分批加载视频播放
一、需求说明
老板给我出个题目,让我把的电影文件上传到minio文件系统,再通过WEB端分配加载视频播放,类似于我们普通的电影网站。小编把Java代码共享出来。是真正的能拿过来直接用的。小编我亲自测过。
1.1minio版本
<dependency><groupId>io.minio</groupId><artifactId>minio</artifactId><version>7.1.4</version></dependency>
1.2maven依赖
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"><modelVersion>4.0.0</modelVersion><groupId>com.sun</groupId><artifactId>springboot-sun</artifactId><version>1.0-SNAPSHOT</version><!--springboot工程需要继承的父工程--><parent><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>2.1.8.RELEASE</version></parent><dependencies><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-thymeleaf</artifactId></dependency><!--web开发的起步依赖--><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency><dependency><groupId>org.mybatis.spring.boot</groupId><artifactId>mybatis-spring-boot-starter</artifactId><version>2.1.0</version></dependency><dependency><groupId>mysql</groupId><artifactId>mysql-connector-java</artifactId><!--<scope>runtime</scope>--></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-test</artifactId><scope>test</scope></dependency><dependency><groupId>io.minio</groupId><artifactId>minio</artifactId><version>7.1.4</version></dependency><dependency><groupId>org.apache.commons</groupId><artifactId>commons-lang3</artifactId><version>3.12.0</version></dependency><dependency><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><version>1.18.28</version><scope>provided</scope></dependency><dependency><groupId>joda-time</groupId><artifactId>joda-time</artifactId><version>2.10.12</version>
</dependency></dependencies></project>
1.3分段视频播放Controller
package com.sun.controller;import io.minio.GetObjectArgs;
import io.minio.MinioClient;
import io.minio.ObjectStat;
import io.minio.StatObjectArgs;
import io.minio.errors.*;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestHeader;
import org.springframework.web.bind.annotation.RestController;import com.sun.minio.util.MinIoUtil;import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;@RestController
public class VideoController2 {@GetMapping("/testDisplay2/{fileName}")public ResponseEntity<byte[]> getVideo(@PathVariable String fileName,@RequestHeader(value = "Range", required = false) String range) {try {String bucketName = "movie";String objectName = fileName + ".mp4";MinioClient minioClient= MinIoUtil.minioClient;// 根据Range请求头来设置分批加载的范围GetObjectArgs.Builder getObjectArgsBuilder = GetObjectArgs.builder().bucket(bucketName).object(objectName);long start=0l;if (range!= null && range.startsWith("bytes=")) {String[] values = range.split("=")[1].split("-");start = Long.parseLong(values[0]);long end = values.length > 1? Long.parseLong(values[1]) : -1;if (end!= -1) {long length = end - start + 1;getObjectArgsBuilder.offset(start).length(length);} else {getObjectArgsBuilder.offset(start);}}// 获取视频流try (InputStream inputStream = minioClient.getObject(getObjectArgsBuilder.build())) {ByteArrayOutputStream outputStream = new ByteArrayOutputStream();byte[] buffer = new byte[1024 * 1024]; // 1MB缓冲区int length;while ((length = inputStream.read(buffer))!= -1) {outputStream.write(buffer, 0, length);}byte[] videoBytes = outputStream.toByteArray();HttpHeaders headers = new HttpHeaders();headers.setContentType(MediaType.parseMediaType("video/mp4"));// 如果是分段请求,设置相应的响应头if (range!= null) {ObjectStat objectStat = minioClient.statObject(StatObjectArgs.builder().bucket(bucketName).object(objectName).build());long fileSize = objectStat.length();headers.set("Content-Range", "bytes " + start + "-" + (start + videoBytes.length - 1) + "/" + fileSize);return new ResponseEntity<>(videoBytes, headers, HttpStatus.PARTIAL_CONTENT);} else {headers.setContentLength(videoBytes.length);return new ResponseEntity<>(videoBytes, headers, HttpStatus.OK);}}} catch (Exception e) {e.printStackTrace();return new ResponseEntity<>(HttpStatus.INTERNAL_SERVER_ERROR);}}
}
1.4 html5 播放
<html>
<title>test</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<head><style type="text/css">.upload {margin-top: 100px;margin-left: 100px;text-align: center;}</style></head>
<body>
<h1 style="text-align: center;margin-top: 20px">test</h1>
<div><!-- <img src="/minio_demo/testClick/testDisplay?url=/data/temp/img/a6efce1b9d16fdfabf36882ab08f8c5495ee7b9f.jpg"> -->
</div><div><!-- <video src="/minio_demo/testClick/testDisplay?md5=a172e15a869fd7224618840c0815dcb1" controls width="640" height="360">您的浏览器不支持视频标签。
</video>--><video src="/minio_demo/testDisplay2/test" controls width="640" height="360">您的浏览器不支持视频标签。
</video></div>
</body>
</html>
1.5上传Controller
package com.sun.controller;import io.minio.errors.*;
import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.support.PropertiesLoaderUtils;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.ModelAndView;import com.sun.minio.util.MinIoContentType;
import com.sun.minio.util.MinIoUtil;
import com.sun.minio.util.MinioBucketEnum;
import com.sun.service.MinioServiceImpl;import javax.servlet.RequestDispatcher;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.InputStream;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.Properties;@RestController
@RequestMapping("testClick")
public class MinioDemoController {@Autowiredprivate MinioServiceImpl minioServiceImpl;/*** 直接页面展示* @param response*/@RequestMapping("/testDisplay")public void testDisplay(HttpServletResponse response,String md5) throws Exception {MinIoUtil.displayFile(minioServiceImpl,MinioBucketEnum.VIDEO_FILES,response,md5);}@RequestMapping("/testDownload")public void testDownLoad(HttpServletResponse response,String md5) throws Exception {MinIoUtil.downloadFile(minioServiceImpl,MinioBucketEnum.VIDEO_FILES,response,md5,"");}@RequestMapping("/testLoad")public void testLoadFile(HttpServletResponse response,String md5,String targetPath) throws Exception{MinIoUtil.loadObject(MinioBucketEnum.VIDEO_FILES,targetPath,md5);}@RequestMapping("/deleteFile")public String deleteFile(String md5) throws Exception{MinIoUtil.deleteObject(minioServiceImpl,MinioBucketEnum.VIDEO_FILES, md5);return "deleteFileSucceed";}@RequestMapping("/testUpload")@ResponseBodypublic String testUpload(HttpServletRequest request, @RequestParam("multipartFile") MultipartFile[] multipartfiles,String url,String pucketName)throws Exception{String filePath = "";for(MultipartFile multipartFile:multipartfiles){String contentType = multipartFile.getContentType();InputStream inputStream = multipartFile.getInputStream();if(!StringUtils.isBlank(url)&&!url.startsWith("/")){url = "/"+url;}if(!StringUtils.isBlank(url)&&!url.endsWith("/")){url += "/";}MinioBucketEnum minioBucketEnum = MinioBucketEnum.VIDEO_FILES;/** if(pucketName.equals("monthlytext")){ minioBucketEnum =* MinioBucketEnum.MONTHLY_TEXT; } if(pucketName.equals("email")){* minioBucketEnum = MinioBucketEnum.VIDEO_FILES; }* if(pucketName.equals("excel")){ minioBucketEnum = MinioBucketEnum.EXCEL; }*/String md5 = MinIoUtil.upload(minioServiceImpl,minioBucketEnum,url + multipartFile.getOriginalFilename(), inputStream, MinIoContentType.getContentType(contentType));filePath+="<p>"+md5+"</p>";if(multipartFile.getOriginalFilename().contains(".mp4")) {filePath = "<video src=\"/minio_demo/testClick/testDisplay?md5="+md5+"\" controls width=\"640\" height=\"360\">\n" +" 您的浏览器不支持视频标签。\n" +"</video>";}// 构建包含两个a标签的HTML代码字符串StringBuilder htmlBuilder = new StringBuilder();htmlBuilder.append("<a href=\"/minio_demo/testClick/deleteFile?md5=").append(md5).append("\">删除文件</a>");htmlBuilder.append(" "); // 增加一些空格间隔两个标签htmlBuilder.append("<a href=\"/minio_demo/testClick/testDownload?md5=").append(md5).append("\">下载文件</a>");// 得到最终的HTML字符串String htmlString = htmlBuilder.toString();filePath=filePath+htmlString;// filePath+="<p>"+md5+"</p>";}return filePath;}@RequestMapping("/testMv1")public ModelAndView testMv1(HttpServletRequest request){ModelAndView modelAndView = new ModelAndView();modelAndView.setViewName("/minio/demo");return modelAndView;}@RequestMapping("/testMv2")public ModelAndView testMv2(HttpServletRequest request){ModelAndView modelAndView = new ModelAndView();modelAndView.setViewName("/minio/demo2");return modelAndView;}@RequestMapping("/testMv3")public ModelAndView testMv3(){ModelAndView mv = new ModelAndView();mv.setViewName("/minio/demo3");return mv;}
}
1.6 上传HTML5
<!DOCTYPE html>
<html lang="zh-CN">
<head><title>文件上传页面</title><meta http-equiv="Content-Type" content="text/html; charset=utf-8"><style type="text/css">/* 整体页面样式 */body {font-family: Arial, sans-serif;margin: 0;padding: 0;display: flex;flex-direction: column;justify-content: center;align-items: center;min-height: 100vh;background-color: #f4f4f4;}/* 标题样式 */h1 {color: #333;margin-bottom: 30px;}/* 上传表单容器样式 */.upload {background-color: #fff;padding: 30px;border-radius: 10px;box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);width: 400px;}/* 表单段落样式 */.upload p {margin-bottom: 20px;}/* 文件选择输入框样式 */.upload input[type="file"] {width: 100%;padding: 10px;border: 1px solid #ccc;border-radius: 5px;}/* 文本输入框样式 */.upload input[type="text"] {width: 100%;padding: 10px;border: 1px solid #ccc;border-radius: 5px;}/* 提交按钮样式 */.upload input[type="submit"] {background-color: #007BFF;color: #fff;padding: 10px 20px;border: none;border-radius: 5px;cursor: pointer;transition: background-color 0.3s ease;}.upload input[type="submit"]:hover {background-color: #0056b3;}</style>
</head>
<body><h1>文件上传至MinIO</h1><div class="upload"><form action="/minio_demo/testClick/testUpload" method="post" enctype="multipart/form-data"><p>选择文件: <input type="file" name="multipartFile" /></p><p>桶名称: <input type="text" name="pucketName" value="movie" /></p><p>文件目录: <input type="text" name="url" hidden /></p><p><input type="submit" value="上传并检测" /></p></form></div>
</body>
</html>
1.7 MinIoUtil工具类
package com.sun.minio.util;import io.minio.*;
import io.minio.errors.*;
import org.apache.commons.lang3.StringUtils;
import org.joda.time.DateTime;
import org.slf4j.LoggerFactory;
import org.springframework.beans.BeansException;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.core.io.support.PropertiesLoaderUtils;
import org.springframework.web.multipart.MultipartFile;import com.sun.doman.MinioRecord;
import com.sun.service.MinioServiceImpl;import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServletResponse;
import javax.validation.constraints.NotNull;
import java.io.*;
import java.net.URLEncoder;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
import java.util.concurrent.TimeUnit;/*** minio工具类*//*** minio工具类*/
public class MinIoUtil {private static MinioServiceImpl minioService = null;private static final org.slf4j.Logger Logger = LoggerFactory.getLogger(MinIoUtil.class);public static MinioClient minioClient = null;private static String enpoint;private static String username;private static String password;static {try{Properties properties = PropertiesLoaderUtils.loadAllProperties("application.properties");enpoint = properties.getProperty("minioEnpoint");Logger.info("获取minio[enpoint]:"+enpoint);username = properties.getProperty("minioUserName");Logger.info("获取minio[minioUserName]:"+username);password = properties.getProperty("minioPassword");Logger.info("获取minio[minioPassword]"+password);minioClient = MinioClient.builder().endpoint(enpoint).credentials(username,password).build();for(MinioBucketEnum minioBucketEnum: MinioBucketEnum.values()){if(!bucketExist(minioBucketEnum.getBucketName())){createBucket(minioBucketEnum.getBucketName());}}}catch (Exception e){Logger.error("获取minio连接失败",e);e.printStackTrace();}}//检查桶是否存在public static boolean bucketExist(String buckeyName) throws IOException, InvalidKeyException, InvalidResponseException, InsufficientDataException, NoSuchAlgorithmException, ServerException, InternalException, XmlParserException, InvalidBucketNameException, ErrorResponseException {return minioClient.bucketExists(BucketExistsArgs.builder().bucket(buckeyName).build());}//创建桶public static void createBucket(String bucketName) throws IOException, InvalidKeyException, InvalidResponseException, RegionConflictException, InsufficientDataException, NoSuchAlgorithmException, ServerException, InternalException, XmlParserException, InvalidBucketNameException, ErrorResponseException {minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucketName).build());}/**** @return*/public Map<String,String> getAccess(){Map<String,String> map = new HashMap<>();map.put("username",username);map.put("password",password);return map;}/**** @param name 文件名,可以是单纯的文件名test.pdf,也可以是类似aaa/bbb/ccc/test.pdf,如果没有重复文件名,则会为在minio中的文件路径* @param inputStream 文件流* @param contentType* @return*/public static String upload(MinioServiceImpl minioService ,MinioBucketEnum minioBucketEnum, String name, InputStream inputStream, MinIoContentType contentType)throws Exception{if(StringUtils.isBlank(name)||inputStream==null){return null;}try{BufferedInputStream bis = new BufferedInputStream(inputStream);String md5Val = MD5Util.md5HashCode32(bis);MinioRecord byMd5Val = minioService.getByMd5Val(minioBucketEnum.getBucketName(),md5Val);if(byMd5Val==null){if(objectExist(minioBucketEnum.getBucketName(),name)){//minio存在同名文件,需要改一下文件名name = getOtherName(name,md5Val);}boolean flag = false;int count = 3;for(int i = 0;i<count;i++){//失败尝试3次.最后一次还失败抛异常if(flag){break;}try{minioClient.putObject(PutObjectArgs.builder().bucket(minioBucketEnum.getBucketName()).object(name).contentType(contentType == null ? MinIoContentType.STREAM.contentType : contentType.contentType).stream(bis, bis.available(), partSize).build());flag = true;}catch (Exception e){if (i==count-1){throw e;}TimeUnit.MILLISECONDS.sleep(200);}}minioService.insertOne(MinioRecord.builder().bucketName(minioBucketEnum.getBucketName()).md5Val(md5Val).minioFilePath(name).remainNum(1).createTime(DateTime.now().toString("yyyy-MM-dd HH:mm:ss")).updateTime(DateTime.now().toString("yyyy-MM-dd HH:mm:ss")).build());}else{byMd5Val.setRemainNum((byMd5Val.getRemainNum()==null?0:byMd5Val.getRemainNum())+1);minioService.updateRemainNum(byMd5Val);}return md5Val;}catch (Exception e){Logger.error("上传文件失败,name:"+name,e);throw e;}finally {try{inputStream.close();}catch (Exception e){}}}public static String upload(MinioBucketEnum minioBucketEnum, String name, InputStream inputStream, MinIoContentType contentType)throws Exception{if(StringUtils.isBlank(name)||inputStream==null){return null;}try{BufferedInputStream bis = new BufferedInputStream(inputStream);String md5Val = MD5Util.md5HashCode32(bis);MinioRecord byMd5Val = minioService.getByMd5Val(minioBucketEnum.getBucketName(),md5Val);if(byMd5Val==null){if(objectExist(minioBucketEnum.getBucketName(),name)){//minio存在同名文件,需要改一下文件名name = getOtherName(name,md5Val);}boolean flag = false;int count = 3;for(int i = 0;i<count;i++){//失败尝试3次.最后一次还失败抛异常if(flag){break;}try{minioClient.putObject(PutObjectArgs.builder().bucket(minioBucketEnum.getBucketName()).object(name).contentType(contentType == null ? MinIoContentType.STREAM.contentType : contentType.contentType).stream(bis, bis.available(), partSize).build());flag = true;}catch (Exception e){if (i==count-1){throw e;}TimeUnit.MILLISECONDS.sleep(200);}}minioService.insertOne(MinioRecord.builder().bucketName(minioBucketEnum.getBucketName()).md5Val(md5Val).minioFilePath(name).remainNum(1).createTime(DateTime.now().toString("yyyy-MM-dd HH:mm:ss")).updateTime(DateTime.now().toString("yyyy-MM-dd HH:mm:ss")).build());}else{byMd5Val.setRemainNum((byMd5Val.getRemainNum()==null?0:byMd5Val.getRemainNum())+1);minioService.updateRemainNum(byMd5Val);}return md5Val;}catch (Exception e){Logger.error("上传文件失败,name:"+name,e);throw e;}finally {try{inputStream.close();}catch (Exception e){}}}/**** @param name 文件名,可以是单纯的文件名test.pdf,也可以是类似aaa/bbb/ccc/test.pdf,如果没有重复文件名,则会为在minio中的文件路径* @param filePath 需要上传的文件路径* @param contentType* @return*/public static String upload(MinioBucketEnum minioBucketEnum, String name, String filePath, MinIoContentType contentType) throws Exception {File file = new File(filePath);if(!file.exists()||minioBucketEnum==null){return null;}return upload(minioBucketEnum,name, new FileInputStream(file), contentType);}/*** 基于MultipartFile文件上传* @param minioBucketEnum 桶信息* @param file 文件* @param contentType 内容类型* @return 返回结果* @throws Exception 异常*/public static String upload(@NotNull MinioBucketEnum minioBucketEnum, @NotNull MultipartFile file, MinIoContentType contentType) throws Exception {String filename = file.getOriginalFilename();byte[] bytes = file.getBytes();ByteArrayInputStream bais = new ByteArrayInputStream(bytes);String md5Str = upload(minioBucketEnum, filename, bais, contentType);bais.close();return md5Str;}/**** @param response* @param md5ValOrOldPath 新文件上传则是md5值,旧文件则依然使用旧文件的路径* @param exportFileName 最终给到前端的文件名*/public static void downloadFile(MinioServiceImpl minioService ,MinioBucketEnum minioBucketEnum, HttpServletResponse response, String md5ValOrOldPath,String exportFileName){downloadFileByMd5(minioService,minioBucketEnum,response,checkOutMd5Val(md5ValOrOldPath),exportFileName);}/*** 根据文件md5值获取* @param md5Val* @return*/private static void downloadFileByMd5(MinioServiceImpl minioService ,MinioBucketEnum minioBucketEnum, HttpServletResponse response, String md5Val,String exportFileName){if(StringUtils.isBlank(md5Val)||response==null){return ;}try {MinioRecord byMd5Val = minioService.getByMd5Val(minioBucketEnum.getBucketName(),md5Val);if(byMd5Val!=null){ObjectStat objectStat = minioClient.statObject(StatObjectArgs.builder().bucket(minioBucketEnum.getBucketName()).object(byMd5Val.getMinioFilePath()).build());DownloadUtils.downloadFile(response, minioClient.getObject(GetObjectArgs.builder().bucket(minioBucketEnum.getBucketName()).object(byMd5Val.getMinioFilePath()).build()), objectStat.name(), objectStat.length(), objectStat.contentType(), true, -1,exportFileName);}} catch (Exception e) {Logger.error("下载文件失败",e);e.printStackTrace();}}/**** @param response* @param md5ValOrOldPath 新文件上传则是md5值,旧文件则依然使用旧文件的路径,最终都是通过文件md5找文件* @throws Exception*/public static void displayFile(MinioServiceImpl minioService ,MinioBucketEnum minioBucketEnum, HttpServletResponse response, String md5ValOrOldPath) throws Exception {displayFileByMd5( minioService ,minioBucketEnum,response,checkOutMd5Val(md5ValOrOldPath));}/*** 根据MD5值展示文件* @param response* @param md5Val*/private static void displayFileByMd5(MinioServiceImpl minioService ,MinioBucketEnum minioBucketEnum, HttpServletResponse response, String md5Val) throws Exception {MinioRecord byMd5Val = minioService.getByMd5Val(minioBucketEnum.getBucketName(),md5Val);ObjectStat objectStat = minioClient.statObject(StatObjectArgs.builder().bucket(minioBucketEnum.getBucketName()).object(byMd5Val.getMinioFilePath()).build());String objectName = objectStat.name();String fileName = objectName.substring(objectName.lastIndexOf("/")+1);response.setCharacterEncoding("utf-8");response.setHeader("Content-Type",objectStat.contentType());response.addHeader("Content-Disposition","inline;filename=" + URLEncoder.encode(fileName, "UTF-8"));response.addHeader("Content-Length","" + objectStat.length());try(ServletOutputStream outputStream = response.getOutputStream();BufferedInputStream bufferedInputStream = new BufferedInputStream(getObjectInputStream(minioBucketEnum.getBucketName(),objectName));BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(outputStream);){int lenth;byte[] bytes = new byte[1024];while ((lenth=bufferedInputStream.read(bytes,0,bytes.length))!=-1){bufferedOutputStream.write(bytes,0,lenth);}bufferedOutputStream.flush();}catch (Exception e){Logger.error("下载失败objectname:"+objectName,e);}}/**** @param name* @param md5Val* @return*/private static String getOtherName(String name,String md5Val){try{String preFix = name.substring(0,name.lastIndexOf("/")+1);if(!name.contains(".")){return preFix+md5Val;}String suffix = name.substring(name.lastIndexOf("."));return preFix+md5Val+suffix;}catch (Exception e){//最起码把md5的值返回作为文件名return md5Val;}}/*** 文件下载,可以下载到本地某个地址* @param targetFilaPath 目标文件* @param md5ValOrOldPath 要下载到的目标文件地址,文件如果已经存在就无法下载* @throws Exception*/public static void loadObject(MinioBucketEnum minioBucketEnum, String targetFilaPath, String md5ValOrOldPath) throws Exception{loadOBbjectByMD5(minioBucketEnum,targetFilaPath,checkOutMd5Val(md5ValOrOldPath));}/*** 文件下载,可以下载到本地某个地址* @param targetFilePath 要下载到的目标文件地址,文件如果已经存在就无法下载* @throws Exception*/private static void loadOBbjectByMD5(MinioBucketEnum minioBucketEnum, String targetFilePath, String md5Val) throws Exception {if(StringUtils.isBlank(targetFilePath)){return ;}MinioRecord byMd5Val = minioService.getByMd5Val(minioBucketEnum.getBucketName(),md5Val);if(byMd5Val==null){return ;}File file = new File(targetFilePath);if(file.exists()){return ;}file.createNewFile();loadOBbjectByMD5(minioBucketEnum,new FileOutputStream(file),md5Val);}/**** @param os* @param md5ValOrOldPath 新文件上传则是md5值,旧文件则依然使用旧文件的路径,最终都是通过文件md5找文件* @throws Exception*/public static void loadObject(MinioBucketEnum minioBucketEnum, OutputStream os, String md5ValOrOldPath) throws Exception{//先判断是不是32位的md5值loadOBbjectByMD5(minioBucketEnum,os,checkOutMd5Val(md5ValOrOldPath));}/*** 看下传入参数是否是md5值,为md5值则直接返回..如果是旧文件系统保留的文件路径,需要看下迁移的文件中是否有过该文件,有则返回该文件的实际md5值* 虽然可能会查两次数据库,不过问题不大,这表数据并不大* @param md5ValOrOldPath* @return*/private static String checkOutMd5Val(String md5ValOrOldPath){//先判断是不是32位的md5值if(StringUtils.isBlank(md5ValOrOldPath)){return "";}if(md5ValOrOldPath.length()!=32||md5ValOrOldPath.contains(".")||md5ValOrOldPath.contains("/")||md5ValOrOldPath.contains("\\")){String code = MD5Util.md5HashCode32Str(md5ValOrOldPath);MinioRecord byOldPath = minioService.getByOldPathMD5(code);if(byOldPath==null){return "";}md5ValOrOldPath = byOldPath.getMd5Val();}return md5ValOrOldPath;}/*** 文件下载* @param os 目标输出流* @throws Exception*/private static boolean loadOBbjectByMD5(MinioBucketEnum minioBucketEnum, OutputStream os, String md5Val)throws Exception{if(StringUtils.isBlank(md5Val)||os==null){return false;}MinioRecord byMd5Val = minioService.getByMd5Val(minioBucketEnum.getBucketName(),md5Val);if(byMd5Val==null){return false;}try(BufferedInputStream bufferedInputStream = new BufferedInputStream(getObjectInputStream(minioBucketEnum.getBucketName(),byMd5Val.getMinioFilePath()));BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(os);){int lenth;byte[] bytes = new byte[1024];while ((lenth=bufferedInputStream.read(bytes,0,bytes.length))!=-1){bufferedOutputStream.write(bytes,0,lenth);}bufferedOutputStream.flush();return true;}catch (Exception e){Logger.error("下载失败objname:"+byMd5Val.getMinioFilePath(),e);throw e;}}/*** true存在* @param bucketName* @param name* @return*/private static boolean objectExist(String bucketName,String name)throws Exception{try{ObjectStat objectStat = minioClient.statObject(StatObjectArgs.builder().bucket(bucketName).object(name).build());if(!StringUtils.isBlank(objectStat.name())){return true;}return false;}catch (Exception e){return false;}}private static final long partSize = 1024*1024*5l;/*** 覆盖并上传文件* @param bucketName* @param name* @param inputStream* @throws Exception*//*** 获取对象流* @param bucketName* @param objectName* @return*/private static InputStream getObjectInputStream(String bucketName,String objectName) throws Exception{int count = 3;for(int i = 0;i<count;i++){try{InputStream object = minioClient.getObject(GetObjectArgs.builder().bucket(bucketName).object(objectName).build());return object;}catch (Exception e2){if(i==count-1){throw e2;}TimeUnit.MILLISECONDS.sleep(200);}}return null;}/*** 获取对象流* @param bucketName* @param objectName* @return*/public static InputStream getObjectInputStreamByMd5Val(String bucketName,String objectName) throws IOException, InvalidKeyException, InvalidResponseException, InsufficientDataException, NoSuchAlgorithmException, ServerException, InternalException, XmlParserException, InvalidBucketNameException, ErrorResponseException {MinioRecord byMd5Val = minioService.getByMd5Val(bucketName, objectName);InputStream object = null;try {object = minioClient.getObject(GetObjectArgs.builder().bucket(bucketName).object(byMd5Val.getMinioFilePath()).build());} catch (Exception e) {e.printStackTrace();object = minioClient.getObject(GetObjectArgs.builder().bucket(bucketName).object(byMd5Val.getMinioFilePath()).build());}return object;}/*** 根据md5值删除* 新文件上传则是md5值,旧文件则依然使用旧文件的路径,最终都是通过文件md5找文件* @return*/public static void deleteObject(MinioServiceImpl minioService,MinioBucketEnum minioBucketEnum, String md5ValOrOldPath) throws Exception{String md5Val = checkOutMd5Val(md5ValOrOldPath);MinioRecord byMd5Val = minioService.getByMd5Val(minioBucketEnum.getBucketName(),md5Val);if(byMd5Val==null){return;}if(byMd5Val.getRemainNum()!=null&&byMd5Val.getRemainNum()<=1){if(objectExist(minioBucketEnum.getBucketName(),byMd5Val.getMinioFilePath())){minioClient.removeObject(RemoveObjectArgs.builder().bucket(minioBucketEnum.getBucketName()).object(byMd5Val.getMinioFilePath()).build());minioService.deleteRecord(minioBucketEnum.getBucketName(),md5Val);}}else{Integer remainNum = byMd5Val.getRemainNum();if(remainNum==null){byMd5Val.setRemainNum(1);}else{byMd5Val.setRemainNum(remainNum-1);}minioService.updateRemainNum(byMd5Val);}}/*** 仅供判断该文件是否存在.* @param minioMd5Value* @return*/public static boolean isExist(MinioBucketEnum minioBucketEnum, String minioMd5Value) {MinioRecord byMd5Val = minioService.getByMd5Val(minioBucketEnum.getBucketName(),minioMd5Value);if(byMd5Val==null){return false;}else{return true;}}/*** 返回minio数据库表的记录* @return*/public static MinioRecord getMinioRecordByMd5Value(MinioBucketEnum minioBucketEnum, String minioMd5Value){return minioService.getByMd5Val(minioBucketEnum.getBucketName(),minioMd5Value);}}
package com.sun.minio.util;import org.apache.commons.lang3.StringUtils;public enum MinIoContentType {PDF("application/pdf"),STREAM("application/octet-stream"),IMAGE("image/jpeg"),VEDIO("video/mp4"),TEXT("text/html");public String contentType;private MinIoContentType(String contentType){this.contentType = contentType;}public static MinIoContentType getContentType(String contentType){for(MinIoContentType minIoContentType : MinIoContentType.values()){if(minIoContentType.contentType.equals(contentType)){return minIoContentType;}}return STREAM;}public static MinIoContentType getContentTypeByFileName(String fileName){if(StringUtils.isBlank(fileName)){return STREAM;}String substring = fileName.substring(fileName.lastIndexOf("."), fileName.length());if(StringUtils.isBlank(substring)){return STREAM;}substring = substring.toLowerCase();if("pdf".equals(substring)){return PDF;}if("png".equals(substring)||"jpg".equals(substring)||"jpng".equals(substring)||"gif".equals(substring)){return IMAGE;}if("mp4".equals(substring)||"avi".equals(substring)||"mkv".equals(substring)||"mov".equals(substring)||"rmvb".equals(substring)||"FLV".equals(substring)||"rmvb".equals(substring)||"rm".equals(substring)){return VEDIO;}if("txt".equals(substring)){return TEXT;}return STREAM;}
}
package com.sun.minio.util;public enum MinioBucketEnum {//EMAIL("email"),//EXCEL("excel"),//TYPICAL("typical"),//FIRE_CONTROL("firecontrol"),//MONTHLY_TEXT("monthlytext");VIDEO_FILES("movie");private String bucketName;private MinioBucketEnum(String bucketName){this.bucketName = bucketName;}public String getBucketName(){return bucketName;}
}
package com.sun.minio.util;import java.io.BufferedInputStream;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.InputStream;
import java.security.MessageDigest;
/*** 网上copy的代码,获取文件的32位MD5值*/
public class MD5Util {/*** 保证文件的MD5值为32位* @param filePath 文件路径* @return* @throws FileNotFoundException*/public static String md5HashCode32(String filePath) throws Exception{BufferedInputStream bfis = new BufferedInputStream(new FileInputStream(filePath));return md5HashCode32(bfis);}/*** java计算文件32位md5值* @param fis 输入流* @return*/public static String md5HashCode32(InputStream fis) {try {fis.mark(Integer.MAX_VALUE);//拿到一个MD5转换器,如果想使用SHA-1或SHA-256,则传入SHA-1,SHA-256MessageDigest md = MessageDigest.getInstance("MD5");//分多次将一个文件读入,对于大型文件而言,比较推荐这种方式,占用内存比较少。byte[] buffer = new byte[1024];int length = -1;while ((length = fis.read(buffer, 0, 1024)) != -1) {md.update(buffer, 0, length);}fis.reset();//转换并返回包含16个元素字节数组,返回数值范围为-128到127byte[] md5Bytes = md.digest();StringBuffer hexValue = new StringBuffer();for (int i = 0; i < md5Bytes.length; i++) {int val = ((int) md5Bytes[i]) & 0xff;//解释参见最下方if (val < 16) {/*** 如果小于16,那么val值的16进制形式必然为一位,* 因为十进制0,1...9,10,11,12,13,14,15 对应的 16进制为 0,1...9,a,b,c,d,e,f;* 此处高位补0。*/hexValue.append("0");}//这里借助了Integer类的方法实现16进制的转换hexValue.append(Integer.toHexString(val));}return hexValue.toString();} catch (Exception e) {e.printStackTrace();return "";}}/***字符串经过md5加密成32位*/public static String md5HashCode32Str(String plainText) {try {MessageDigest md = MessageDigest.getInstance("MD5");md.update(plainText.getBytes());byte[] md5Bytes = md.digest();StringBuffer hexValue = new StringBuffer();for (int i = 0; i < md5Bytes.length; i++) {int val = ((int) md5Bytes[i]) & 0xff;//解释参见最下方if (val < 16) {/*** 如果小于16,那么val值的16进制形式必然为一位,* 因为十进制0,1...9,10,11,12,13,14,15 对应的 16进制为 0,1...9,a,b,c,d,e,f;* 此处高位补0。*/hexValue.append("0");}//这里借助了Integer类的方法实现16进制的转换hexValue.append(Integer.toHexString(val));}return hexValue.toString();} catch (Exception e) {e.printStackTrace();return "";}}
}
package com.sun.minio.util;import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.util.StringUtils;import javax.servlet.http.HttpServletResponse;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URLEncoder;public class DownloadUtils {private static final String JSON_APPLICATION = "application/json";private static final Logger log = LoggerFactory.getLogger(DownloadUtils.class.getName());public static boolean downloadFile( HttpServletResponse response, InputStream inputStream, String objectName, long fileLen, String contentType, boolean closeInputStream, long maxAge,String exportFileName) throws Exception {String fileName = StringUtils.isEmpty(exportFileName)?objectName.substring(objectName.lastIndexOf("/")+1):exportFileName;if (!StringUtils.isEmpty(contentType)) {response.setContentType(contentType);} else {response.setContentType("application/octet-stream");}response.setCharacterEncoding("utf-8");response.setHeader("Content-Disposition", "attachment;filename=" + URLEncoder.encode(fileName, "UTF-8"));response.setHeader("X-Actual-Content-Length", String.valueOf(fileLen));cors(response, maxAge);OutputStream out = response.getOutputStream();byte[] buffer = new byte[1024];int len;try {while ((len = inputStream.read(buffer)) != -1) {out.write(buffer, 0, len);}} catch (Exception e) {log.info("download file error.e=" + e);return false;} finally {if (closeInputStream) {inputStream.close();}out.flush();out.close();}return true;}public static void cors(HttpServletResponse response, long maxAge) {response.setHeader("Access-Control-Allow-Origin", "*");response.setHeader("Access-Control-Allow-Methods", "POST, GET, OPTIONS, DELETE, HEAD");if (maxAge > 0) {response.setHeader("Access-Control-Max-Age", String.valueOf(maxAge));response.setHeader("Cache-Control", "max-age=" + maxAge);}response.setHeader("Access-Control-Allow-Headers", "*");response.setHeader("Access-Control-Expose-Headers", "Cache-Control, Accept-Ranges, Content-Encoding, Content-Length, Content-Range, X-Actual-Content-Length, Content-Disposition");}
}
相关文章:

Java工程师实现视频文件上传minio文件系统存储及网页实现分批加载视频播放
Java工程师实现minio存储大型视频文件网页实现分批加载视频播放 一、需求说明 老板给我出个题目,让我把的电影文件上传到minio文件系统,再通过WEB端分配加载视频播放,类似于我们普通的电影网站。小编把Java代码共享出来。是真正的能拿过来直…...

Redis(二)value 的五种常见数据类型简述
目录 一、string(字符串) 1、raw 2、int 3、embstr 二、hash(哈希表) 1、hashtable 2、ziplist 三、list(列表) 编辑 1、linkedlist 2、ziplist 3、quicklist(redis 3.2后的列表内…...

Docker 环境中搭建 Redis 哨兵模式集群的步骤与问题解决
在 Docker 环境中搭建 Redis 哨兵模式集群的步骤与问题解决 在 Redis 高可用架构中,哨兵模式(Sentinel)是确保 Redis 集群在出现故障时自动切换主节点的一种机制。通过使用 Redis 哨兵,我们可以实现 Redis 集群的监控、故障检测和…...

【网页自动化】篡改猴入门教程
安装篡改猴 打开浏览器扩展商店(Edge、Chrome、Firefox 等)。搜索 Tampermonkey 并安装。 如图安装后,浏览器右上角会显示一个带有猴子图标的按钮。 创建用户脚本 已进入篡改猴管理面板点击创建 脚本注释说明 name:脚本名称。…...

【顶刊TPAMI 2025】多头编码(MHE)之极限分类 Part 4:MHE表示能力
目录 1 MHE的表示能力2 基于Frobenius-范数的低秩逼近3 基于CE的低秩近似 论文:Multi-Head Encoding for Extreme Label Classification 作者:Daojun Liang, Haixia Zhang, Dongfeng Yuan and Minggao Zhang 单位:山东大学 代码:h…...
Github - unexpected disconnect while reading sideband packet
Open git global config: git config --global -eLet’s try to resolve the issue by increasing buffer: git config --global http.postBuffer 52428800Try to clone again. If that doesn’t work! > You can try the partial fetch method and disabling compressi…...

Ubuntu 环境安装 之 RabbitMQ 快速入手
Hi~!这里是奋斗的明志,很荣幸您能阅读我的文章,诚请评论指点,欢迎欢迎 ~~ 🌱🌱个人主页:奋斗的明志 🌱🌱所属专栏:RabbitMQ 📚本系列文章为个人学…...

UE5中实现右键开镜效果
右键之后添加时间轴,然后设置视野即可。Set Field Of View 时间轴设置,第一个点设置0,90度,因为默认的就是90度 第二个点看武器的类型或者倍境来设置,时间就是开镜时间,值越小开镜速度越快,第二个值就是视野…...

Apache HTTPD 换行解析漏洞(CVE-2017-15715)
漏洞简介 pache HTTPD是一款HTTP服务器,它可以通过mod_php来运行PHP网页。其2.4.0~2.4.29版本中存在一个解析漏洞,在解析PHP时,1.php\x0A将被按照PHP后缀进行解析,导致绕过一些服务器的安全策略。 漏洞环境 vulhub/httpd/CVE-2…...

Excel重新踩坑5:二级下拉列表制作;★数据透视表;
0、在excel中函数公式不仅可以写在单元格里面,还可以写在公式里面。 1、二级下拉列表制作: 2、数据透视表: 概念:通过拖拉就能实现复杂函数才能实现的数据统计问题。 概览:在插入选项中有个数据透视表,数…...
力扣--35.搜索插入位置
题目 给定一个排序数组和一个目标值,在数组中找到目标值,并返回其索引。如果目标值不存在于数组中,返回它将会被按顺序插入的位置。 请必须使用时间复杂度为 O(log n) 的算法。 示例 1: 输入: nums [1,3,5,6], target 5 输出: 2 示例 …...
C# 设计模式(行为型模式):模板方法模式
C# 设计模式(行为型模式):模板方法模式 在开发过程中,我们经常会遇到一类问题:一些操作的整体步骤是固定的,但某些具体步骤的实现会因为场景不同而有所变化。模板方法模式(Template Method Pat…...

Leetcode打卡:设计一个ATM机器
执行结果:通过 题目 2241 设计一个ATM机器 一个 ATM 机器,存有 5 种面值的钞票:20 ,50 ,100 ,200 和 500 美元。初始时,ATM 机是空的。用户可以用它存或者取任意数目的钱。 取款时,…...
【TCP】SYN、ACK、FIN、RST、PSH、URG的全称
在 TCP 协议中,SYN、ACK、FIN、RST、PSH 和 URG 都是控制标志位(Flags),每个标志位对应不同的功能。它们的全称如下: URG:(URGent)紧急 ACK:(ACKnowledgment)确认 PSH:(PuSH)推送 RS…...

【OceanBase】使用 Superset 连接 OceanBase 数据库并进行数据可视化分析
文章目录 前言一、前提条件二、操作步骤2.1 准备云主机实例2.2 安装docker-compose2.3 使用docker-compose安装Superset2.3.1 克隆 Superset 的 GitHub 存储库2.3.2 通过 Docker Compose 启动 Superset 2.4 开通 OB Cloud 云数据库2.5 获取连接串2.6 使用 Superset 连接 OceanB…...

【通识安全】应急救护常识23则
一、异物入眼 任何细小的物体或液体,哪怕是一粒沙子或是一滴洗涤剂进入眼中,都会引起眼部疼痛,甚至损伤眼角膜。 急救办法:首先是用力且频繁地眨眼,用泪水将异物冲刷出去。如果不奏效,就将眼皮捏起&#…...
C语言:cJSON将struct结构体与JSON互相转换
文章目录 struct 转 jsonjson 转 struct 文档: https://github.com/DaveGamble/cJSON 项目结构 . ├── libs │ ├── cJSON.c │ └── cJSON.h └── main.c示例 struct 转 json #include "libs/cJSON.h" #include <stdio.h>// defi…...
在Linux中,如何查看和修改网络接口配置?
在Linux中,查看和修改网络接口配置主要依赖于几个命令行工具。这里详细介绍两种传统的命令行方式以及一些图形化工具(前提:系统支持): 一、临时性修改 1. 使用ifconfig命令(部分系统已被弃用)…...

使用深度学习来实现图像超分辨率 综述!
今天给大家介绍一篇图像超分辨率邻域的综述,这篇综述总结了图像超分辨率领域的几方面:problem settings、数据集、performance metrics、SR方法、特定领域应用以结构组件形式,同时,总结超分方法的优点与限制。讨论了存在的问题和挑…...

基于深度学习的视觉检测小项目(六) 项目的信号和变量的规划
• 关于前后端分离 当前流行的一种常见的前后端分离模式是vueflask,vueflask模式的前端和后端之间进行数据的传递通常是借助 API(应用程序编程接口)来完成的。vue通过调用后端提供的 API 来获取或提交数据。例如,前端可能通过发送…...

YSYX学习记录(八)
C语言,练习0: 先创建一个文件夹,我用的是物理机: 安装build-essential 练习1: 我注释掉了 #include <stdio.h> 出现下面错误 在你的文本编辑器中打开ex1文件,随机修改或删除一部分,之后…...

深入理解JavaScript设计模式之单例模式
目录 什么是单例模式为什么需要单例模式常见应用场景包括 单例模式实现透明单例模式实现不透明单例模式用代理实现单例模式javaScript中的单例模式使用命名空间使用闭包封装私有变量 惰性单例通用的惰性单例 结语 什么是单例模式 单例模式(Singleton Pattern&#…...

AI书签管理工具开发全记录(十九):嵌入资源处理
1.前言 📝 在上一篇文章中,我们完成了书签的导入导出功能。本篇文章我们研究如何处理嵌入资源,方便后续将资源打包到一个可执行文件中。 2.embed介绍 🎯 Go 1.16 引入了革命性的 embed 包,彻底改变了静态资源管理的…...
MySQL JOIN 表过多的优化思路
当 MySQL 查询涉及大量表 JOIN 时,性能会显著下降。以下是优化思路和简易实现方法: 一、核心优化思路 减少 JOIN 数量 数据冗余:添加必要的冗余字段(如订单表直接存储用户名)合并表:将频繁关联的小表合并成…...

并发编程 - go版
1.并发编程基础概念 进程和线程 A. 进程是程序在操作系统中的一次执行过程,系统进行资源分配和调度的一个独立单位。B. 线程是进程的一个执行实体,是CPU调度和分派的基本单位,它是比进程更小的能独立运行的基本单位。C.一个进程可以创建和撤销多个线程;同一个进程中…...

day36-多路IO复用
一、基本概念 (服务器多客户端模型) 定义:单线程或单进程同时监测若干个文件描述符是否可以执行IO操作的能力 作用:应用程序通常需要处理来自多条事件流中的事件,比如我现在用的电脑,需要同时处理键盘鼠标…...
Caliper 配置文件解析:fisco-bcos.json
config.yaml 文件 config.yaml 是 Caliper 的主配置文件,通常包含以下内容: test:name: fisco-bcos-test # 测试名称description: Performance test of FISCO-BCOS # 测试描述workers:type: local # 工作进程类型number: 5 # 工作进程数量monitor:type: - docker- pro…...
Vite中定义@软链接
在webpack中可以直接通过符号表示src路径,但是vite中默认不可以。 如何实现: vite中提供了resolve.alias:通过别名在指向一个具体的路径 在vite.config.js中 import { join } from pathexport default defineConfig({plugins: [vue()],//…...

Xcode 16 集成 cocoapods 报错
基于 Xcode 16 新建工程项目,集成 cocoapods 执行 pod init 报错 ### Error RuntimeError - PBXGroup attempted to initialize an object with unknown ISA PBXFileSystemSynchronizedRootGroup from attributes: {"isa">"PBXFileSystemSynchro…...
FOPLP vs CoWoS
以下是 FOPLP(Fan-out panel-level packaging 扇出型面板级封装)与 CoWoS(Chip on Wafer on Substrate)两种先进封装技术的详细对比分析,涵盖技术原理、性能、成本、应用场景及市场趋势等维度: 一、技术原…...