统一删除失效批次材料

统一存储上删除失效的材料，在数据库中会保留失效批次的记录，文件表（file_info）与拓展表（storage_file）中记录被删除；

注：多存储头地区，该方法不会进行材料删除

1、添加sql

1、sys_dict 中新增 deleteInvalidBatchFile

查看是否存在该数据：

select * from sys_dict where category = 'script' and label = 'deleteInvalidBatchFile';

不存在，则插入：

INSERT INTO `business`.`sys_dict`(`id`, `pid`, `category`, `label`, `value`, `sn`, `create_time`, `update_time`, `deleted`, `bak`, `label_bak`) VALUES (1695377034627842741, 1681937547180167200, 'script', 'deleteInvalidBatchFile', 'import cn.hutool.extra.spring.SpringUtil\r\nimport com.baomidou.mybatisplus.core.conditions.query.QueryWrapper\r\nimport com.fasterxml.jackson.databind.ObjectMapper\r\nimport com.fasterxml.jackson.databind.node.NullNode\r\nimport com.whrp.dossier.caseinfo.entity.FileInfo\r\nimport com.whrp.dossier.caseinfo.entity.StorageFile\r\nimport com.whrp.dossier.caseinfo.service.impl.CommonService\r\nimport com.whrp.dossier.caseinfo.service.impl.FileInfoServiceImpl\r\nimport com.whrp.dossier.caseinfo.service.impl.StorageFileServiceImpl\r\nimport io.minio.MinioClient\r\nimport io.minio.errors.ErrorResponseException\r\nimport io.minio.errors.InsufficientDataException\r\nimport io.minio.errors.InternalException\r\nimport io.minio.errors.InvalidBucketNameException\r\nimport io.minio.errors.InvalidResponseException\r\nimport io.minio.errors.XmlParserException\r\nimport org.apache.commons.lang3.time.FastDateFormat\r\nimport org.apache.logging.log4j.LogManager\r\nimport org.apache.logging.log4j.Logger\r\nimport org.springframework.jdbc.core.JdbcTemplate\r\nimport org.springframework.util.ObjectUtils\r\nimport org.springframework.util.StringUtils\r\n\r\nimport java.security.InvalidKeyException\r\nimport java.security.NoSuchAlgorithmException\r\nimport java.util.stream.Collectors\r\ndef main(param){\r\n    Logger logger = LogManager.getLogger();\r\n    ObjectMapper objectMapper = new ObjectMapper();\r\n    JdbcTemplate jdbcTemplate = SpringUtil.getBean(JdbcTemplate.class);\r\n    CommonService commonService = SpringUtil.getBean(CommonService.class);\r\n    FileInfoServiceImpl fileInfoService = SpringUtil.getBean(FileInfoServiceImpl.class);\r\n    StorageFileServiceImpl storageFileService = SpringUtil.getBean(StorageFileServiceImpl.class);\r\n    //通过param获取参数\r\n    //def paramMap = objectMapper.convertValue(param, HashMap.class);\r\n    def startTime = param[\"startTime\"];\r\n    def endTime = param[\"endTime\"];\r\n    def caseId = param[\"caseId\"];\r\n    //判断开始时间和结束时间大小，开始时间不能大于结束时间,需要将时间转换为时间戳\r\n    def sql = \"select sub_case_id as subCaseId, case_id as  caseId from sub_case_info where invalid_flag = 1\";\r\n    def startTimestamp = null;\r\n    if(StringUtils.hasText(startTime)){\r\n        startTimestamp = FastDateFormat.getInstance(\"yyyy-MM-dd HH:mm:ss\").parse(startTime).getTime();\r\n        sql = sql+\" and create_time > \" + startTimestamp;\r\n    }\r\n    Long endTimestamp = null;\r\n    if(StringUtils.hasText(endTime)){\r\n        endTimestamp = FastDateFormat.getInstance(\"yyyy-MM-dd HH:mm:ss\").parse(endTime).getTime();\r\n        sql = sql+\" and create_time < \" + endTimestamp;\r\n    }\r\n    def map = [:];\r\n    if(startTimestamp != null && endTime != null){\r\n        if(startTimestamp.compareTo(endTimestamp) > 0){\r\n            logger.error(\"开始时间：{}不能大于结束时间：{}\", startTime, endTime);\r\n            map.put(\"message\",\"开始时间：\"+startTime+\"不能大于结束时间：{}\"+endTime);\r\n            return map;\r\n        }\r\n    }\r\n    if(caseId != null){\r\n        sql = sql+\" and case_id  = \" + caseId;\r\n    }\r\n    def lastPercent = 0, currentIndex = 0;\r\n    //查询批次信息\r\n    def deleteFileNum = 0;\r\n    def fileTotal = 0;\r\n    def subCaseInfos =   jdbcTemplate.queryForList(sql);\r\n\r\n    if(subCaseInfos == null || subCaseInfos.size() == 0){\r\n        logger.error(\"没有查询到失效批次\",);\r\n        map.put(\"fileTotal\",fileTotal);\r\n        map.put(\"deleteFileNum\",deleteFileNum);\r\n        return map;\r\n    }\r\n    MinioClient minioClient = commonService.getStorageClient(caseId);\r\n    def amount = subCaseInfos.size();\r\n    for (def i = 0 ; i<amount ; i++) {\r\n        try{\r\n            def _caseId = subCaseInfos[i].get(\"caseId\");\r\n            def subCaseId = subCaseInfos[i].get(\"subCaseId\");\r\n            def fileInfos = fileInfoService.queryByCaseIdAndSubCaseId(_caseId, new String[]{subCaseId}, false, null);\r\n            def storageFiles = storageFileService.queryByCaseId(_caseId);\r\n            for (def j = 0 ; j<fileInfos.size() ; j++) {\r\n                def fileId = fileInfos[j].getFileId();\r\n                def list = storageFiles.stream().filter(storageFile ->\r\n                        storageFile.getFileInfoId().compareTo(fileId) == 0).collect(Collectors.toList());\r\n                if(!ObjectUtils.isEmpty(list)){\r\n                    for (StorageFile storageFile : list) {\r\n                        def filePath = storageFile.getStoragePath();\r\n                        if(StringUtils.hasText(filePath)){\r\n                            logger.info(\"开始删除caseId：{}，fileId:{}的拓展文件\", _caseId, fileId);\r\n                            fileTotal++;\r\n                            if (deleteFile(minioClient, filePath, _caseId)) {\r\n                                deleteFileNum++;\r\n                                logger.info(\"开始删除文件id:{}该文件数据库数据\", fileId);\r\n                                storageFileService.removeByIdAndCaseId(storageFile.getId(), _caseId);\r\n                            }\r\n                        }\r\n                    }\r\n                }\r\n                //获取原文件\r\n                if(!ObjectUtils.isEmpty(fileInfos[j].getExtra())){\r\n                    //删除原图是否成功标志\r\n                    boolean deleteOriflag = true;\r\n                    //删除预处理图是否成功标志\r\n                    boolean deletePreflag = true;\r\n                    def filePath = fileInfos[j].getExtra().at(\"/_original_file_path\") instanceof NullNode ? null\r\n                            : fileInfos[j].getExtra().at(\"/_original_file_path\").asText();\r\n                    if (StringUtils.hasText(filePath)) {\r\n                        fileTotal++;\r\n                        logger.info(\"开始删除caseId：{}，fileId:{}的原文件\", _caseId, fileId);\r\n                        if (deleteOriflag = deleteFile(minioClient, filePath, _caseId)) {\r\n                            deleteFileNum++;\r\n                        }\r\n                    }\r\n                    def preFilePath = fileInfos[j].getExtra().at(\"/_pre_file_path\") instanceof NullNode ? null\r\n                            : fileInfos[j].getExtra().at(\"/_pre_file_path\").asText();\r\n                    if (StringUtils.hasText(preFilePath)) {\r\n                        fileTotal++;\r\n                        logger.info(\"开始删除caseId：{}，fileId:{}的预处理文件\", _caseId, fileId);\r\n                        if (deletePreflag = deleteFile(minioClient, preFilePath, _caseId)) {\r\n                            deleteFileNum++;\r\n                        }\r\n                    }\r\n                    if(deleteOriflag && deletePreflag){\r\n                        logger.info(\"开始删除该文件id:{}数据库数据\", fileId);\r\n                        jdbcTemplate.execute(\"delete from file_info where case_id = \" +_caseId+ \" and file_id = \" +fileId);\r\n                    }\r\n                }\r\n            }\r\n        }catch (Exception e){\r\n            logger.error(\"\", e);\r\n        }\r\n        //打印进度\r\n        def currentPercent = Math.round(((++currentIndex) / (float) amount) * 100);\r\n        if (lastPercent != currentPercent) {\r\n            logger.info(\"进度:{}%\", currentPercent);\r\n            lastPercent = currentPercent;\r\n        }\r\n    }\r\n    map.put(\"fileTotal\",fileTotal);\r\n    map.put(\"deleteFileNum\",deleteFileNum);\r\n    return map\r\n}\r\n\r\ndef deleteFile(MinioClient minioClient, String filePath, Long caseId) throws InvalidBucketNameException, InsufficientDataException, ErrorResponseException, IOException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {\r\n    //判断存储中是否存在该文件\r\n    Logger logger = LogManager.getLogger();\r\n    int index = filePath.indexOf(\":\");\r\n    if(index != -1){\r\n        logger.info(\"删除filePath:{}地址文件为多存储头，不删除\",filePath);\r\n        return false;\r\n    }\r\n    FileInfoServiceImpl fileInfoService = SpringUtil.getBean(FileInfoServiceImpl.class);\r\n    if(fileInfoService.getFileSize(caseId, filePath) != 0){\r\n        logger.info(\"删除filePath:{}的文件 -- start\",filePath);\r\n        String[] filePaths = fileInfoService.parseMinioPath(filePath);\r\n        if(filePaths == null){\r\n            logger.error(\"filePath:{}解析失败\", filePath);\r\n            return false;\r\n        }\r\n        minioClient.removeObject(filePaths[0],filePaths[1]);\r\n        logger.info(\"删除filePath:{}的文件 -- end\", filePath);\r\n        return true;\r\n    }else{\r\n        logger.error(\"filePath:{}的文件不存在\", filePath);\r\n        return true;\r\n    }\r\n}', 0, '2023-10-30 16:16:24', '2023-10-31 15:13:26', 0, NULL, NULL);

添加之后，清除缓存

2、调用执行脚本接口

打开swagger，搜索 script

修改接口，为/caseinfo/script/deleteInvalidBatchFile

参数：

{
    "caseId":1784852276322021377,
    "startTime":"2025-03-13 00:00:00",
    "endTime":"2025-03-13 23:59:59"
}

1、筛选某段时间内 caseId:

2、筛选时间大于 2025-03-13 15:10:00 时间的 caseId:

3、筛选时间小于 2025-03-13 00:00:00 时间的 caseId:

4、筛选某段时间内所有案件：

3、返回数据

{
  "fileTotal": 4,
  "deleteFileNum": 4
}

• fileTotal：失效批次下的文件总数
• deleteFileNum： 实际删除的文件总数