IAV-86/feat: Implement comprehensive file manager service with batch operations and advanced transaction management
Description
This MR introduces the iavia-file-manager-service - a complete file management microservice with AWS S3 storage, advanced transaction management, and comprehensive audit capabilities.
Key Features
🚀 Core Functionality
- Multi-file upload with batch support and validation (extension, size, MIME type)
- Agent Family KB specialized upload endpoint
- File download with streaming and presigned/public URL generation
- Metadata management (single, batch, existence check)
- Smart deletion (single with rollback, bulk with detailed tracking)
- Search & listing (by usage type, uploader, folder, name)
- Comprehensive audit with FileAccessLog for all operations
🔒 Transaction Management (2-Phase)
- Upload: S3 first → DB save → Auto cleanup orphaned files on failure
- Delete: DB delete → S3 delete → Auto rollback on S3 failure
-
Bulk delete: Track fully deleted, DB-only deleted (
⚠ ️), and failed files
📊 Result DTOs
- FileUploadResultDTO: Success/failure breakdown
- BulkDeleteResultDTO: Three categories (fully deleted, DB-only, failed) + failure reasons
- PresignedUrlDTO: Temporary secure access
- FileDownloadDTO: Download info with Resource
Code Quality
Refactoring
-
✅ **~300 lines eliminated through centralized helpers -
✅ 2 key helper methods:getFileMetadataOrThrow,getS3KeyFromMetadata -
✅ Enhanced S3 cleanup: Continues even if individual deletions fail -
✅ Proper transactions:@TransactionalwithREQUIRES_NEWfor audit logs
Documentation
- FILE_MANAGER_TECHNICAL_DOCUMENTATION.md: Complete service architecture and patterns
- FILE_MANAGER_EXTERNAL_INTEGRATION_GUIDE.md: Step-by-step integration for external services
Configuration (Examples of non-real values for properties Table in config-server)
| Application | Profile | Label | Key | Value |
|---|---|---|---|---|
| iavia-agent-service-customer-care | local | develop | aws.credentials.accessKey | ABCDEFGH |
| iavia-agent-service-customer-care | local | develop | aws.credentials.secretKey | aZ7E8r9t21Y |
| iavia-agent-service-customer-care | local | develop | aws.bucketName | iavia-customer-care-dev |
| iavia-agent-service-customer-care | local | develop | aws.bucketPolicy | classpath:bucketPolicy.json |
| iavia-agent-service-customer-care | local | develop | aws.preSignedUrlExpiration.defaultTTL | 300 |
| iavia-agent-service-customer-care | local | develop | aws.lifecycle.documentsTTL | 30 |
| iavia-agent-service-customer-care | local | develop | aws.lifecycle.imagesTTL | 90 |
| iavia-agent-service-customer-care | local | develop | aws.lifecycle.videosTTL | 60 |
| iavia-agent-service-customer-care | local | develop | aws.max-file-size.images | 10MB |
| iavia-agent-service-customer-care | local | develop | aws.max-file-size.videos | 100MB |
| iavia-agent-service-customer-care | local | develop | aws.max-file-size.documents | 15MB |
| iavia-agent-service-customer-care | local | develop | aws.max-file-size.default-size | 5MB |
| iavia-agent-service-customer-care | local | develop | aws.expiration-action.images | delete |
| iavia-agent-service-customer-care | local | develop | aws.expiration-action.videos | archive |
| iavia-agent-service-customer-care | local | develop | aws.expiration-action.documents | manual_review |
| iavia-agent-service-customer-care | local | develop | aws.allowedExtensions | "jpg,jpeg,png,pdf,docx" |
| iavia-agent-service-customer-care | local | develop | aws.pagination.pageSize | 20 |
Testing
-
Upload (single, batch, with failures) -
Download and URL generation -
Metadata operations -
Delete (single with rollback, bulk with partial failures) -
Search and listing -
Access log recording
Important Notes
dbOnlyDeletedUids is not empty
Closes IAV-86