aws-sdk-java-v2-s3

Overview

Provides patterns for S3 operations: bucket management, object upload/download with multipart support, presigned URLs, S3 Transfer Manager, and S3-specific configurations using AWS SDK for Java 2.x.

When to Use

• Creating, listing, or deleting S3 buckets with proper configuration
• Uploading or downloading objects from S3 with metadata and encryption
• Working with multipart uploads for large files (>100MB) with error handling
• Generating presigned URLs for temporary access to S3 objects
• Copying or moving objects between S3 buckets with metadata preservation
• Setting object metadata, storage classes, and access controls
• Implementing S3 Transfer Manager for optimized file transfers
• Integrating S3 with Spring Boot applications for cloud storage

Quick Reference

Storage Classes

Instructions

1. Add Dependencies

<dependency>    <groupId>software.amazon.awssdk</groupId>    <artifactId>s3</artifactId>    <version>2.20.0</version></dependency><dependency>    <groupId>software.amazon.awssdk</groupId>    <artifactId>s3-transfer-manager</artifactId>    <version>2.20.0</version></dependency>

2. Create S3 Client

S3Client s3Client = S3Client.builder()    .region(Region.US_EAST_1)    .build();// With retry logicS3Client s3Client = S3Client.builder()    .region(Region.US_EAST_1)    .overrideConfiguration(b -> b        .retryPolicy(RetryPolicy.builder()            .numRetries(3)            .build()))    .build();

3. Create Bucket

CreateBucketRequest request = CreateBucketRequest.builder()    .bucket(bucketName)    .build();s3Client.createBucket(request);// Wait until readys3Client.waiter().waitUntilBucketExists(    HeadBucketRequest.builder().bucket(bucketName).build());

4. Upload Object

PutObjectRequest request = PutObjectRequest.builder()    .bucket(bucketName)    .key(key)    .contentType("application/pdf")    .serverSideEncryption(ServerSideEncryption.AES256)    .storageClass(StorageClass.STANDARD_IA)    .build();s3Client.putObject(request, RequestBody.fromFile(Paths.get(filePath)));// Validate upload completionHeadObjectResponse headResp = s3Client.headObject(HeadObjectRequest.builder()    .bucket(bucketName)    .key(key)    .build());

5. Download Object

GetObjectRequest request = GetObjectRequest.builder()    .bucket(bucketName)    .key(key)    .build();s3Client.getObject(request, Paths.get(destPath));

6. Generate Presigned URL

try (S3Presigner presigner = S3Presigner.create()) {    GetObjectRequest getRequest = GetObjectRequest.builder()        .bucket(bucketName)        .key(key)        .build();    GetObjectPresignRequest presignRequest = GetObjectPresignRequest.builder()        .signatureDuration(Duration.ofMinutes(10))        .getObjectRequest(getRequest)        .build();    String url = presigner.presignGetObject(presignRequest).url().toString();}

7. Use Transfer Manager (Large Files)

try (S3TransferManager tm = S3TransferManager.create()) {    UploadFileRequest request = UploadFileRequest.builder()        .putObjectRequest(req -> req.bucket(bucketName).key(key))        .source(Paths.get(filePath))        .build();    FileUpload upload = tm.uploadFile(request);    CompletedFileUpload result = upload.completionFuture().join();}

Best Practices

Performance

• Use S3 Transfer Manager: Automatic multipart uploads for files >100MB
• Reuse S3 Client: Clients are thread-safe; reuse throughout application
• Enable async operations: Use S3AsyncClient for I/O-bound operations
• Configure timeouts: Set appropriate timeouts for large file operations

Security

• Use temporary credentials: IAM roles or AWS STS for short-lived tokens
• Enable encryption: Use AES-256 or AWS KMS for sensitive data
• Use presigned URLs: Avoid exposing credentials with temporary access
• Validate metadata: Sanitize user-provided metadata

Error Handling

• Implement retry logic: Exponential backoff for network operations
• Handle throttling: Proper handling of 429 responses
• Clean up failures: Abort failed multipart uploads

Cost Optimization

• Use appropriate storage classes: STANDARD, STANDARD_IA, INTELLIGENT_TIERING
• Implement lifecycle policies: Automatic transition/expiration
• Minimize API calls: Use batch operations when possible

Constraints and Warnings

• Object Size: Single PUT limited to 5GB; use multipart for larger files
• Bucket Names: Must be globally unique across all AWS accounts
• Object Immutability: Objects cannot be modified; must be replaced entirely
• Eventual Consistency: List operations may have slight delays after uploads
• Presigned URLs: Maximum expiration time is 7 days
• Multipart Uploads: Parts must be at least 5MB except last part

Examples

Complete Upload Workflow with Validation

// 1. Upload with validationPutObjectRequest putRequest = PutObjectRequest.builder()    .bucket(bucketName)    .key(key)    .contentType(contentType)    .build();s3Client.putObject(putRequest, RequestBody.fromFile(Paths.get(filePath)));// 2. Validate with headObjectHeadObjectResponse headResp = s3Client.headObject(HeadObjectRequest.builder()    .bucket(bucketName)    .key(key)    .build());// 3. Verify metadatalong fileSize = Files.size(Paths.get(filePath));if (headResp.contentLength() != fileSize) {    throw new IllegalStateException("Upload size mismatch");}

Multipart Upload with Abort-on-Failure

// 1. Initiate multipart uploadCreateMultipartUploadRequest createRequest = CreateMultipartUploadRequest.builder()    .bucket(bucketName)    .key(key)    .build();CreateMultipartUploadResponse multipartUpload = s3Client.createMultipartUpload(createRequest);String uploadId = multipartUpload.uploadId();try {    // 2. Upload parts    List<CompletedPart> parts = new ArrayList<>();    int partNumber = 1;    byte[] fileBytes = Files.readAllBytes(Paths.get(filePath));    int chunkSize = 5 * 1024 * 1024; // 5MB minimum    for (int offset = 0; offset < fileBytes.length; offset += chunkSize) {        int length = Math.min(chunkSize, fileBytes.length - offset);        UploadPartRequest uploadPartRequest = UploadPartRequest.builder()            .bucket(bucketName)            .key(key)            .uploadId(uploadId)            .partNumber(partNumber)            .build();        UploadPartResponse partResponse = s3Client.uploadPart(uploadPartRequest,            RequestBody.fromBytes(Arrays.copyOfRange(fileBytes, offset, offset + length)));        parts.add(CompletedPart.builder()            .partNumber(partNumber)            .eTag(partResponse.eTag())            .build());        partNumber++;    }    // 3. Complete multipart upload    CompleteMultipartUploadRequest completeRequest = CompleteMultipartUploadRequest.builder()        .bucket(bucketName)        .key(key)        .uploadId(uploadId)        .multipartUpload(CompletedMultipartUpload.builder().parts(parts).build())        .build();    s3Client.completeMultipartUpload(completeRequest);} catch (Exception e) {    // 4. Abort on failure    AbortMultipartUploadRequest abortRequest = AbortMultipartUploadRequest.builder()        .bucket(bucketName)        .key(key)        .uploadId(uploadId)        .build();    s3Client.abortMultipartUpload(abortRequest);    throw new RuntimeException("Upload failed, cleanup performed", e);}

References

• references/s3-client-setup.md — Client configuration and basic operations
• references/s3-object-operations.md — Advanced object operations
• references/s3-transfer-patterns.md — Transfer Manager and multipart uploads
• references/s3-spring-boot-integration.md — Spring Boot integration patterns
• AWS S3 Developer Guide
• AWS SDK for Java 2.x S3 API

Related Skills

• aws-sdk-java-v2-core - Core AWS SDK patterns and configuration
• spring-boot-dependency-injection - Spring dependency injection patterns