Amazon S3 Presigned URL从密钥中删除斜杠

Question

我使用Java Amazon SDK与S3一起存储上传的文件.我想保留原始文件名,我把它放在密钥的末尾,但我也使用虚拟目录结构 - 类似于<dirname>/<uuid>/<originalFilename>.

问题是,当我想使用api生成预先下载的URL以供下载时:

URL url = s3Client.generatePresignedUrl(generatePresignedUrlRequest);
return url.toExternalForm();

sdk url转义整个键,包括斜杠.虽然它仍然有效,但这意味着下载文件的名称包括整个密钥,而不仅仅是最后的原始文件名位.我知道应该可以在不转义斜杠的情况下执行此操作,但我正在尝试避免重写SDK中已有的大量代码.这有一个共同的解决方案吗？我知道我使用了遵循相同模式的Web应用程序,并且没有斜杠转义问题.

Answer 1

这是当前Java SDK中的一个错误:

如果你看一下https://github.com/aws/aws-sdk-java/blob/master/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java#L2820

内部调用的方法presignRequest具有以下代码:

    String resourcePath = "/" +
        ((bucketName != null) ? bucketName + "/" : "") +
        ((key != null) ? ServiceUtils.urlEncode(key) : "") +
        ((subResource != null) ? "?" + subResource : "");

在签名之前,密钥是URL编码,我认为是错误.

您可能能够继承AmazonS3Client并覆盖函数来修复此问题.

在某些地方,建议使用url.getQuery()原始awsURL(https://forums.aws.amazon.com/thread.jspa?messageID=356271)并将其作为前缀.但是,正如您所说,这会产生错误,因为资源键与签名不匹配.

以下问题可能也有关系,我没有查看建议的workarround:

如何使用amazon sdk为虚拟域生成预先签名的Amazon S3网址？

亚马逊之前认识并修复了类似的错误:https: //forums.aws.amazon.com/thread.jspa？messageID = 418537

所以我希望它将在下一个版本中修复.

Answer 2

我仍然希望有一个比这更好的解决方案，但看到 @aKzenT 已经证实了我的结论，即没有现有的解决方案，我写了一个。它只是 AmazonS3Client 的一个简单子类。我担心它很脆弱，因为我必须从我覆盖的方法中复制大量代码，但这似乎是最小的解决方案。我可以确认它在我自己的代码库中运行良好。我将代码发布在要点中，但为了完整的答案：

import com.amazonaws.AmazonWebServiceRequest;
import com.amazonaws.HttpMethod;
import com.amazonaws.Request;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.handlers.RequestHandler;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.Headers;
import com.amazonaws.services.s3.internal.S3QueryStringSigner;
import com.amazonaws.services.s3.internal.ServiceUtils;

import java.util.Date;

/**
 * This class should be a drop in replacement for AmazonS3Client as long as you use the single credential
 * constructor. It could probably be modified to add additional constructors if needed, but this is the one we use.
 * Supporting all of them didn't seem trivial because of some dependencies in the original presignRequest method.
 *
 * The only real purpose of this class is to change the behavior of generating presigned URLs. The original version
 * escaped slashes in the key and this one does not. Pretty url paths are kept intact.
 *
 * @author Russell Leggett
 */
public class PrettyUrlS3Client extends AmazonS3Client{
    private AWSCredentials awsCredentials;

    /**
     * This constructor is the only one provided because it is only one I needed, and it
     * retains awsCredentials which might be needed in the presignRequest method
     *
     * @param awsCredentials
     */
    public PrettyUrlS3Client(AWSCredentials awsCredentials) {
        super(awsCredentials);
        this.awsCredentials = awsCredentials;
    }

    /**
     * WARNING: This method is an override of the AmazonS3Client presignRequest
     * and copies most of the code. Should be careful of updates to the original.
     *
     * @param request
     * @param methodName
     * @param bucketName
     * @param key
     * @param expiration
     * @param subResource
     * @param <T>
     */
    @Override
    protected <T> void presignRequest(Request<T> request, HttpMethod methodName, String bucketName, String key, Date expiration, String subResource) {

        // Run any additional request handlers if present
        if (requestHandlers != null) {
            for (RequestHandler requestHandler : requestHandlers) {
                requestHandler.beforeRequest(request);
            }
        }
        String resourcePath = "/" +
                ((bucketName != null) ? bucketName + "/" : "") +
                ((key != null) ? keyToEscapedPath(key)/* CHANGED: this is the primary change */ : "") +
                ((subResource != null) ? "?" + subResource : "");

        //the request apparently needs the resource path without a starting '/'
        request.setResourcePath(resourcePath.substring(1));//CHANGED: needed to match the signature with the URL generated from the request
        AWSCredentials credentials = awsCredentials;
        AmazonWebServiceRequest originalRequest = request.getOriginalRequest();
        if (originalRequest != null && originalRequest.getRequestCredentials() != null) {
            credentials = originalRequest.getRequestCredentials();
        }

        new S3QueryStringSigner<T>(methodName.toString(), resourcePath, expiration).sign(request, credentials);

        // The Amazon S3 DevPay token header is a special exception and can be safely moved
        // from the request's headers into the query string to ensure that it travels along
        // with the pre-signed URL when it's sent back to Amazon S3.
        if (request.getHeaders().containsKey(Headers.SECURITY_TOKEN)) {
            String value = request.getHeaders().get(Headers.SECURITY_TOKEN);
            request.addParameter(Headers.SECURITY_TOKEN, value);
            request.getHeaders().remove(Headers.SECURITY_TOKEN);
        }
    }

    /**
     * A simple utility method which url escapes an S3 key, but leaves the
     * slashes (/) unescaped so they can stay part of the url.
     * @param key
     * @return
     */
    public static String keyToEscapedPath(String key){
        String[] keyParts = key.split("/");
        StringBuilder result = new StringBuilder();
        for(String part : keyParts){
            if(result.length()>0){
                result.append("/");
            }
            result.append(ServiceUtils.urlEncode(part));
        }
        return result.toString().replaceAll("%7E","~");
    }
}

更新我更新了要点和这段代码来解决我遇到的问题。即使使用标准客户端也会发生这种情况，但转义 ~ 修复了它。有关更多详细信息，请参阅要点/跟踪我可能做出的任何进一步更改。