要上传到S3的批处理文件

Question

我需要开发一个批处理文件,它会将多个小文本文件(4KB - 100KB)发送到S3存储桶中的文件夹.受到来自http://tmont.com/blargh/2014/1/uploading-to-s3-in-bash的bash脚本的启发,我开发了下面列出的批处理脚本.此脚本使用以下依赖项:

• openssl.exe(以及两个库libeay32.dll和ssleay32.dll)

• b64.exe(https://sourceforge.net/projects/base64/)

• curl.exe

• grep.exe

效果很好; 但是,有时文件或多个文件不会上传,我会收到"HTTP/1.1 403 Forbidden/SignatureDoesNotMatch."这似乎是签名问题,可能是在创建签名时.这种行为似乎没有一种模式.为了测试,我使用8个文本文件进行上传.我可以多次运行它,导致每次都上传所有文件,下次脚本只会上传5,6或7个.我有点不知道为什么会发生这种情况.任何建议,将不胜感激.

@echo off
setlocal enableDelayedExpansion

for /f %%f in ('dir /b *.txt') do (

REM Set date/time variables
for /f "tokens=1-4 delims=/ " %%a in ('date /t') do set dow=%%a&&set day=%%c&& set month=%%b&&set year=%%d
for /f "tokens=1-4 delims=:,. " %%h in ('echo %time%') do set hour=%%h&set min=%%i&set sec1=%%j&set sec2=%%k

REM Obtain three letter month value for DateValue HTTP Header
IF !month! == 01 set mname=Jan
IF !month! == 02 set mname=Feb
IF !month! == 03 set mname=Mar
IF !month! == 04 set mname=Apr
IF !month! == 05 set mname=May
IF !month! == 06 set mname=Jun
IF !month! == 07 set mname=Jul
IF !month! == 08 set mname=Aug
IF !month! == 09 set mname=Sep
IF !month! == 10 set mname=Oct
IF !month! == 11 set mname=Nov
IF !month! == 12 set mname=Dec

REM Set time as a 4 digit value if leading zero does not exist
for /f "tokens=1" %%u in ('echo %time%') do set t=%%u
IF "!hour:~1,1!"==":" set t=0!hour!

REM Obtain the ActiveBias value and convert to decimal 
for /f "tokens=3" %%a in ('reg query HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\TimeZoneInformation /v ActiveTimeBias ^| grep -i "ActiveTimeBias"') do set /a abias=%%a

REM Set the + or - sign variable to reflect the timezone offset
IF "!abias:~0,1!"=="-" (set si=+) ELSE (set si=-)
for /f "tokens=1 delims=-" %%t in ('echo !abias!') do set tzc=%%t

REM Calculate to obtain floating points (decimal values)
set /a tzd=100*!tzc!/60

REM Calculate the active bias to obtain the hour
set /a tze=!tzc!/60

REM Set the minutes based on the result of the floating point calculation
IF "!tzd!"=="0" (set en=00 && set si=)
IF "!tzd:~1!"=="00" (set en=00) ELSE IF "!tzd:~2!"=="00" (set en=00 && set tz=!tzd:~0,2!) 
IF "!tzd:~1!"=="50" (set en=30) ELSE IF "!tzd:~2!"=="50" (set en=30 && set tz=!tzd:~0,2!) 
IF "!tzd:~1!"=="75" (set en=45) ELSE IF "!tzd:~2!"=="75" (set en=45 && set tz=!tzd:~0,2!) 

REM Adding a 0 to the beginning of a single digit hour value
IF !tze! LSS 10 (set tz=0!tze!) 

REM Set the date/timestamp to meet required format
set dateValue=!dow!, !day! !mname! !year! !hour!:!min!:!sec1! !si!!tz!!en!

REM Preparing the HTTP header field
set file=%%f
set bucket=yourbucketname
set resource=/!bucket!/upload/!file!
set contentType=text/plain

REM You MUST have two returns after set NL=^
setlocal enableDelayedExpansion
set NL=^


set stringToSign=PUT!NL!!NL!!contentType!!NL!!dateValue!!NL!!resource!
<nul set /p ".=!stringToSign!" > put.tmp

set S3KEY="Your S3 Key Here"
set S3SECRET="Your S3 Secrect Key Here"

for /f "tokens=*" %%a in ('type put.tmp ^| openssl sha1 -hmac !S3SECRET! -binary ^| b64 -e') do set signature=%%a

REM Sending the data
curl -vvv --no-alpn --http2 -1 -S -X PUT -T "!file!" -H "Host: !bucket!.s3.amazonaws.com" -H "Date: !dateValue!" -H "Content-Type: !contentType!" -H "Authorization: AWS !S3KEY!:!signature!" "https://!bucket!.s3.amazonaws.com/upload/!file!"

REM Reset Variables
set "day="
set "month="
set "year="
set "retry="
set "dow="
set "hour="
set "min="
set "sec1="
set "sec2="
set "dateValue="
set "file="
set "mname="
set "bucket="
set "resource="
set "contentType="
set "stringToSign="
set "S3KEY="
set "S3SECRET="
set "t="

del put.tmp

)

Answer 1

为什么不使用具有明确S3支持的Windows命令行客户端？无论如何,您的批处理文件离不开第三方依赖项.

亚马逊有自己的aws-cli.

如果您需要更轻量级的解决方案,请使用WinSCP:

winscp.com /log=s3.log /command ^
    "open s3://%S3KEY%:%S3SECRET%@s3.amazonaws.com/" ^
    "put *.txt /%BUCKET%/" ^
    "exit"

您需要对凭据中的特殊字符进行URL编码.WinSCP GUI可以为您生成一个S3脚本模板,如上所述.

(我是WinSCP的作者)