C# gRPC 文件流式传输，原始文件比流式文件小

Question

我在设置请求流类型 gRPC 架构时遇到一些问题。下面的代码仅用于测试目的，缺少各种验证检查，但主要问题是原始文件始终小于收到的文件。

这里的原因可能是编码吗？无论文件类型是什么，最终结果总是文件大小不同。

Protobuf接口：

syntax = "proto3";
package FileTransfer;
option csharp_namespace = "FileTransferProto";

service FileTransferService {  
    rpc DownloadFile(FileRequest) returns (stream ChunkMsg);
}  

message ChunkMsg {
    string FileName = 1;
    int64 FileSize = 2;
    bytes Chunk = 3;
}

message FileRequest {
    string FilePath = 1;
}

服务器端（发送）：

    public override async Task DownloadFile(FileRequest request, IServerStreamWriter<ChunkMsg> responseStream, ServerCallContext context)
    {
        string filePath = request.FilePath;

        if (!File.Exists(filePath)) { return; }

        FileInfo fileInfo = new FileInfo(filePath);

        ChunkMsg chunk = new ChunkMsg();
        chunk.FileName = Path.GetFileName(filePath);
        chunk.FileSize = fileInfo.Length;

        int fileChunkSize = 64 * 1024;

        byte[] fileByteArray = File.ReadAllBytes(filePath);
        byte[] fileChunk = new byte[fileChunkSize];
        int fileOffset = 0;

        while (fileOffset < fileByteArray.Length && !context.CancellationToken.IsCancellationRequested)
        {
            int length = Math.Min(fileChunkSize, fileByteArray.Length - fileOffset);
            Buffer.BlockCopy(fileByteArray, fileOffset, fileChunk, 0, length);
            fileOffset += length;
            ByteString byteString = ByteString.CopyFrom(fileChunk);

            chunk.Chunk = byteString;
            await responseStream.WriteAsync(chunk).ConfigureAwait(false);
        }            
    }

客户端（接收）：

    public static async Task GetFile(string filePath)
    {
        var channel = Grpc.Net.Client.GrpcChannel.ForAddress("https://localhost:5001/", new GrpcChannelOptions
        {
            MaxReceiveMessageSize = 5 * 1024 * 1024, // 5 MB
            MaxSendMessageSize = 5 * 1024 * 1024, // 5 MB
        });

        var client = new FileTransferProto.FileTransferService.FileTransferServiceClient(channel);

        var request = new FileRequest { FilePath = filePath };
        string tempFileName = $"temp_{DateTime.UtcNow.ToString("yyyyMMdd_HHmmss")}.tmp";
        string finalFileName = tempFileName;

        using (var call = client.DownloadFile(request))
        {
            await using (Stream fs = File.OpenWrite(tempFileName))
            {
                await foreach (ChunkMsg chunkMsg in call.ResponseStream.ReadAllAsync().ConfigureAwait(false))
                {
                    Int64 totalSize = chunkMsg.FileSize;
                    string tempFinalFilePath = chunkMsg.FileName;

                    if (!string.IsNullOrEmpty(tempFinalFilePath))
                    {
                        finalFileName = chunkMsg.FileName;
                    }

                    fs.Write(chunkMsg.Chunk.ToByteArray());
                }
            }
        }

        if (finalFileName != tempFileName)
        {
            File.Move(tempFileName, finalFileName);
        }
    }

Answer 1

为了补充马克的答案，我觉得你可以稍微简化你的代码。

using var fs = File.Open(filePath, System.IO.FileMode.Open);
int bytesRead;
var buffer = new byte[fileChunkSize];
while ((bytesRead = await fs.ReadAsync(buffer)) > 0)
{
     await call.RequestStream.WriteAsync(new ChunkMsg
     {
          // Here the correct number of bytes must be sent which is starting from
          // index 0 up to the number of read bytes from the file stream.
          // If you solely pass 'buffer' here, the same bug would be present.
          Chunk = ByteString.CopyFrom(buffer[0..bytesRead]),
     });
}

我使用了 C# 8.0 中的数组范围运算符，这使得它更清晰，或者您也可以使用它的重载，ByteString.CopyFrom该重载接受偏移量和要包含的字节数。

Answer 2

在您的写入循环中，您实际发送的块是针对超大缓冲区的，而不是考虑length。这意味着最后一个段包含一些垃圾并且尺寸过大。接收到的有效负载将增大同样的量。因此：请确保length在构建要发送的块时考虑到这一点。