fix: set a minio part size when the content size is unknown (#38753)

## Background

`MinioStorage.Save` is called with `size = -1` on several paths,
including Actions logs, Actions artifacts, repository archives, avatars
and attachments. With an unknown size (-1) minio-go assumes a 5TiB
object and allocates a single part-sized buffer of 528MiB per upload,
regardless of the real payload size, which can exhaust the memory of
small instances.

Measured against a local S3 stub, five sequential uploads of a 4KiB
payload grew RSS by 1041MiB before the fix and by 34MiB after it.

## Fix

Pass an explicit 16MiB part size in that case, the same value minio-go
uses as minimum part size
(https://github.com/minio/minio-go/blob/v7.2.1/constants.go#L28).

Uploads with a known size are left untouched, since minio-go already
derives a part size proportional to the real object size.

## Note:

One behaviour change: with an unknown size the object is now limited to
16MiB * 10000 parts = 156.25GiB
(https://github.com/minio/minio-go/blob/v7.2.1/api-put-object-common.go#L112-L116).
`putObjectMultipartStreamNoLength` completes the upload once it runs out
of parts without checking that the reader was drained, so a stream past
that limit is silently truncated rather than rejected. The previous
limit was 5TiB. No payload Gitea uploads comes close to either.

Parts are also uploaded serially, so a smaller part size means
proportionally more round trips for large unknown-size uploads.

---------

Co-authored-by: silverwind <me@silverwind.io>
This commit is contained in:
Zettat123
2026-08-02 23:47:43 +00:00
committed by GitHub
co-authored by silverwind
parent fdc1d613da
commit a3b747801c
+6
View File
@@ -26,6 +26,8 @@ import (
var _ ObjectStorage = &MinioStorage{}
const unknownSizePartSize = 1024 * 1024 * 16 // same as minio-go's minPartSize
type minioObject struct {
*minio.Object
}
@@ -228,6 +230,10 @@ func (m *MinioStorage) Save(path string, r io.Reader, size int64) (int64, error)
// * https://www.backblaze.com/b2/docs/s3_compatible_api.html
// do not support "x-amz-checksum-algorithm" header, so use legacy MD5 checksum
SendContentMd5: m.cfg.ChecksumAlgorithm == "md5",
// with an unknown size (-1) minio-go assumes a 5TiB object and buffers a 528MiB part for it, even
// for a payload of a few KiB, so pin the part size there, a known size derives its own
PartSize: util.Iif[uint64](size < 0, unknownSizePartSize, 0),
},
)
if err != nil {