This adds a middleware for overload protection that is intended to help protect against malicious scrapers.
It does this via [`codel`](https://github.com/bohde/codel), which will perform the following:
1. Limit the number of in-flight requests to some user-defined max
2. When in-flight requests have reached their begin queuing requests.
Logged-in requests having priority above logged-out requests
3. Once a request has been queued for too long,
it has a probabilistic chance to be rejected based on how overloaded the entire system is.
When a server experiences more traffic than it can handle,
this keeps latency low for logged-in users and rejects just
enough requests from logged-out users to not overload the service.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
This PR uniform all temporary directory usage so that it will be easier
to manage.
Relate to #31792
- [x] Added a new setting to allow users to configure the global
temporary directory.
- [x] Move all temporary files and directories to be placed under
os.Temp()/gitea.
- [x] `setting.Repository.Local.LocalCopyPath` now will be
`setting.TempPath/local-repo` and the customized path is removed.
```diff
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;[repository.local]
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;;
-;; Path for local repository copy. Defaults to TEMP_PATH + `local-repo`, this is deprecated and cannot be changed
-;LOCAL_COPY_PATH = local-repo
```
- [x] `setting.Repository.Upload.TempPath` now will be
`settting.TempPath/uploads` and the customized path is removed.
```diff
;[repository.upload]
-;;
-;; Path for uploads. Defaults to TEMP_PATH + `uploads`
-;TEMP_PATH = uploads
```
- [x] `setting.Packages.ChunkedUploadPath` now will be
`settting.TempPath/package-upload` and the customized path is removed.
```diff
;[packages]
-;;
-;; Path for chunked uploads. Defaults it's `package-upload` under `TEMP_PATH` unless it's an absolute path.
-;CHUNKED_UPLOAD_PATH = package-upload
```
- [x] `setting.SSH.KeyTestPath` now will be
`settting.TempPath/ssh_key_test` and the customized path is removed.
```diff
[server]
-;;
-;; Directory to create temporary files in when testing public keys using ssh-keygen,
-;; default is the system temporary directory.
-;SSH_KEY_TEST_PATH =
```
TODO:
- [ ] setting.PprofDataPath haven't been changed because it may need to
be kept until somebody read it but temp path may be clean up any time.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fix#33966
```
;; User must sign in to view anything.
;; It could be set to "expensive" to block anonymous users accessing some pages which consume a lot of resources,
;; for example: block anonymous AI crawlers from accessing repo code pages.
;; The "expensive" mode is experimental and subject to change.
;REQUIRE_SIGNIN_VIEW = false
```
In history, Gitea could use the system config `/etc/gitconfig` because
some users said that "they might put certNonceSeed in it"
Actually, we shouldn't not use it, because it also causes conflicts
(there are already some fixes like #28848)
To make the system clear, I think it's worth to introduce the breaking
change: add `GIT_CONFIG_NOSYSTEM=1` to all git commands.
## ⚠️ BREAKING ⚠️
For most users, nothing need to do.
If you have made changes to `/etc/gitconfig` to affect Gitea's behavior,
you need to move these config options to Gitea's internal git config
file, it is usually in Gitea's `{AppDataPath}/home/.git` directory.
unset XDG_CONFIG_HOME early to enable gitea to manage git configuration.
simple error checking to satisfy the linting. Closes#33039
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Closes#25898
The problem was that the default settings weren't being loaded
---------
Signed-off-by: cassiozareck <cassiomilczareck@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
In history (from some legacy frameworks), both `:name` and `name` are
supported as path path name, `:name` is an alias to `name`.
To make code consistent, now we should only use `name` but not `:name`.
Also added panic check in related functions to make sure the name won't
be abused in case some downstreams still use them.
Usually enterprise/organization users would like to only allow OAuth2
login.
This PR adds a new config option to disable the password-based login
form. It is a simple and clear approach and won't block the future
login-system refactoring works.
Fix a TODO in #24821
Replace #21851Close#7633 , close#13606
Targeting issue #32271
This modification allows native Kubernetes + AWS (EKS) authentication
with the Minio client, to Amazon S3 using the IRSA role assigned to a
Service account by replacing the hard coded reference to the
`DefaultIAMRoleEndpoint` with an optional configurable endpoint.
Internally, Minio's `credentials.IAM` provider implements a discovery
flow for IAM Endpoints if it is not set.
For backwards compatibility:
- We have added a configuration mechanism for an `IamEndpoint` to retain
the unit test safety in `minio_test.go`.
- We believe existing clients will continue to function the same without
needing to provide a new config property since the internals of Minio
client also often resolve to the `http://169.254.169.254` default
endpoint that was being hard coded before
To test, we were able to build a docker image from source and, observe
it choosing the expected IAM endpoint, and see files uploaded via the
client.
Remove unmaintainable sanitizer rules. No need to add special "class"
regexp rules anymore, use RenderInternal.SafeAttr instead, more details
(and examples) are in the tests
Resolve#30350
The action unit of mirrors and templates should be disabled by default.
This PR adds `DEFAULT_MIRROR_REPO_UNITS` and
`DEFAULT_TEMPLATE_REPO_UNITS` options to allow users to specify default
units for mirrors and templates.
Thanks to @lng2020 for the
[idea](https://github.com/go-gitea/gitea/issues/30350#issuecomment-2053942243)
This contains two backwards-compatible changes:
* in the lfs http_client, the number of lfs oids requested per batch is
loaded from lfs_client#BATCH_SIZE and defaulted to the previous value of
20
* in the lfs server/service, the max number of lfs oids allowed in a
batch api request is loaded from server#LFS_MAX_BATCH_SIZE and defaults
to 'nil' which equates to the previous behavior of 'infinite'
This fixes#32306
---------
Signed-off-by: Royce Remer <royceremer@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
These settings can allow users to only display the repositories explore page.
Thanks to yp05327 and wxiaoguang !
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Close#31801. Follow #31761.
Since there are so many benefits of compression and there are no reports
of related issues after weeks, it should be fine to enable compression
by default.
This will allow instance admins to view signup pattern patterns for
public instances. It is modelled after discourse, mastodon, and
MediaWiki's approaches.
Note: This has privacy implications, but as the above-stated open-source
projects take this approach, especially MediaWiki, which I have no doubt
looked into this thoroughly, it is likely okay for us, too. However, I
would be appreciative of any feedback on how this could be improved.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
https://github.com/go-fed/httpsig seems to be unmaintained.
Switch to github.com/42wim/httpsig which has removed deprecated crypto
and default sha256 signing for ssh rsa.
No impact for those that use ed25519 ssh certificates.
This is a breaking change for:
- gitea.com/gitea/tea (go-sdk) - I'll be sending a PR there too
- activitypub using deprecated crypto (is this actually used?)
Support compression for Actions logs to save storage space and
bandwidth. Inspired by
https://github.com/go-gitea/gitea/issues/24256#issuecomment-1521153015
The biggest challenge is that the compression format should support
[seekable](https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/zstd_seekable_compression_format.md).
So when users are viewing a part of the log lines, Gitea doesn't need to
download the whole compressed file and decompress it.
That means gzip cannot help here. And I did research, there aren't too
many choices, like bgzip and xz, but I think zstd is the most popular
one. It has an implementation in Golang with
[zstd](https://github.com/klauspost/compress/tree/master/zstd) and
[zstd-seekable-format-go](https://github.com/SaveTheRbtz/zstd-seekable-format-go),
and what is better is that it has good compatibility: a seekable format
zstd file can be read by a regular zstd reader.
This PR introduces a new package `zstd` to combine and wrap the two
packages, to provide a unified and easy-to-use API.
And a new setting `LOG_COMPRESSION` is added to the config, although I
don't see any reason why not to use compression, I think's it's a good
idea to keep the default with `none` to be consistent with old versions.
`LOG_COMPRESSION` takes effect for only new log files, it adds `.zst` as
an extension to the file name, so Gitea can determine if it needs
decompression according to the file name when reading. Old files will
keep the format since it's not worth converting them, as they will be
cleared after #31735.
<img width="541" alt="image"
src="https://github.com/user-attachments/assets/e9598764-a4e0-4b68-8c2b-f769265183c9">
close #27031
If the rpm package does not contain a matching gpg signature, the
installation will fail. See (#27031) , now auto-signing rpm uploads.
This option is turned off by default for compatibility.
Make it posible to let mails show e.g.:
`Max Musternam (via gitea.kithara.com) <gitea@kithara.com>`
Docs: https://gitea.com/gitea/docs/pulls/23
---
*Sponsored by Kithara Software GmbH*
We have some instances that only allow using an external authentication
source for authentication. In this case, users changing their email,
password, or linked OpenID connections will not have any effect, and
we'd like to prevent showing that to them to prevent confusion.
Included in this are several changes to support this:
* A new setting to disable user managed authentication credentials
(email, password & OpenID connections)
* A new setting to disable user managed MFA (2FA codes & WebAuthn)
* Fix an issue where some templates had separate logic for determining
if a feature was disabled since it didn't check the globally disabled
features
* Hide more user setting pages in the navbar when their settings aren't
enabled
---------
Co-authored-by: Kyle D <kdumontnu@gmail.com>
Fix#31361, and add tests
And this PR introduces an undocumented & debug-purpose-only config
option: `USE_SUB_URL_PATH`. It does nothing for end users, it only helps
the development of sub-path related problems.
And also fix#31366
Co-authored-by: @ExplodingDragon
