mirror of
https://github.com/go-gitea/gitea.git
synced 2026-04-03 19:03:05 +02:00
23c662ebb1
Follow up #36842 Migration `326` can be prohibitively slow on large instances because it scans and rewrites all commit status target URLs generated by Gitea Actions in the database. This PR refactors migration `326` to perform a partial update instead of rewriting every legacy target URL. The reason for this partial rewrite is that **smaller legacy run/job indexes are the most likely to be ambiguous with run/job ID-based URLs** during runtime resolution, so this change prioritizes that subset while avoiding the cost of rewriting all legacy records. To preserve access to old links, this PR introduces `resolveCurrentRunForView` to handle both ID-based URLs and index-based URLs: - For job pages (`/actions/runs/{run}/jobs/{job}`), it first tries to confirm that the URL is ID-based. It does so by checking whether `{job}` can be treated as an existing job ID in the repository and whether that job belongs to `{run}`. If that match cannot be confirmed, it falls back to treating the URL as legacy `run index + job index`, resolves the corresponding run and job, and redirects to the correct ID-based URL. - When both ID-based and index-based interpretations are valid at the same time, the resolver **prefers the ID-based interpretation by default**. For example, if a repository contains one run-job pair (`run_id=3, run_index=2, job_id=4`), and also another run-job pair (`run_id=1100, run_index=3, job_id=1200, job_index=4`), then `/actions/runs/3/jobs/4` is ambiguous. In that case, the resolver treats it as the ID-based URL by default and shows the page for `run_id=3, job_id=4`. Users can still explicitly force the legacy index-based interpretation with `?by_index=1`, which would resolve the same URL to `/actions/runs/1100/jobs/1200`. - For run summary pages (`/actions/runs/{run}`), it uses a best-effort strategy: by default it first treats `{run}` as a run ID, and if no such run exists in the repository, it falls back to treating `{run}` as a legacy run index and redirects to the ID-based URL. Users can also explicitly force the legacy interpretation with `?by_index=1`. - This summary-page compatibility is best-effort, not a strict ambiguity check. For example, if a repository contains two runs: runA (`id=7, index=3`) and runB (`id=99, index=7`), then `/actions/runs/7` will resolve to runA by default, even though the old index-based URL originally referred to runB. The table below shows how valid legacy index-based target URLs are handled before and after migration `326`. Lower-range legacy URLs are rewritten to ID-based URLs, while higher-range legacy URLs remain unchanged in the database but are still handled correctly by `resolveCurrentRunForView` at runtime. | run_id | run_index | job_id | job_index | old target URL | updated by migration 326 | current target URL | can be resolved correctly | |---|---|---|---|---|---|---|---| | 3 | 2 | 4 | 1 | `/user2/repo2/actions/runs/2/jobs/1` | true | `/user2/repo2/actions/runs/3/jobs/4` | true | | 4 | 3 | 8 | 4 | `/user2/repo2/actions/runs/3/jobs/4` | true | `/user2/repo2/actions/runs/4/jobs/8` | true (without migration 326, this URL will resolve to run(`id=3`)) | | 80 | 20 | 170 | 0 | `/user2/repo2/actions/runs/20/jobs/0` | true | `/user2/repo2/actions/runs/80/jobs/170` | true | | 1500 | 900 | 1600 | 0 | `/user2/repo2/actions/runs/900/jobs/0` | false | `/user2/repo2/actions/runs/900/jobs/0` | true | | 2400 | 1500 | 2600 | 0 | `/user2/repo2/actions/runs/1500/jobs/0` | false | `/user2/repo2/actions/runs/1500/jobs/0` | true | | 2400 | 1500 | 2601 | 1 | `/user2/repo2/actions/runs/1500/jobs/1` | false | `/user2/repo2/actions/runs/1500/jobs/1` | true | For users who already ran the old migration `326`, this change has no functional impact. Their historical URLs are already stored in the ID-based form, and ID-based URLs continue to resolve correctly. For users who have not run the old migration `326`, only a subset of legacy target URLs will now be rewritten during upgrade. This avoids the extreme runtime cost of the previous full migration, while all remaining legacy target URLs continue to work through the web-layer compatibility logic. Many thanks to @wxiaoguang for the suggestions.
Integration tests
Integration tests can be run with make commands for the appropriate backends, namely:
make test-sqlite
make test-pgsql
make test-mysql
make test-mssql
Make sure to perform a clean build before running tests:
make clean build
Run tests via local act_runner
Run all jobs
act_runner exec -W ./.github/workflows/pull-db-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest
Warning: This file defines many jobs, so it will be resource-intensive and therefor not recommended.
Run single job
act_runner exec -W ./.github/workflows/pull-db-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest -j <job_name>
You can list all job names via:
act_runner exec -W ./.github/workflows/pull-db-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest -l
Run sqlite integration tests
Start tests
make test-sqlite
Run MySQL integration tests
Setup a MySQL database inside docker
docker run -e "MYSQL_DATABASE=test" -e "MYSQL_ALLOW_EMPTY_PASSWORD=yes" -p 3306:3306 --rm --name mysql mysql:latest #(just ctrl-c to stop db and clean the container)
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --rm --name elasticsearch elasticsearch:7.6.0 #(in a second terminal, just ctrl-c to stop db and clean the container)
Start tests based on the database container
TEST_MYSQL_HOST=localhost:3306 TEST_MYSQL_DBNAME=test TEST_MYSQL_USERNAME=root TEST_MYSQL_PASSWORD='' make test-mysql
Run pgsql integration tests
Setup a pgsql database inside docker
docker run -e "POSTGRES_DB=test" -e "POSTGRES_USER=postgres" -e "POSTGRES_PASSWORD=postgres" -p 5432:5432 --rm --name pgsql postgres:latest #(just ctrl-c to stop db and clean the container)
Setup minio inside docker
docker run --rm -p 9000:9000 -e MINIO_ROOT_USER=123456 -e MINIO_ROOT_PASSWORD=12345678 --name minio bitnamilegacy/minio:2023.8.31
Start tests based on the database container
TEST_MINIO_ENDPOINT=localhost:9000 TEST_PGSQL_HOST=localhost:5432 TEST_PGSQL_DBNAME=postgres TEST_PGSQL_USERNAME=postgres TEST_PGSQL_PASSWORD=postgres make test-pgsql
Run mssql integration tests
Setup a mssql database inside docker
docker run -e "ACCEPT_EULA=Y" -e "MSSQL_PID=Standard" -e "SA_PASSWORD=MwantsaSecurePassword1" -p 1433:1433 --rm --name mssql microsoft/mssql-server-linux:latest #(just ctrl-c to stop db and clean the container)
Start tests based on the database container
TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=gitea_test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql
Running individual tests
Example command to run GPG test:
For SQLite:
make test-sqlite#GPG
For other databases(replace mssql to mysql, or pgsql):
TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql#GPG
Setting timeouts for declaring long-tests and long-flushes
We appreciate that some testing machines may not be very powerful and the default timeouts for declaring a slow test or a slow clean-up flush may not be appropriate.
You can set the following environment variables:
GITEA_TEST_SLOW_RUN="10s" GITEA_TEST_SLOW_FLUSH="1s" make test-sqlite