0
0
mirror of https://github.com/go-gitea/gitea.git synced 2026-06-02 21:08:57 +02:00
Zettat123 0359746abe
feat(actions)!: improve support for reusable workflows (#37478)
## Summary

This PR improves reusable workflow support for Gitea Actions. The
parsing of the called workflow now happens on Gitea side, not on the
runner. When the caller becomes ready, Gitea fetches the called workflow
source, parses it, and inserts each child job into the database as a
`ActionRunJob` linked to the caller via `ParentCallJobID`. As a result,
every callee job is dispatched as its own task and its logs surface as
an independent job entry in the UI, rather than being inlined into the
caller's "Set up job" step.

This PR supports two kinds of `uses` : 
- same-repo call: `uses: ./.gitea/workflows/foo.yaml`
- cross-repo call: `uses: OWNER/REPO/.gitea/workflows/foo.yaml@REF`

## **⚠️ BREAKING ⚠️**
External reusable workflows (`uses:
https://other-gitea-instance/OWNER/REPO/.gitea/workflows/test.yaml@REF`)
are no longer supported. To keep using them, clone the repositories to
the local instance.

## Main changes

### Execution model

- Each caller job carries `IsReusableCaller=true` and won't be fetched
by runners.
- `ParentCallJobID` can link a called job to its caller.
- Caller status is derived from its direct children.


### Workflow syntax

- `jobparser` now supports parsing `on: workflow_call` trigger with
`inputs:`, `outputs:`, and `secrets:` declarations.
- **Max nesting depth**: capped at `MaxReusableCallLevels = 9`, which
means a top-level caller may have at most 9 nested callers below it.
- **Cycle prevention**: at expansion time, `checkCallerChain` walks the
caller's ancestor chain via `ParentCallJobID` and rejects if the same
`uses:` string appears anywhere upstream (`reusable workflow call cycle
detected`). This catches both direct (`A -> A`) and indirect (`A -> B ->
A`) cycles.

### Cross-repo access

- To share reusable workflows from private repos, use `Collaborative
Owners` introduced by #32562

### Rerun semantics

- `expandRerunJobIDs` partitions the latest attempt's jobs into:
- a **rerun set**: jobs being rerun + downstream siblings within the
same scope.
- an **ancestor set**: reusable callers whose only *some* descendants
are being rerun (the caller itself is not).
- Cloning behavior for callers in `execRerunPlan`:
- **Caller is fully rerun** (caller's `AttemptJobID` in `rerunSet`):
none of its descendants are cloned. The caller is cloned with
`IsCallerExpanded=false`, and re-expansion (which reinserts the children
fresh) happens later when the resolver brings the caller to `Waiting`
again.
- **Caller is in ancestor set** (only some descendants rerun): the
caller is pass-through (`Status` will be updated by its fresh children).
Its non-rerun descendants are also pass-through clones (point
`SourceTaskID` at the original task). Their `ParentCallJobID` is
remapped to the new attempt's caller row.

### UI

- Job list in `RepoActionView.vue` is now tree-shaped: callers indent
their children. Callers default to collapsed.
- New caller detail page using `WorkflowGraph` to show direct children
only; the run summary's `WorkflowGraph` shows top-level callers and
their immediate descendants.

### Known trade-offs

- **Caller expansion runs inside the enclosing write transaction.**
`expandReusableWorkflowCaller` performs a git read of the called
workflow while holding the row locks that update the caller and insert
its children. This is intentional: the caller-row update and child-row
inserts must commit atomically. None of the call sites is hot (each
caller is expanded once per attempt), so the trade-off is acceptable.

- **A malformed `if:` expression on a job leaves it `Blocked`
silently.** `evaluateJobIf` now runs server-side as part of resolver
passes; deterministic expression errors (typos, undefined context
fields) are logged but do not surface in the UI. This is the same
behavior the resolver already had for concurrency-expression errors.
Distinguishing transient DB errors from user-authored expression errors
and writing the latter back as `StatusFailure` is a follow-up.


#### Screenshots

<img width="1600" alt="image"
src="https://github.com/user-attachments/assets/bfaa9b7a-07e9-4127-8de9-a81f86e82828"
/>

<img width="1600" alt="image"
src="https://github.com/user-attachments/assets/8af109b3-ef28-4b53-aaad-d4632b923224"
/>


## References

-
https://docs.github.com/en/actions/how-tos/reuse-automations/reuse-workflows
-
https://docs.github.com/en/actions/reference/workflows-and-actions/reusing-workflow-configurations

---

Replace #36388

---------

Signed-off-by: Zettat123 <zettat123@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Claude (Opus 4.7) <noreply@anthropic.com>
2026-05-30 08:31:14 +02:00

256 lines
8.8 KiB
Go

// Copyright 2025 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package actions
import (
"context"
"fmt"
actions_model "gitea.dev/models/actions"
"gitea.dev/models/db"
"gitea.dev/modules/actions/jobparser"
"gitea.dev/modules/log"
"gitea.dev/modules/util"
act_model "gitea.com/gitea/runner/act/model"
"go.yaml.in/yaml/v4"
)
// PrepareRunAndInsert prepares a run and inserts it into the database
// It parses the workflow content, evaluates concurrency if needed, and inserts the run and its jobs into the database.
// The title will be cut off at 255 characters if it's longer than 255 characters.
func PrepareRunAndInsert(ctx context.Context, content []byte, run *actions_model.ActionRun, inputsWithDefaults map[string]any) error {
if err := run.LoadAttributes(ctx); err != nil {
return fmt.Errorf("LoadAttributes: %w", err)
}
vars, err := actions_model.GetVariablesOfRun(ctx, run)
if err != nil {
return fmt.Errorf("GetVariablesOfRun: %w", err)
}
wfRawConcurrency, err := jobparser.ReadWorkflowRawConcurrency(content)
if err != nil {
return fmt.Errorf("ReadWorkflowRawConcurrency: %w", err)
}
if err = InsertRun(ctx, run, content, vars, inputsWithDefaults, wfRawConcurrency); err != nil {
return fmt.Errorf("InsertRun: %w", err)
}
// Load the newly inserted jobs with all fields from database (the job models in InsertRun are partial, so load again)
allJobs, err := db.Find[actions_model.ActionRunJob](ctx, actions_model.FindRunJobOptions{RunID: run.ID})
if err != nil {
return fmt.Errorf("FindRunJob: %w", err)
}
CreateCommitStatusForRunJobs(ctx, run, allJobs...)
NotifyWorkflowJobsAndRunsStatusUpdate(ctx, allJobs)
return nil
}
// InsertRun inserts a run
// The title will be cut off at 255 characters if it's longer than 255 characters.
func InsertRun(ctx context.Context, run *actions_model.ActionRun, content []byte, vars map[string]string, inputs map[string]any, wfRawConcurrency *act_model.RawConcurrency) error {
var cancelledConcurrencyJobs []*actions_model.ActionRunJob
var hasWaitingCallerJobs bool
if err := db.WithTx(ctx, func(ctx context.Context) error {
index, err := db.GetNextResourceIndex(ctx, "action_run_index", run.RepoID)
if err != nil {
return err
}
run.Index = index
run.Title = util.EllipsisDisplayString(run.Title, 255)
run.Status = actions_model.StatusWaiting
if wfRawConcurrency != nil {
rawConcurrency, err := yaml.Marshal(wfRawConcurrency)
if err != nil {
return fmt.Errorf("marshal raw concurrency: %w", err)
}
run.RawConcurrency = string(rawConcurrency)
}
// Insert before parsing jobs or evaluating workflow-level concurrency
// so that run.ID is populated. Expressions referencing github.run_id —
// in run-name, job names, runs-on, or a workflow-level concurrency
// group like `${{ github.head_ref || github.run_id }}` — would otherwise
// interpolate to an empty string.
if err := db.Insert(ctx, run); err != nil {
return err
}
runAttempt := &actions_model.ActionRunAttempt{
RepoID: run.RepoID,
RunID: run.ID,
Attempt: 1,
TriggerUserID: run.TriggerUserID,
Status: actions_model.StatusWaiting,
}
if wfRawConcurrency != nil {
if err := EvaluateRunConcurrencyFillModel(ctx, run, runAttempt, wfRawConcurrency, vars, inputs); err != nil {
return fmt.Errorf("EvaluateRunConcurrencyFillModel: %w", err)
}
// check run (workflow-level) concurrency
var jobsToCancel []*actions_model.ActionRunJob
runAttempt.Status, jobsToCancel, err = PrepareToStartRunWithConcurrency(ctx, runAttempt)
if err != nil {
return err
}
cancelledConcurrencyJobs = append(cancelledConcurrencyJobs, jobsToCancel...)
}
if err := db.Insert(ctx, runAttempt); err != nil {
return err
}
run.LatestAttemptID = runAttempt.ID
giteaCtx := GenerateGiteaContext(ctx, run, runAttempt, nil)
jobs, err := jobparser.Parse(content, jobparser.WithVars(vars), jobparser.WithGitContext(giteaCtx.ToGitHubContext()), jobparser.WithInputs(inputs))
if err != nil {
return fmt.Errorf("parse workflow: %w", err)
}
titleChanged := len(jobs) > 0 && jobs[0].RunName != ""
if titleChanged {
run.Title = util.EllipsisDisplayString(jobs[0].RunName, 255)
}
cols := []string{"latest_attempt_id"}
if titleChanged {
cols = append(cols, "title")
}
if err := actions_model.UpdateRun(ctx, run, cols...); err != nil {
return err
}
runJobs := make([]*actions_model.ActionRunJob, 0, len(jobs))
var hasWaitingJobs bool
for _, v := range jobs {
id, job := v.Job()
needs := job.Needs()
if err := v.SetJob(id, job.EraseNeeds()); err != nil {
return err
}
payload, _ := v.Marshal()
isReusableWorkflowCaller := job.Uses != ""
shouldBlockJob := runAttempt.Status == actions_model.StatusBlocked || len(needs) > 0 || run.NeedApproval
attemptJobID, err := actions_model.GetNextAttemptJobID(ctx, run.ID)
if err != nil {
return fmt.Errorf("alloc attempt_job_id: %w", err)
}
job.Name = util.EllipsisDisplayString(job.Name, 255)
runJob := &actions_model.ActionRunJob{
RunID: run.ID,
RunAttemptID: runAttempt.ID,
RepoID: run.RepoID,
OwnerID: run.OwnerID,
CommitSHA: run.CommitSHA,
IsForkPullRequest: run.IsForkPullRequest,
Name: job.Name,
Attempt: runAttempt.Attempt,
WorkflowPayload: payload,
JobID: id,
AttemptJobID: attemptJobID,
Needs: needs,
RunsOn: job.RunsOn(),
Status: util.Iif(shouldBlockJob, actions_model.StatusBlocked, actions_model.StatusWaiting),
WorkflowSourceRepoID: run.RepoID,
WorkflowSourceCommitSHA: run.CommitSHA,
}
// Parse workflow/job permissions (no clamping here)
if perms := ExtractJobPermissionsFromWorkflow(v, job); perms != nil {
runJob.TokenPermissions = perms
}
if isReusableWorkflowCaller {
runJob.IsReusableCaller = true
runJob.CallUses = job.Uses
}
// check job concurrency
if job.RawConcurrency != nil {
rawConcurrency, err := yaml.Marshal(job.RawConcurrency)
if err != nil {
return fmt.Errorf("marshal raw concurrency: %w", err)
}
runJob.RawConcurrency = string(rawConcurrency)
// do not evaluate job concurrency when it requires `needs`, the jobs with `needs` will be evaluated later by job emitter
if len(needs) == 0 {
err = EvaluateJobConcurrencyFillModel(ctx, run, runAttempt, runJob, vars, inputs)
if err != nil {
return fmt.Errorf("evaluate job concurrency: %w", err)
}
}
// If a job needs other jobs ("needs" is not empty), its status is set to StatusBlocked at the entry of the loop
// No need to check job concurrency for a blocked job (it will be checked by job emitter later)
if runJob.Status == actions_model.StatusWaiting {
var jobsToCancel []*actions_model.ActionRunJob
runJob.Status, jobsToCancel, err = PrepareToStartJobWithConcurrency(ctx, runJob)
if err != nil {
return fmt.Errorf("prepare to start job with concurrency: %w", err)
}
cancelledConcurrencyJobs = append(cancelledConcurrencyJobs, jobsToCancel...)
}
}
// A reusable caller is never dispatched to a runner, so it must not drive the task-version bump.
hasWaitingJobs = hasWaitingJobs || (runJob.Status == actions_model.StatusWaiting && !isReusableWorkflowCaller)
if err := db.Insert(ctx, runJob); err != nil {
return err
}
// expand reusable caller
if isReusableWorkflowCaller && runJob.Status == actions_model.StatusWaiting {
if err := expandReusableWorkflowCaller(ctx, run, runAttempt, runJob, vars); err != nil {
return fmt.Errorf("inline trigger caller %d ready: %w", runJob.ID, err)
}
// refresh the caller status
if err := actions_model.RefreshReusableCallerStatus(ctx, runJob); err != nil {
return fmt.Errorf("refresh caller %d status: %w", runJob.ID, err)
}
hasWaitingCallerJobs = true
}
runJobs = append(runJobs, runJob)
}
runAttempt.Status = actions_model.AggregateJobStatus(runJobs)
if err := actions_model.UpdateRunAttempt(ctx, runAttempt, "status"); err != nil {
return err
}
// if there is a job in the waiting status, increase tasks version.
if hasWaitingJobs {
if err := actions_model.IncreaseTaskVersion(ctx, run.OwnerID, run.RepoID); err != nil {
return err
}
}
return nil
}); err != nil {
return err
}
NotifyWorkflowJobsAndRunsStatusUpdate(ctx, cancelledConcurrencyJobs)
EmitJobsIfReadyByJobs(cancelledConcurrencyJobs)
// Post-commit kick for expanded callers: let job_emitter resolve its child jobs
if hasWaitingCallerJobs {
if err := EmitJobsIfReadyByRun(run.ID); err != nil {
log.Error("emit run %d after InsertRun: %v", run.ID, err)
}
}
return nil
}