mirror of
https://github.com/go-gitea/gitea.git
synced 2026-06-03 12:17:10 +02:00
## Summary This PR improves reusable workflow support for Gitea Actions. The parsing of the called workflow now happens on Gitea side, not on the runner. When the caller becomes ready, Gitea fetches the called workflow source, parses it, and inserts each child job into the database as a `ActionRunJob` linked to the caller via `ParentCallJobID`. As a result, every callee job is dispatched as its own task and its logs surface as an independent job entry in the UI, rather than being inlined into the caller's "Set up job" step. This PR supports two kinds of `uses` : - same-repo call: `uses: ./.gitea/workflows/foo.yaml` - cross-repo call: `uses: OWNER/REPO/.gitea/workflows/foo.yaml@REF` ## **⚠️ BREAKING ⚠️** External reusable workflows (`uses: https://other-gitea-instance/OWNER/REPO/.gitea/workflows/test.yaml@REF`) are no longer supported. To keep using them, clone the repositories to the local instance. ## Main changes ### Execution model - Each caller job carries `IsReusableCaller=true` and won't be fetched by runners. - `ParentCallJobID` can link a called job to its caller. - Caller status is derived from its direct children. ### Workflow syntax - `jobparser` now supports parsing `on: workflow_call` trigger with `inputs:`, `outputs:`, and `secrets:` declarations. - **Max nesting depth**: capped at `MaxReusableCallLevels = 9`, which means a top-level caller may have at most 9 nested callers below it. - **Cycle prevention**: at expansion time, `checkCallerChain` walks the caller's ancestor chain via `ParentCallJobID` and rejects if the same `uses:` string appears anywhere upstream (`reusable workflow call cycle detected`). This catches both direct (`A -> A`) and indirect (`A -> B -> A`) cycles. ### Cross-repo access - To share reusable workflows from private repos, use `Collaborative Owners` introduced by #32562 ### Rerun semantics - `expandRerunJobIDs` partitions the latest attempt's jobs into: - a **rerun set**: jobs being rerun + downstream siblings within the same scope. - an **ancestor set**: reusable callers whose only *some* descendants are being rerun (the caller itself is not). - Cloning behavior for callers in `execRerunPlan`: - **Caller is fully rerun** (caller's `AttemptJobID` in `rerunSet`): none of its descendants are cloned. The caller is cloned with `IsCallerExpanded=false`, and re-expansion (which reinserts the children fresh) happens later when the resolver brings the caller to `Waiting` again. - **Caller is in ancestor set** (only some descendants rerun): the caller is pass-through (`Status` will be updated by its fresh children). Its non-rerun descendants are also pass-through clones (point `SourceTaskID` at the original task). Their `ParentCallJobID` is remapped to the new attempt's caller row. ### UI - Job list in `RepoActionView.vue` is now tree-shaped: callers indent their children. Callers default to collapsed. - New caller detail page using `WorkflowGraph` to show direct children only; the run summary's `WorkflowGraph` shows top-level callers and their immediate descendants. ### Known trade-offs - **Caller expansion runs inside the enclosing write transaction.** `expandReusableWorkflowCaller` performs a git read of the called workflow while holding the row locks that update the caller and insert its children. This is intentional: the caller-row update and child-row inserts must commit atomically. None of the call sites is hot (each caller is expanded once per attempt), so the trade-off is acceptable. - **A malformed `if:` expression on a job leaves it `Blocked` silently.** `evaluateJobIf` now runs server-side as part of resolver passes; deterministic expression errors (typos, undefined context fields) are logged but do not surface in the UI. This is the same behavior the resolver already had for concurrency-expression errors. Distinguishing transient DB errors from user-authored expression errors and writing the latter back as `StatusFailure` is a follow-up. #### Screenshots <img width="1600" alt="image" src="https://github.com/user-attachments/assets/bfaa9b7a-07e9-4127-8de9-a81f86e82828" /> <img width="1600" alt="image" src="https://github.com/user-attachments/assets/8af109b3-ef28-4b53-aaad-d4632b923224" /> ## References - https://docs.github.com/en/actions/how-tos/reuse-automations/reuse-workflows - https://docs.github.com/en/actions/reference/workflows-and-actions/reusing-workflow-configurations --- Replace #36388 --------- Signed-off-by: Zettat123 <zettat123@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: Claude (Opus 4.7) <noreply@anthropic.com>
343 lines
13 KiB
Go
343 lines
13 KiB
Go
// Copyright 2026 The Gitea Authors. All rights reserved.
|
|
// SPDX-License-Identifier: MIT
|
|
|
|
package actions
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
|
|
actions_model "gitea.dev/models/actions"
|
|
"gitea.dev/models/db"
|
|
perm_model "gitea.dev/models/perm"
|
|
access_model "gitea.dev/models/perm/access"
|
|
repo_model "gitea.dev/models/repo"
|
|
"gitea.dev/modules/actions/jobparser"
|
|
"gitea.dev/modules/container"
|
|
"gitea.dev/modules/gitrepo"
|
|
"gitea.dev/modules/json"
|
|
api "gitea.dev/modules/structs"
|
|
"gitea.dev/modules/util"
|
|
"gitea.dev/services/convert"
|
|
|
|
"xorm.io/builder"
|
|
)
|
|
|
|
// MaxReusableCallLevels caps how deep a reusable workflow can nest:
|
|
// a top-level caller may have at most MaxReusableCallLevels nested callers below it.
|
|
const MaxReusableCallLevels = 9
|
|
|
|
// loadReusableWorkflowSource resolves the workflow file referenced by a caller's `uses:` and returns its raw bytes,
|
|
// along with the (repo_id, commit_sha) the file was loaded from.
|
|
func loadReusableWorkflowSource(ctx context.Context, run *actions_model.ActionRun, caller *actions_model.ActionRunJob, ref *jobparser.UsesRef) (content []byte, sourceRepoID int64, sourceCommitSHA string, err error) {
|
|
if err := run.LoadAttributes(ctx); err != nil {
|
|
return nil, 0, "", err
|
|
}
|
|
|
|
switch ref.Kind {
|
|
case jobparser.UsesKindLocalSameRepo:
|
|
// `./` is resolved against the workflow file containing the `uses:` - i.e. the caller's own source repo + commit.
|
|
callerRepo, err := repo_model.GetRepositoryByID(ctx, caller.WorkflowSourceRepoID)
|
|
if err != nil {
|
|
return nil, 0, "", fmt.Errorf("look up caller source repo %d: %w", caller.WorkflowSourceRepoID, err)
|
|
}
|
|
bytes, resolvedSHA, err := readWorkflowFromRepo(ctx, callerRepo, caller.WorkflowSourceCommitSHA, ref.Path)
|
|
if err != nil {
|
|
return nil, 0, "", err
|
|
}
|
|
return bytes, callerRepo.ID, resolvedSHA, nil
|
|
|
|
case jobparser.UsesKindLocalCrossRepo:
|
|
repo, err := repo_model.GetRepositoryByOwnerAndName(ctx, ref.Owner, ref.Repo)
|
|
if err != nil {
|
|
return nil, 0, "", fmt.Errorf("look up cross-repo workflow source %q: %w", ref.Owner+"/"+ref.Repo, err)
|
|
}
|
|
ok, err := access_model.CanReadWorkflowCrossRepo(ctx, repo, run)
|
|
if err != nil {
|
|
return nil, 0, "", err
|
|
}
|
|
if !ok {
|
|
return nil, 0, "", fmt.Errorf("no permission to read reusable workflow from %s/%s", ref.Owner, ref.Repo)
|
|
}
|
|
bytes, resolvedSHA, err := readWorkflowFromRepo(ctx, repo, ref.Ref, ref.Path)
|
|
if err != nil {
|
|
return nil, 0, "", err
|
|
}
|
|
return bytes, repo.ID, resolvedSHA, nil
|
|
}
|
|
return nil, 0, "", fmt.Errorf("unsupported uses kind %d", ref.Kind)
|
|
}
|
|
|
|
// readWorkflowFromRepo loads a workflow file from `repo` at `refOrSHA` and returns its content plus the resolved commit SHA.
|
|
func readWorkflowFromRepo(ctx context.Context, repo *repo_model.Repository, refOrSHA, path string) ([]byte, string, error) {
|
|
gitRepo, err := gitrepo.OpenRepository(ctx, repo)
|
|
if err != nil {
|
|
return nil, "", fmt.Errorf("open repo %s: %w", repo.FullName(), err)
|
|
}
|
|
defer gitRepo.Close()
|
|
|
|
commit, err := gitRepo.GetCommit(refOrSHA)
|
|
if err != nil {
|
|
return nil, "", fmt.Errorf("get commit %q in %s: %w", refOrSHA, repo.FullName(), err)
|
|
}
|
|
str, err := commit.GetFileContent(path, 1024*1024)
|
|
if err != nil {
|
|
return nil, "", fmt.Errorf("read %s@%s:%s: %w", repo.FullName(), refOrSHA, path, err)
|
|
}
|
|
return []byte(str), commit.ID.String(), nil
|
|
}
|
|
|
|
// checkCallerChain walks `caller`'s ancestor chain (via ParentJobID) and:
|
|
// - rejects cycles (caller.CallUses appearing in any ancestor's CallUses)
|
|
// - enforces MaxReusableCallLevels on the number of ancestors above `caller`
|
|
//
|
|
// Cycle detection is intentionally *syntactic* (string equality on CallUses), not semantic.
|
|
// So `owner/repo/lib.yml@v1` and `owner/repo/lib.yml@refs/heads/v1` resolving to the same commit are NOT treated as the same node.
|
|
// Going semantic (Owner, Repo, Path, ResolvedSHA tuples) would require extra git reads.
|
|
func checkCallerChain(ctx context.Context, caller *actions_model.ActionRunJob) error {
|
|
if caller.ParentJobID == 0 {
|
|
return nil // top-level caller: depth 0, no ancestors to walk
|
|
}
|
|
|
|
visited := make(container.Set[string])
|
|
visited.Add(caller.CallUses)
|
|
|
|
depth := 0
|
|
current := caller
|
|
for current.ParentJobID != 0 {
|
|
next, err := actions_model.GetRunJobByRunAndID(ctx, current.RunID, current.ParentJobID)
|
|
if err != nil {
|
|
return fmt.Errorf("walk caller chain: %w", err)
|
|
}
|
|
current = next
|
|
depth++
|
|
if depth > MaxReusableCallLevels {
|
|
return fmt.Errorf("reusable workflow call exceeds the maximum nesting level of %d at %q", MaxReusableCallLevels, caller.CallUses)
|
|
}
|
|
if current.IsReusableCaller && current.CallUses != "" {
|
|
if visited.Contains(current.CallUses) {
|
|
return fmt.Errorf("reusable workflow call cycle detected: %q", current.CallUses)
|
|
}
|
|
visited.Add(current.CallUses)
|
|
}
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// expandReusableWorkflowCaller loads and parses the target reusable workflow and inserts the caller's direct child jobs.
|
|
// It expands only ONE level: a child that is itself a reusable caller is inserted Blocked and expanded later by a subsequent resolver pass.
|
|
// It does NOT schedule a follow-up resolver pass; the caller of this function is responsible for emitting.
|
|
//
|
|
// All call sites (PrepareRunAndInsert, execRerunPlan, checkJobsOfCurrentRunAttempt, ApproveRuns) invoke this inside their enclosing write transaction,
|
|
// because the caller row update and the child-row inserts must commit atomically.
|
|
// Be aware this is not cheap inside a tx: it does a git read, YAML parsing, and `${{ }}` expression evaluation.
|
|
// None of the call sites is hot: each caller is expanded once per attempt.
|
|
func expandReusableWorkflowCaller(ctx context.Context, run *actions_model.ActionRun, attempt *actions_model.ActionRunAttempt, caller *actions_model.ActionRunJob, vars map[string]string) error {
|
|
// Already expanded by an earlier call, skip
|
|
if caller.IsExpanded {
|
|
return nil
|
|
}
|
|
|
|
// 1. Cycle + depth check via the ParentJobID chain.
|
|
if err := checkCallerChain(ctx, caller); err != nil {
|
|
return err
|
|
}
|
|
|
|
// 2. Parse the caller's own job (Uses, With, RawSecrets) from its WorkflowPayload.
|
|
parsedJob, err := caller.ParseJob()
|
|
if err != nil {
|
|
return fmt.Errorf("parse caller job %d: %w", caller.ID, err)
|
|
}
|
|
|
|
// 3. Load called-workflow source.
|
|
ref, err := jobparser.ParseUses(parsedJob.Uses)
|
|
if err != nil {
|
|
return fmt.Errorf("parse uses %q: %w", parsedJob.Uses, err)
|
|
}
|
|
content, contentSourceRepoID, contentSourceCommitSHA, err := loadReusableWorkflowSource(ctx, run, caller, ref)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
// 4. Parse the called workflow's spec (used by both secret validation and input evaluation).
|
|
wcSpec, err := jobparser.ParseWorkflowCallSpec(content)
|
|
if err != nil {
|
|
return fmt.Errorf("parse called workflow spec: %w", err)
|
|
}
|
|
|
|
// 5. Resolve caller's `secrets:` and validate it against the callee's schema.
|
|
inherit, secretsMap, err := jobparser.ParseCallerSecrets(parsedJob.RawSecrets)
|
|
if err != nil {
|
|
return fmt.Errorf("caller secrets %q: %w", caller.JobID, err)
|
|
}
|
|
// Under `secrets: inherit` the caller forwards all of its own secrets verbatim and does NOT name them individually,
|
|
// so required-secret presence cannot be verified at expansion time and a missing required secret will surface at job runtime.
|
|
// This matches GitHub Actions' behavior.
|
|
if !inherit {
|
|
if err := jobparser.ValidateCallerSecrets(wcSpec, secretsMap); err != nil {
|
|
return fmt.Errorf("caller %q secrets: %w", caller.JobID, err)
|
|
}
|
|
}
|
|
switch {
|
|
case inherit:
|
|
caller.CallSecrets = jobparser.SecretsInherit
|
|
case len(secretsMap) > 0:
|
|
mapBytes, err := json.Marshal(secretsMap)
|
|
if err != nil {
|
|
return fmt.Errorf("marshal caller secret map: %w", err)
|
|
}
|
|
caller.CallSecrets = string(mapBytes)
|
|
}
|
|
caller.ReusableWorkflowContent = content
|
|
|
|
// 6. Evaluate caller's `with:`, then match against the callee schema.
|
|
workflowCallInputs := map[string]any{}
|
|
if len(wcSpec.Inputs) > 0 {
|
|
jobResults, err := findJobNeedsAndFillJobResults(ctx, caller)
|
|
if err != nil {
|
|
return fmt.Errorf("find caller needs: %w", err)
|
|
}
|
|
parentInputs, err := getInputsForJob(ctx, run, caller)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
callerGitCtx := GenerateGiteaContext(ctx, run, attempt, caller)
|
|
evaluated, err := jobparser.EvaluateCallerWith(
|
|
caller.JobID, parsedJob,
|
|
callerGitCtx, jobResults, vars, parentInputs,
|
|
)
|
|
if err != nil {
|
|
return fmt.Errorf("evaluate caller with: %w", err)
|
|
}
|
|
workflowCallInputs, err = jobparser.MatchCallerInputsAgainstSpec(wcSpec, evaluated)
|
|
if err != nil {
|
|
return fmt.Errorf("caller %q inputs: %w", caller.JobID, err)
|
|
}
|
|
}
|
|
|
|
// 7. Build CallPayload (persisted in step 9).
|
|
callPayload, err := (&api.WorkflowCallPayload{
|
|
Workflow: run.WorkflowID,
|
|
Ref: run.Ref,
|
|
Repository: convert.ToRepo(ctx, run.Repo, access_model.Permission{AccessMode: perm_model.AccessModeNone}),
|
|
Sender: convert.ToUserWithAccessMode(ctx, run.TriggerUser, perm_model.AccessModeNone),
|
|
Inputs: workflowCallInputs,
|
|
}).JSONPayload()
|
|
if err != nil {
|
|
return fmt.Errorf("build call payload: %w", err)
|
|
}
|
|
|
|
// 8. Insert direct children of this caller.
|
|
existingChildren, err := actions_model.GetDirectChildJobsByParent(ctx, caller)
|
|
if err != nil {
|
|
return fmt.Errorf("get existing children of caller %d: %w", caller.ID, err)
|
|
}
|
|
if len(existingChildren) > 0 {
|
|
// Should not happen - child jobs cannot be expanded before the caller gets ready
|
|
return fmt.Errorf("invariant violation: caller %d has %d pre-existing children", caller.ID, len(existingChildren))
|
|
}
|
|
if err := insertCallerChildren(ctx, run, attempt, caller, content, contentSourceRepoID, contentSourceCommitSHA, vars, workflowCallInputs); err != nil {
|
|
return err
|
|
}
|
|
|
|
// 9. Update caller-related cols.
|
|
caller.CallPayload = string(callPayload)
|
|
caller.IsExpanded = true
|
|
n, err := actions_model.UpdateRunJob(ctx, caller,
|
|
builder.Eq{"is_expanded": false},
|
|
"call_secrets", "reusable_workflow_content", "call_payload", "is_expanded")
|
|
if err != nil {
|
|
return fmt.Errorf("commit caller %d expansion: %w", caller.ID, err)
|
|
}
|
|
if n == 0 {
|
|
return fmt.Errorf("caller %d already expanded by another writer", caller.ID)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// insertCallerChildren parses the called workflow with the caller's resolved inputs and inserts each parsed job.
|
|
func insertCallerChildren(ctx context.Context, run *actions_model.ActionRun, attempt *actions_model.ActionRunAttempt, caller *actions_model.ActionRunJob, content []byte, sourceRepoID int64, sourceCommitSHA string, vars map[string]string, inputs map[string]any) error {
|
|
// Parse the called workflow with the caller's `inputs`
|
|
gitCtx := GenerateGiteaContext(ctx, run, attempt, nil)
|
|
if event, ok := gitCtx["event"].(map[string]any); ok {
|
|
event["inputs"] = inputs
|
|
}
|
|
gitCtx["event_name"] = "workflow_call"
|
|
|
|
childWorkflows, err := jobparser.Parse(content,
|
|
jobparser.WithVars(vars),
|
|
jobparser.WithGitContext(gitCtx.ToGitHubContext()),
|
|
jobparser.WithInputs(inputs),
|
|
)
|
|
if err != nil {
|
|
return fmt.Errorf("parse called workflow for caller %d: %w", caller.ID, err)
|
|
}
|
|
if len(childWorkflows) == 0 {
|
|
return fmt.Errorf("called workflow for caller %d (uses %q) has no jobs", caller.ID, caller.CallUses)
|
|
}
|
|
|
|
priorChildren, err := actions_model.GetPriorAttemptChildrenByParent(ctx, run.ID, attempt.ID, caller.AttemptJobID)
|
|
if err != nil {
|
|
return fmt.Errorf("lookup prior-attempt children of caller %d: %w", caller.ID, err)
|
|
}
|
|
|
|
for _, sw := range childWorkflows {
|
|
jobID, parsedChild := sw.Job()
|
|
if parsedChild == nil {
|
|
continue
|
|
}
|
|
needs := parsedChild.Needs()
|
|
if err := sw.SetJob(jobID, parsedChild.EraseNeeds()); err != nil {
|
|
return err
|
|
}
|
|
payload, err := sw.Marshal()
|
|
if err != nil {
|
|
return fmt.Errorf("marshal child %q under caller %d: %w", jobID, caller.ID, err)
|
|
}
|
|
|
|
parsedChild.Name = util.EllipsisDisplayString(parsedChild.Name, 255)
|
|
|
|
// AttemptJobID: prefer a prior-attempt match by (JobID, Name) and fall back to a fresh allocator value for newly-appearing logical jobs.
|
|
// The two-level key disambiguates matrix instances (same JobID, different Names) and distinct jobs that legally share the same Name (different JobIDs).
|
|
var attemptJobID int64
|
|
if priorChild, ok := priorChildren[jobID][parsedChild.Name]; ok {
|
|
attemptJobID = priorChild.AttemptJobID
|
|
} else {
|
|
attemptJobID, err = actions_model.GetNextAttemptJobID(ctx, run.ID)
|
|
if err != nil {
|
|
return fmt.Errorf("alloc attempt_job_id for child %q: %w", jobID, err)
|
|
}
|
|
}
|
|
child := &actions_model.ActionRunJob{
|
|
RunID: run.ID,
|
|
RunAttemptID: attempt.ID,
|
|
RepoID: run.RepoID,
|
|
OwnerID: run.OwnerID,
|
|
CommitSHA: run.CommitSHA,
|
|
IsForkPullRequest: run.IsForkPullRequest,
|
|
Name: parsedChild.Name,
|
|
Attempt: attempt.Attempt,
|
|
WorkflowPayload: payload,
|
|
JobID: jobID,
|
|
AttemptJobID: attemptJobID,
|
|
Needs: needs,
|
|
RunsOn: parsedChild.RunsOn(),
|
|
Status: actions_model.StatusBlocked,
|
|
ParentJobID: caller.ID,
|
|
WorkflowSourceRepoID: sourceRepoID,
|
|
WorkflowSourceCommitSHA: sourceCommitSHA,
|
|
}
|
|
if perms := ExtractJobPermissionsFromWorkflow(sw, parsedChild); perms != nil {
|
|
child.TokenPermissions = perms
|
|
}
|
|
if parsedChild.Uses != "" {
|
|
child.IsReusableCaller = true
|
|
child.CallUses = parsedChild.Uses
|
|
}
|
|
if err := db.Insert(ctx, child); err != nil {
|
|
return fmt.Errorf("insert child %q under caller %d: %w", jobID, caller.ID, err)
|
|
}
|
|
}
|
|
return nil
|
|
}
|