Add Claude PR review workflow #41
Loading…
Reference in a new issue
No description provided.
Delete branch "ci/claude-pr-review"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Adds a
claude-reviewworkflow that reviews every same-repo pull request with Claude Code and posts one comment, updated in place on later pushes.What it does
renovate/…ordependabot/…): a table of every item updated, a changelog per dependency looked up from upstream, concerns checked against how Balam uses the package, and a merge recommendation.Setup required
Add one Actions secret:
ANTHROPIC_API_KEYorCLAUDE_CODE_OAUTH_TOKEN.Optional repository variables:
CLAUDE_REVIEW_MODEL(defaultclaude-opus-5-5)CLAUDE_REVIEW_WAIT_SECONDS(default600; set0if the runner only runs one job at a time, otherwise the wait blocks the jobs it is waiting for)CLAUDE_REVIEW_WEB_DOMAINS(hosts Claude may fetch changelogs from on dependency PRs; defaults to GitHub, GitLab, Go, npm and Docker Hub hosts)Security
Read,GrepandGlob, confined to the checkout and the gathered context; dependency PRs add web search and fetches to the allowed hosts. No shell, no writes, no MCP servers.Testing
scripts/test_claude_pr_review.shruns the script end to end against stubbedcurlandclaude, and runs intest / go.🔎 ojo scan results
Details (5)
Codecov Results 📊
✅ Patch coverage is 100.00%. Project has 4967 uncovered lines.
Generated by Codecov Action
09e2ec84cc7a5307d124Summary
This PR adds a
claude-reviewForgejo workflow. On every pull request opened from a branch of this repository,scripts/claude-pr-review.shcollects the PR metadata, diff, commit statuses and comments, runs Claude Code headless with read-only tools, and posts one comment that is edited on later pushes. Renovate and Dependabot branches get a dependency-update prompt (bot.md); all other PRs get a code-review prompt (human.md). Both sharecommon.md. The PR also adds a stubbed end-to-end test that runs intest / go, a README section, and a Renovate regex manager that keeps the pinned Claude Code version up to date. The diff matches the title and description. Every security measure the description lists is present in the code.This review was produced by the script and prompts in this PR.
maindoes not have.forgejo/claude-review/yet, soprompt_filefalls back to the PR's own copy (scripts/claude-pr-review.sh:187). As the README says, a human should readscripts/claude-pr-review.shand the workflow directly rather than rely on this comment.Findings
renovate/…gets the dependency review instead of a code review. Seescripts/claude-pr-review.sh:30and:95.renovate/xwith arbitrary code changes.humanmode whenever the diff touches files other thango.mod,go.sum,web/package*.json, Dockerfiles or.forgejo/workflows/*.scripts/claude-pr-review.sh:136and:261.forgejo-actions. Matching onuser.login == $bot and contains($marker)means any comment from that account that contains the marker anywhere counts as the previous review..body | startswith($marker).startswith($self)(scripts/claude-pr-review.sh:106) also drops any other workflow whose name begins withclaude-review. Usestartswith($self + " /").scripts/claude-pr-review.sh:118.statusesruns on the left of||, errexit is off inside it. A failedcurlinapi_listtherefore leaveschunkempty, which equals the emptypreviousat:76, so the loop breaks and outputs[].echofallback is never reached. A persistently failing API is logged as "(no checks reported yet)", and the pending note says the same.curl's exit status insideapi_list, or drop the fallback and its comment.CLAUDE_REVIEW_COMMENT_AUTHOR,CLAUDE_REVIEW_BOT_BRANCH_REandCLAUDE_REVIEW_BOT_AUTHOR_RE(scripts/claude-pr-review.sh:30-38) are read by the script, but the workflow does not pass them and the README does not mention them.forgejo-actions, every run creates a new comment, and earlier reviews are fed back to Claude as reviewer comments.COMMENT_AUTHOR(and finding 1's author) throughvars.*and add them to the README table.common.md:15tells Claude to readAGENTS.mdfrom the PR checkout. Prompts are taken from the base branch, but a PR can still weaken the conventions by editingAGENTS.md. Consider giving Claude the base-branchAGENTS.mdas well, or telling it to flag changes toAGENTS.md.fileMatchmay be deprecated. Atrenovate.json:8, recent Renovate releases renamefileMatchtomanagerFilePatternsand migrate the old name automatically. I could not check which Renovate version the shared workflow (colibrisec/forgejo-workflows) uses, so treat this as something to confirm.No tenant-isolation,
internal/crypto, RBAC or serialization concerns apply: the PR changes no Go or TypeScript code.Completeness
scripts/test_claude_pr_review.shcovers prompt selection from the base branch, the tool lists, token stripping, pagination and repeat detection, the planted marker, self-exclusion, in-place update, refusing a review that contains a credential, and a push during the review. Untested, as the description says: the wait loop, the "PR head moved" exit at diff time, and truncation.scripts/test_claude_pr_review.sh:145) only checks exit 0 and that nothing was written. It does not check stdout for "not posting", so an unrelated earlyexit 0would also pass.test.yml:23relies onjqalready being on the runner image.security-scan.ymlandclaude-review.ymlinstall it explicitly. It works today, sincetest / gopassed.STATUS.mddoes not track CI, so it needs no change. Migrations,AllModels(), theapi/spec and frontend changes do not apply.CI and comments
test / web: failed. The PR changes nothing underweb/, and failing after 7s points to a setup ornpm cistep, but I could not see the job log, so I can't confirm it is unrelated. Check whethermainfails the same way.web/package-lock.jsonis not changed here.js-unreachable-codehit atweb/src/pages/Languages.tsx:39is aneslint-disablecomment after areturn, which looks like a false positive. The file is not changed here.golang.org/x/crypto v0.56.0(go.mod:12) being flagged foropenpgp. No Go file in the repo importsopenpgp, andgo.modis not changed here.comments.jsonby design, so I cannot check whether their findings were addressed beyond what the commit messages claim.Recommendations
Before merging:
test / webfails, or confirm it fails the same way onmain.Can wait:
AGENTS.mdfrom the base branch (finding 6).fileMatchvsmanagerFilePatterns(finding 7).minimumReleaseAgefor@anthropic-ai/claude-code. A Renovate PR runs the workflow from its own branch, so a new release would run withANTHROPIC_API_KEYandpull-requests: writeas soon as the PR opens.Verdict
Merge after fixes: any branch named
renovate/…can currently swap the code review for a dependency-only review, andtest / webis failing.Claude code review of
dbe96dbe5f· claude-opus-5-5View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.