Data flow

This page explains where the data of a ReviewFixLoop run goes. The handling of personal data is described in the Privacy Policy.

Terms used on this page

• Harness: a coding agent program, such as a vendor's command-line tool (a "vendor CLI").
• Reviewer: a combination of a harness, a model and an effort setting that reviews a pull request.
• Resolver: the coding agent that conducts a review of a pull request: it decides how to treat each finding and changes the code.
• Run: one execution of the review-and-fix loop on one pull request.
• Finding: an issue that a reviewer reports in a run.
• Judgment: the resolver's decision on a finding.
• Judge: the automated classifier with which the service assigns a tag, a severity and duplicate relations to each finding.
• Workspace: the unit of the service to which runs belong. Members of a workspace can see its runs.

The three layers

The CLI divides the data of a run into three layers.

• L0 is needed to recompute the leaderboard.
• L1 is needed to judge findings and their duplicates. All free text is treated as L1.
• L2 stays in the run's directory on your machine unless your workspace keeps session logs or you upload it with a bug report. Keeping session logs is off by default; only the workspace's owners can turn it on, from the workspace's page or with review-fix-loop workspace session-logs. While it is on, the CLI uploads each run's L2 when the run ends, and every member of the workspace can read it on the run's page.

Where the data goes

GitHub

• Sign-in to the web site uses GitHub OAuth for identity only. It requests no scopes and obtains no access to your repositories. The service stores your GitHub numeric user id, your login, your GitHub account's creation time, the country code of the sign-up request, and the version of the notice you agreed to at sign-up with the time of that consent.
• ReviewFixLoop does not ask you to install a GitHub App on your repositories. Reads from GitHub happen on your machine with your own gh.
• When the repository's .review-fix-loop.jsonc on the base branch sets "prSummary": true, the CLI writes each ended run's summary as a comment on the pull request with your own gh. The summary holds the titles and locations of the run's findings, their judgments and the reasons for them, and the resolver's decisions. Whoever can read the pull request can read it, anyone on a public repository. The service records whether a run had this setting on and receives nothing of the comment.
• The service accepts the repository and pull request information that the CLI reports, and does not check it against GitHub.
• The service itself reads only public data from GitHub: the public profile of a login a workspace owner invites, and a manifest file being registered. It authenticates these reads as its sign-in OAuth App, which cannot read anything private.

The judge

• The service sends the file, line, title, body and evidence of each finding, and of its duplicate candidates, to external AI models that judge its tag, severity and duplicate relations.
• L0 records which judge judged each finding.

Re-judging to measure validity

To measure the validity of resolver judgments, the service samples findings judged in the last seven days. It sends the finding's file, line, title, body, evidence and the resolver's stated reason to an external AI model of a different family from the resolver's. Findings and reasons in which a secret-looking string is detected are left out. The agreement rate is published on the leaderboard. Re-judging does not change the original judgment, reward or rating.

Runs in the personal workspace of a deleted account are left out of new re-judging. Data that was already sent cannot be recalled.

Escaped defects

The public leaderboard shows, for each reviewer, how many defects the reviewers of its runs missed and that were fixed after the pull request was merged.

• Once a day per repository, the CLI looks in your checkout, without fetching or changing anything, for commits on the default branch that change lines a pull request added. It looks only at pull requests merged within the last 30 days whose run is of a workspace that contributes to the public leaderboard and reviewed the head that was merged.
• For each such commit, the CLI sends the commit's id, its message and the parts of its change (hunks, with three lines around each) that change lines the pull request added. A commit in which a secret-looking string is detected is not sent.
• The service sends the message and the hunks, with the file, line, title and body of the run's findings in those files, to the judge, which answers whether the commit fixes a defect and whether a finding described it. The service stores only the commit's id and the judge's probabilities, not the message or the hunks.
• The leaderboard shows the counts as information only. They do not change any rating.

Sending to external AI models

• The judge and re-judging send data with the service's own credentials. This is separate from the vendor CLIs that run on your machine with your credentials.
• What is sent includes findings from private repositories.
• The service requests a route on which the data it sends is not used to train models. It does not require that the providers retain no data at all.
• The providers are companies in the United States of America. You consent to this provision when you sign up; the Privacy Policy states what you are told then.
• The providers the service uses can change. Their names are available on request to the contact in the Privacy Policy.

Vendor credentials

• The official manifests only launch the unmodified vendor CLI that you installed and configured, on your machine.
• ReviewFixLoop does not read, store or relay your vendor credentials, and does not record which authentication method a vendor CLI uses.
• The agent embedded in the ReviewFixLoop CLI is ReviewFixLoop's own. It reads, on your machine, the model provider's API key you configured for it (in your environment, your user configuration, or a command your repository's configuration names), and sends the key only to the endpoint configured with it. The key never reaches the service, and the service never supplies an endpoint or a command for the key.

See Policy toward each vendor's terms.

What a reviewer can do on your machine

A reviewer runs with the same authority as a coding agent you launch yourself: your environment variables, your home directory, your credentials, the configuration of your vendor CLI, and your network. A reviewer that could do less than your own agent would review worse. The official manifests therefore start each vendor CLI without approval prompts and without the CLI's own sandbox.

If you would rather keep the agent away from the rest of your machine, turn on the sandbox ("sandbox": true in your user configuration or in the repository's .review-fix-loop.jsonc on the base branch). Every reviewer of the run then runs inside your operating system's sandbox (Seatbelt on macOS, bubblewrap on Linux) and reviews with less than your own agent has:

• It reads its copy of the pull request, the objects of your checkout that the copy borrows instead of duplicating them (your repository's git history, including commits you have not pushed), the tools on your PATH, the toolchains it needs (mise, rustup, Playwright's browsers, also where they sit in your home directory), and the operating system's own files. Nothing else on your machine is readable: not the rest of your home directory, your other repositories, other volumes or temporary directories, wherever they are. It writes only its copy and its own scratch space.
• Its home directory is an empty one of its own, and it receives only harmless variables of your environment (such as PATH and the locale) and the variables the harness signs in with. A vendor CLI therefore signs in with a key or token in your environment, such as CLAUDE_CODE_OAUTH_TOKEN for Claude Code or CODEX_API_KEY for Codex, not with its saved login.
• It connects only to its model's API and to the public package registries and GitHub, without your credentials, and not to the other programs on your machine (their Unix sockets and local ports, such as a database or a container engine). On macOS it cannot start a server unless your user configuration allows it to listen on the run's ports ("allowLocalBinding": true), where other hosts of your network could then reach that server. You can allow more hosts, paths or variables in your user configuration, and a repository can allow more hosts in its configuration on the base branch.
• Its copy receives none of the files git ignores in your checkout, and no dependency is installed in it beforehand: an install runs the repository's own scripts, which must not run outside the sandbox. The reviewer installs what it needs inside the sandbox, from the public registries.
• A resolver commits and pushes with your credentials, so it never runs in the sandbox.
• On Linux the sandbox needs bubblewrap and socat installed and, on Ubuntu 24.04 or later, user namespaces allowed for bubblewrap (an AppArmor profile, or kernel.apparmor_restrict_unprivileged_userns=0). When something is missing, the review fails instead of running with less protection.

What a reviewer does not share with you is its place of work:

• Each reviewer works in its own copy of the pull request, in a repository of its own under ~/.cache/review-fix-loop. Outside the sandbox, the copy holds the reviewed commit and copies of the files git ignores in your checkout (installed dependencies, local configuration such as .env files, generated files), so a reviewer can run the project as your own agent does. A few ignored files are left out because a copy of them would not work or would point back at your checkout: Python virtual environments, directories that are repositories or worktrees of their own, a .tmp directory at the root, ignored lockfiles, and files on another file system mounted inside your checkout. What a reviewer writes there does not reach your checkout, with two exceptions. A dependency file that your package manager hard-links from its cache when it installs (bun and pnpm on Linux) is the same file in every checkout installed from that cache, as when you install two checkouts yourself. A symbolic link that the repository itself commits with an absolute path into your checkout also still leads there.
• Those files are copied only when your checkout shares history with the reviewed commit, its dependency lockfiles equal the reviewed commit's, and the file system can copy without duplicating the data. Otherwise the CLI installs the dependencies in the copy with your own package manager (bun, pnpm, npm or uv) and your own environment, so a private registry authenticates as it does when you install yourself. When the install fails, or that package manager is not installed on your machine, the review still runs, without the dependencies installed.
• ReviewFixLoop's own token (REVIEW_FIX_LOOP_API_TOKEN) is removed from a reviewer's environment.

Because a reviewer has your authority, the code it runs must come from an author you trust. A pull request by an author outside the trusted set is not run unless you say so:

• The default is everyone for a private or internal repository and collaborators for a public one. A repository whose visibility cannot be read counts as public.
• "trust" in .review-fix-loop.jsonc sets it for a repository. It is read from the pull request's base branch, never from the pull request itself, so a pull request cannot trust itself.
• review --trust <value> sets it for one run.
• A refused pull request starts no reviewer and installs nothing. The CLI says why and how to review it anyway.
• The CLI judges a pull request by its author and by the repository its branch is in, as GitHub reports them. It does not examine who pushed each commit.

Secrets

• Before sending, the CLI masks strings in evidence that look like secrets.
• Before uploading L2, as session logs or with a bug report, the CLI masks strings in common credential formats in it. Masking does not find every secret: a secret in another format is uploaded as it is.

What is public

• Everything else belongs to a workspace and is visible only to that workspace's members, except a run summary the CLI posts to a pull request (prSummary, see "GitHub" above), which is visible to whoever can read the pull request.
• The aggregates rest on AI judgments, and the leaderboard says so.
• Free use requires contributing to the public leaderboard.

Retention

See "Retention" in the Privacy Policy for the periods for L0, L1 and L2. Its "Account deletion" section describes which account data is removed, the temporary digest used to prevent resetting free limits, and what run data remains.