Pipeline Standards¶

Standards for the repository and for the pipeline that carries a change to an org. Each standard applies the Principles, and each names the workflow, ruleset, or setting that enforces it, as the code and metadata standards name their rules. The reasoning is in the Governance sections, from Workflow architecture to Governance at scale, and the workflow files are in Appendix A.

• Version Control: the repository, who can reach it, its branches and what protects them, and how a change enters a protected branch
• Sandboxes: the orgs that the pipeline's stages use, the data they hold, and who has access to each one

The standards on this page are the ones every pull request and every org is held to, whichever branch strategy and release tool a team uses.

Formatting¶

• The repository has a Prettier configuration file, and every source file is formatted to it (Version Control: Prettier has the settings and two sample configuration files). Formatting is then never a matter for review, and a pull request shows only the lines that changed in substance. Enforced by the Prettier Formatting Verification workflow, which runs prettier --check on every pull request and fails when a file is not formatted.

Quality Gates¶

Each standard here is one gate on a pull request.

• No secret is committed. Enforced by the Secret Scanning workflow, which in Appendix A is a placeholder for the scanner the organisation chooses.
• Changed code and metadata pass static analysis. One severity threshold, 2, is set once at the top of each caller workflow and passed to the scan: a violation at severity 1 or 2 blocks the pull request, and one at severity 3, 4, or 5 is reported and does not block (Severity Levels). Enforced by the Static Code Analysis workflow, which takes the threshold from the caller workflows.
• The Apex tests that bear on the change pass: the tests in the pull request, and the tests that exercise the code it changes. Enforced by the Apex Test Selection workflow, which chooses the tests, and the Metadata Validation and Apex Tests workflow, which runs them.
• The results of static code analysis and of the Apex tests are published on the pull request as tables, whichever tool produced them: the blocking violations apart from the advisory ones, the failing tests, and the classes below the coverage threshold. A developer reads them on the pull request and not in a log. Enforced by the Static Code Analysis and Metadata Validation and Apex Tests workflows, which write each table to a comment on the pull request, updated on every push, and to the job summary of the run (Developer Feedback Quality and Shift-Left Visibility).
• The Jest tests of the changed Lightning Web Components pass. Enforced by the LWC Jest Tests workflow.
• The change deploys to the target org. Enforced by the Metadata Validation and Apex Tests workflow, which runs a deployment that changes nothing.
• A change that deletes metadata is approved by a person, even when every automated gate passes. Enforced by the Destructive Change Detection workflow.
• No Tier 0 metadata is committed (Tier 0). Enforced by the Prohibited Metadata Check workflow.
• A pull request is labelled with the highest metadata risk tier among its files, so that the right reviewers see it. Enforced by the Metadata Risk Tier workflow.

Static Analysis¶

• Salesforce Code Analyzer runs on every push, as it does on every pull request, under the same code-analyzer.yml. Enforced by the CI on Push caller, which runs the Static Code Analysis workflow.
• A push is scanned the way a pull request is: only its delta package, built with sfdx-git-delta from the commit before the push to the new commit, so that the delta holds every commit of the push. A push never scans the whole repository: in a large repository that is slow, and the violations of old code would fill the summary of every push. Enforced by the CI on Push caller, which runs the Static Code Analysis workflow in its ci mode.
• A step that needs an org, or that scans or tests the change, runs only when the delta package holds deployable metadata: the workflow first checks the package.xml of the delta package for a <types> element. Enforced by the Generate Delta Package workflow and each gate that reads the delta package.
• The results of a scan are uploaded to GitHub code scanning as SARIF, so that the repository's Security tab keeps a record of every finding. Enforced by the Static Code Analysis workflow.
• A pull request scan stores its complete results, as a CSV file, a JSON file, and a SARIF file, in one workflow artifact that is kept for 7 days. The comment on the pull request is a summary, and the artifact has every violation, also for a repository where GitHub code scanning is not available. Enforced by the Static Code Analysis workflow.
• A scan of the whole repository is started by a person, and no push, pull request, or schedule runs it. It writes every violation to a CSV file, a JSON file, and a SARIF file, in one artifact, and it never fails: it is a report for measuring and cleaning up old code, not a gate. Enforced by the On Demand Full Repository Scan caller.

Apex Tests¶

• On a pull request, the Apex tests run inside the validation of the delta package (sf project deploy start --dry-run), so that the code under test is the pull request's own and not the copy already in the org. Enforced by the Metadata Validation and Apex Tests workflow.
• A pull request runs RunRelevantTests (Beta, API 66.0 and later), or RunSpecifiedTests with a computed list where RunRelevantTests cannot be used. Two GitHub Actions variables choose the test level, one for the whole organisation and one for a repository, and the organisation's variable wins when it is set. Enforced by the Apex Test Selection workflow.
• The RunSpecifiedTests list is computed for each pull request and never kept by hand. It comes from four sources: every class and trigger in the delta package, the classes named for them with the Test or Tests suffix, the tests that the author lists in the pull request description, and a short list of tests that always run. It depends on no earlier test run, so the same pull request always gets the same list. Enforced by the Apex Test Selection workflow.
• When a pull request changes Apex and the list holds no test, no test runs, and the gate fails because the changed class or trigger has no coverage. The gate does not fall back to running every local test. Enforced by the Apex Test Selection and Metadata Validation and Apex Tests workflows.
• Under either test level, the tests that ran cover each class and trigger in the deployment by at least 75%, each one separately and not as an overall percentage. A changed class whose tests were not selected fails the gate, and the fix is a test, or a testFor link to the test it already has (Apex: Test Selection). Enforced by the platform, in the validation that the Metadata Validation and Apex Tests workflow runs.

Org Surveillance¶

Five standards are about the org itself, and the first three are checked on a schedule.

• The org matches the repository. A change made in the org and not in version control is found and reported. Enforced by the drift detection of the Org Scan workflow.
• Every Apex class and trigger in the org compiles. Enforced by the Apex Compilation Check workflow.
• Every local Apex test in the org passes. The run is RunLocalTests, which runs the local tests and the tests of unlocked packages that have no namespace. Enforced by the Apex Tests (Full Org) workflow.
• Every scheduled run reports to the team's channel. The drift check and the full-org test run each post their result on every run, whatever they found, and a check that fails or cannot run posts a failure notice. Silence then means that a run did not happen. Enforced by the two scheduled callers, Scheduled Weekly Project Scan and Scheduled Nightly Test Runner.
• RunAllTestsInOrg is not run: not on a pull request, not on a schedule, and not at a deployment. It adds the tests of managed packages, and a failing managed package test does not affect the org's own code or the change being deployed. Checked in review: none of the workflows in Appendix A names that test level.