diff --git a/docs/ci.md b/docs/ci.md index f00b526..794f13c 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -1 +1,9 @@ # Testing + +The CentOS Project has some resources available for each SIG to run some CI jobs/tests for their projects. +We'll document soon how to get onboarded (on request) on the CentOS CI infra platform. + +We offer the following resources : + + * Openshift hosted jenkins (one per project/SIG), using usual authentication from FAS/ACO + * bare-metal and/or Virtual Machines ephemeral nodes on which you can run some tests (including destructive ones), that will be automatically be reinstalled (for bare-metal) or discarded (for VMs) diff --git a/docs/delivery.md b/docs/delivery.md index 97715e0..12c8c98 100644 --- a/docs/delivery.md +++ b/docs/delivery.md @@ -1 +1,149 @@ # Delivery + +## Promoting to testing + +By default, packages built on cbs are just tagged to `candidate` tag and stay in cbs/koji. +If you want your packages to get pushed to the https://buildlogs.centos.org mirror pool, you can tag packages to `testing` + +!!! warning + Worth knowing that while packages are served over https, and repositories metatdata signed, the packages *aren't* signed with gpg key at this time ! Also good to know that only classical pkgs are pushed out, so no src.rpm nor debuginfo packages are sent to testing network + +If you want to tag multiple specific packages/versions to `testing`, you can proceed with one koji/cbs call : + +``` +cbs tag-build <sig_name>-<project>-<version>-testing <pkg1>-1.0.1 <pkg2>-2.3.4 <and_so_on> +``` + +This will trigger a message on the mqtt-based message bus and intercepted by the isolated machine processing requests. At this stage it will : + + * call koji for a `distRepo` tasks (preparing a usable repository with your packages) and wait for it to finish + * sign repomd.xml file once repositories are all processed for all architectures + * push that to the `buildlogs` CDN + +In the next minutes, your up2date repository will appear on https://buildlogs.centos.org/centos/ and so following the tag convention : + +``` +<sig_name>/ +├── <architecture> +│ ├── <project>-<version> +│ │ ├── Packages +│ │ └── repodata + +``` + +As buildlogs.centos.org has its own specific cdn, you can point your users willing to test your packages directly to such url (in your .repo, see below) + + +## Promoting to release and mirrors + +Once you're satisfied with your package[s] quality (after some testing/feedback, up2you to decide when/how), you can proceed with next step, aka pushing to mirror network. + +Same process as for `testing` except that it's now `release` tag : + +``` +cbs tag-build <sig_name>-<project>-<version>-release <pkg1>-1.0.1 <pkg2>-2.3.4 <and_so_on> +``` + +This will trigger a message on the mqtt-based message bus and intercepted by the isolated machine processing requests. At this stage it will : + + * verify which packages need to be signed with the dedicated gpg key for the <sig> + * download , sign and import back into koji signed packages + * call koji for a `distRepo` tasks (preparing a usable repository with your packages) and wait for it to finish + * sign repomd.xml file once repositories are all processed for all architectures + * push various packages to mirrors, depending on the CentOS Version (see below) : + +### CentOS Linux 7 and CentOS Stream 8 + +The packages will appear on the existing mirror network, divided into three categories : + + * 'normal' rpm packages (that people will download/install) => pushed to mirror.centos.org (and picked up by external mirrors too) + * debuginfo rpm packages => pushed to https://debuginfo.centos.org + * src.rpm packages => pushed to https://vault.centos.org + +### CentOS stream 9 and above + +Starting from CentOS Stream 9, all packages will be pushed out in one simple directory. +All packages will be appearing on http://mirror.stream.centos.org, under the SIGs directory (separated from distro content, for a clear distinction about distro versus SIGs generated content) + +This is how it would look like for Stream 9 : + +``` +SIGs/9-stream/<sig_name>/ +├── <architecture> +│ └── <project>-<version> +│ ├── debug +│ │ └── repodata +│ ├── Packages +│ └── repodata + +``` + +## Consuming rpms packages through .repo definition + +When packages are signed and pushed to mirror network, they are automatically (for the `release` level) checked by the mirror crawler[s] and so you don't need to point your users to either mirror.stream.centos.org or mirror.centos.org. + +Instead you can point to the correct `mirrorlist` or `metalink` url instead, depending on the CentOS Linux/Stream version : + +### CentOS Linux 7 and CentOS Stream 8 + +You can call mirrorlist.centos.org by specifying the repo name, archictecture and centos version like this : + +``` +http://mirrorlist.centos.org?release=<centos_release>&arch=<arch>&repo=<sig_name>-<project>-<version> +``` + +Example for the configmanagement sig producing the ansible (project) 29 (version) repo : + +``` + curl 'http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=configmanagement-ansible-29' +``` + +### CentOS Stream 9 and above + +Starting from CentOS Stream 9 , mirrors are now added in [Fedora Mirrormanager](https://admin.fedoraproject.org/mirrormanager/mirrors/CentOS) so you have to call metalink= instead of mirrorlist= + +As MirrorManager has already plenty of fedora/epel repositories to track, the reponame to use to query mirrormanager for metalink is more complex than for previous mirrorlist.centos.org setup. + +The logic goes like this : + +``` +repo=centos-<sig_name>-sig-<project>-<version>-9-stream&arch=<basearch> +``` + +And the whole metalink url being then (with the `infra` SIG producing the `infra` project with version `common`): + +``` +curl 'https://mirrors.centos.org/metalink?repo=centos-infra-sig-infra-common-9-stream&arch=x86_64' +``` + +### centos-release-<sig> package + +To make it convenient for end-users to add both the .repo files used by dnf/yum to automatically find new repositories, and also to ship the dedicated rpm gpg public key to verify the gpg integrity of the shipped packages, SIGs can build and ship a `centos-release-<sig>` package. + +Worth knowing that such packages have to be built through specific cbs tags (see below) and not *your* SIG tag. +Indeed, SIGs content aren't "trusted" by default (at the rpm gpg level) but 8-strea/9-stream will start distributing the rpm gpg public key that will sign these specific centos-release-* packages, and so end-users will be able to `dnf install centos-release-<blah>` directly. + +Once done, end-users will be able to download/consume your repositories. + +To do so, you can create first an [infra ticket](https://pagure.io/centos-infra/issues) to create a project under the /rpms/ namespace on https://git.centos.org (in case it doesn't exist yet) + +How should you name your 'centos-release' package ? Basically following the centos-release-<project> naming convention (see for example the [`openstack` project](https://git.centos.org/rpms/centos-release-openstack/branches), built by the Cloud SIG, and having multiple <versions> , each version for each supported centos distribution being a different branch) + +At the minimum, your git project for your centos-release package should look like : + +``` +├── .centos-release-<project>.metadata +├── SOURCES +│ ├── CentOS-SIG-<project>.repo +│ └── RPM-GPG-KEY-CentOS-SIG-<name> +└── SPECS + └── centos-release-<project>.spec + +``` + +You can then proceed as [described previously](git.md) to push to git and then [submit a build](cbs.md) against specific tags (verify through `cbs list-permissions --mine` that you can build/tag to specific 'extras' tags. If not, see with your SIG group chair/sponsor + + + +!!! important + Don't submit your build to your own SIG tag : instead use the dedicated `extras<8s,9s>-extras-common-release` tag, that each SIG chair will be able to build for diff --git a/docs/git.md b/docs/git.md index ed4b27a..0d82c7d 100644 --- a/docs/git.md +++ b/docs/git.md @@ -94,6 +94,25 @@ d6616b89617914a0dd0fd5cfa06b0afc7a4541c4 SOURCES/centpkg-minimal.tar.gz Now that we have pointer to lookaside cache, and also .spec, we can push back to git and we should be able to proceed with the "build-from-git" on cbs.centos.org. Let's commit first : +!!! important + Even if your package doesn't contain any source pushed to lookaside cache (like for a package just having some small files in SOURCES/ dir, you *need* to have the .<pkg_name>.metadata package present and pushed in git repository + +Here is what the git repository for a pkg should look like : + +``` +. +├── .<pkg_name>.metadata +├── SOURCES +│ ├── <optional_file> +│ └── <optional_patch> +└── SPECS + └── <pkg_name>.spec + + +``` + + + ``` # git add <files> # if needed git commit -a