zlopez / centos / sig-guide

Forked from centos/sig-guide 2 years ago
Clone

Blame docs/delivery.md

80c633
# Delivery
15677d
90be5d
Here is a quick overview of the Delivery process , from storing sources in git, building on koji/cbs and then pushing packages out
90be5d
90be5d
![CBS-SIGS-workflow](img/CBS-SIGs-workflow.png)
90be5d
90be5d
15677d
## Promoting to testing
15677d
15677d
By default, packages built on cbs are just tagged to `candidate` tag and stay in cbs/koji.
90be5d
90be5d
If you want your packages to get pushed to the [buildlogs](https://buildlogs.centos.org) mirror pool, you can tag packages to `testing`
15677d
15677d
!!! warning
15677d
    Worth knowing that while packages are served over https, and repositories metatdata signed, the packages *aren't* signed with gpg key at this time ! Also good to know that only classical pkgs are pushed out, so no src.rpm nor debuginfo packages are sent to testing network 
15677d
15677d
If you want to tag multiple specific packages/versions to `testing`, you can proceed with one koji/cbs call :
15677d
15677d
```
15677d
cbs tag-build <sig_name>-<project>-<version>-testing <pkg1>-1.0.1 <pkg2>-2.3.4 <and_so_on>
15677d
```
15677d
15677d
This will trigger a message on the mqtt-based message bus and intercepted by the isolated machine processing requests. At this stage it will : 
15677d
15677d
  * call koji for a `distRepo` tasks (preparing a usable repository with your packages) and wait for it to finish
15677d
  * sign repomd.xml file once repositories are all processed for all architectures
15677d
  * push that to the `buildlogs` CDN
15677d
38ca94
In the next minutes, your up2date repository will appear under /centos/ on the [buildlogs](https://buildlogs.centos.org/centos/) nodes and so following the tag convention : 
15677d
15677d
```
15677d
<sig_name>/
15677d
├── <architecture>
15677d
│   ├── <project>-<version>
15677d
│   │   ├── Packages
15677d
│   │   └── repodata
15677d
15677d
```
15677d
38ca94
As [buildlogs.centos.org](https://buildlogs.centos.org) has its own specific cdn, you can point your users willing to test your packages directly to such url (in your .repo, see below)
15677d
15677d
15677d
## Promoting to release and mirrors
15677d
15677d
Once you're satisfied with your package[s] quality (after some testing/feedback, up2you to decide when/how), you can proceed with next step, aka pushing to mirror network.
15677d
15677d
Same process as for `testing` except that it's now `release` tag : 
15677d
15677d
```
15677d
cbs tag-build <sig_name>-<project>-<version>-release <pkg1>-1.0.1 <pkg2>-2.3.4 <and_so_on>
15677d
```
15677d
15677d
This will trigger a message on the mqtt-based message bus and intercepted by the isolated machine processing requests. At this stage it will : 
15677d
38ca94
  * verify which packages need to be signed with the [dedicated gpg key](https://www.centos.org/keys/#community-driven-project-keys) for the `SIG` 
15677d
  * download , sign and import back into koji signed packages
15677d
  * call koji for a `distRepo` tasks (preparing a usable repository with your packages) and wait for it to finish
38ca94
  * sign repomd.xml file once repositories are all processed for all architectures (with dedicated gpg key)
15677d
  * push various packages to mirrors, depending on the CentOS Version (see below) :
15677d
15677d
### CentOS Linux 7 and CentOS Stream 8
15677d
15677d
The packages will appear on the existing mirror network, divided into three categories :
15677d
38ca94
  * 'normal' rpm packages (that people will download/install) => pushed to [mirror.centos.org](http://mirror.centos.org) (and picked up by external mirrors too)
38ca94
  * debuginfo rpm packages => pushed to [debuginfo.centos.org](https://debuginfo.centos.org)
38ca94
  * src.rpm packages => pushed to [Vault](https://vault.centos.org)
15677d
15677d
### CentOS stream 9 and above
15677d
15677d
Starting from CentOS Stream 9, all packages will be pushed out in one simple directory.
38ca94
All packages will be appearing on [mirror.stream.centos.org](http://mirror.stream.centos.org), under the SIGs directory (separated from distro content, for a clear distinction about distro versus SIGs generated content)
15677d
15677d
This is how it would look like for Stream 9 : 
15677d
15677d
```
15677d
SIGs/9-stream/<sig_name>/
15677d
├── <architecture>
15677d
│   └── <project>-<version>
15677d
│       ├── debug
15677d
│       │   └── repodata
15677d
│       ├── Packages
15677d
│       └── repodata
15677d
15677d
```
15677d
15677d
## Consuming rpms packages through .repo definition
15677d
15677d
When packages are signed and pushed to mirror network, they are automatically (for the `release` level) checked by the mirror crawler[s] and so you don't need to point your users to either mirror.stream.centos.org or mirror.centos.org.
15677d
15677d
Instead you can point to the correct `mirrorlist` or `metalink` url instead, depending on the CentOS Linux/Stream version : 
15677d
15677d
### CentOS Linux 7 and CentOS Stream 8
15677d
15677d
You can call mirrorlist.centos.org by specifying the repo name, archictecture and centos version like this : 
15677d
15677d
```
38ca94
mirrorlist=http://mirrorlist.centos.org?release=<centos_release>&arch=<arch>&repo=<sig_name>-<project>-<version>
15677d
```
15677d
15677d
Example for the configmanagement sig producing the ansible (project) 29 (version) repo : 
15677d
15677d
```
15677d
 curl 'http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=configmanagement-ansible-29'
15677d
```
15677d
15677d
### CentOS Stream 9 and above
15677d
15677d
Starting from CentOS Stream 9 , mirrors are now added in [Fedora Mirrormanager](https://admin.fedoraproject.org/mirrormanager/mirrors/CentOS) so you have to call metalink= instead of mirrorlist=
15677d
15677d
As MirrorManager has already plenty of fedora/epel repositories to track, the reponame to use to query mirrormanager for metalink is more complex than for previous mirrorlist.centos.org setup.
15677d
b1a9ac
The logic goes like this and notice the difference for `source` and `debuginfo` packages in the metalink uris:
15677d
15677d
```
b1a9ac
# packages
613175
metalink=https://mirrors.centos.org/metalink?repo=centos-<sig_name>-sig-<project>-<version>-9-stream&arch=<basearch>
b1a9ac
# source packages (.src.rpm)
b1a9ac
metalink=https://mirrors.centos.org/metalink?repo=centos-<sig_name>-sig-<project>-<version>-source-9-stream&arch=source
b1a9ac
# debuginfo packages
b1a9ac
metalink=https://mirrors.centos.org/metalink?repo=centos-<sig_name>-sig-<project>-<version>-debug-9-stream&arch=<basearch>
15677d
```
15677d
b1a9ac
You can also always query MirrorManagers to see which metalinks/repositories it already knows about if you're unsure.
b1a9ac
Here is an example for the `infra` SIG producing the `infra` project with version `common`: 
15677d
15677d
```
b1a9ac
curl 'https://mirrors.centos.org/metalink?repo=unknown&arch=x86_64' -s | grep infra
b1a9ac
# repo=centos-infra-sig-infra-common-9-stream&arch=aarch64
b1a9ac
# repo=centos-infra-sig-infra-common-9-stream&arch=ppc64le
b1a9ac
# repo=centos-infra-sig-infra-common-9-stream&arch=x86_64
b1a9ac
# repo=centos-infra-sig-infra-common-debug-9-stream&arch=aarch64
b1a9ac
# repo=centos-infra-sig-infra-common-debug-9-stream&arch=ppc64le
b1a9ac
# repo=centos-infra-sig-infra-common-debug-9-stream&arch=x86_64
b1a9ac
# repo=centos-infra-sig-infra-common-source-9-stream&arch=source
b1a9ac
15677d
```
15677d
15677d
### centos-release-<sig> package
15677d
15677d
To make it convenient for end-users to add both the .repo files used by dnf/yum to automatically find new repositories, and also to ship the dedicated rpm gpg public key to verify the gpg integrity of the shipped packages, SIGs can build and ship a `centos-release-<sig>` package.
15677d
15677d
Worth knowing that such packages have to be built through specific cbs tags (see below) and not *your* SIG tag.
15677d
Indeed, SIGs content aren't "trusted" by default (at the rpm gpg level) but 8-strea/9-stream will start distributing the rpm gpg public key that will sign these specific centos-release-* packages, and so end-users will be able to `dnf install centos-release-<blah>` directly.
15677d
15677d
Once done, end-users will be able to download/consume your repositories.
15677d
15677d
To do so, you can create first an [infra ticket](https://pagure.io/centos-infra/issues) to create a project under the /rpms/ namespace on https://git.centos.org (in case it doesn't exist yet)
15677d
15677d
How should you name your 'centos-release' package ? Basically following the centos-release-<project> naming convention (see for example the [`openstack` project](https://git.centos.org/rpms/centos-release-openstack/branches), built by the Cloud SIG, and having multiple <versions> , each version for each supported centos distribution being a different branch)
15677d
15677d
At the minimum, your git project for your centos-release package should look like : 
15677d
15677d
```
15677d
├── .centos-release-<project>.metadata
15677d
├── SOURCES
15677d
│   ├── CentOS-SIG-<project>.repo
15677d
│   └── RPM-GPG-KEY-CentOS-SIG-<name>
15677d
└── SPECS
15677d
    └── centos-release-<project>.spec
15677d
15677d
```
15677d
15677d
You can then proceed as [described previously](git.md) to push to git and then [submit a build](cbs.md) against specific tags (verify through `cbs list-permissions --mine` that you can build/tag to specific 'extras' tags. If not, see with your SIG group chair/sponsor
15677d
15677d
15677d
15677d
!!! important
15677d
    Don't submit your build to your own SIG tag : instead use the dedicated `extras<8s,9s>-extras-common-release` tag, that each SIG chair will be able to build for