zlopez / centos / sig-guide

Forked from centos/sig-guide 2 years ago
Clone
Blob Blame History Raw
# Delivery

Here is a quick overview of the Delivery process , from storing sources in git, building on koji/cbs and then pushing packages out

![CBS-SIGS-workflow](img/CBS-SIGs-workflow.png)


## Promoting to testing

By default, packages built on cbs are just tagged to `candidate` tag and stay in cbs/koji.

If you want your packages to get pushed to the [buildlogs](https://buildlogs.centos.org) mirror pool, you can tag packages to `testing`

!!! warning
    Worth knowing that while packages are served over https, and repositories metatdata signed, the packages *aren't* signed with gpg key at this time ! Also good to know that only classical pkgs are pushed out, so no src.rpm nor debuginfo packages are sent to testing network 

If you want to tag multiple specific packages/versions to `testing`, you can proceed with one koji/cbs call :

```
cbs tag-build <sig_name>-<project>-<version>-testing <pkg1>-1.0.1 <pkg2>-2.3.4 <and_so_on>
```

This will trigger a message on the mqtt-based message bus and intercepted by the isolated machine processing requests. At this stage it will : 

  * call koji for a `distRepo` tasks (preparing a usable repository with your packages) and wait for it to finish
  * sign repomd.xml file once repositories are all processed for all architectures
  * push that to the `buildlogs` CDN

In the next minutes, your up2date repository will appear under /centos/ on the [buildlogs](https://buildlogs.centos.org/centos/) nodes and so following the tag convention : 

```
<sig_name>/
├── <architecture>
│   ├── <project>-<version>
│   │   ├── Packages
│   │   └── repodata

```

As [buildlogs.centos.org](https://buildlogs.centos.org) has its own specific cdn, you can point your users willing to test your packages directly to such url (in your .repo, see below)


## Promoting to release and mirrors

Once you're satisfied with your package[s] quality (after some testing/feedback, up2you to decide when/how), you can proceed with next step, aka pushing to mirror network.

Same process as for `testing` except that it's now `release` tag : 

```
cbs tag-build <sig_name>-<project>-<version>-release <pkg1>-1.0.1 <pkg2>-2.3.4 <and_so_on>
```

This will trigger a message on the mqtt-based message bus and intercepted by the isolated machine processing requests. At this stage it will : 

  * verify which packages need to be signed with the [dedicated gpg key](https://www.centos.org/keys/#community-driven-project-keys) for the `SIG` 
  * download , sign and import back into koji signed packages
  * call koji for a `distRepo` tasks (preparing a usable repository with your packages) and wait for it to finish
  * sign repomd.xml file once repositories are all processed for all architectures (with dedicated gpg key)
  * push various packages to mirrors, depending on the CentOS Version (see below) :

### CentOS Linux 7 and CentOS Stream 8

The packages will appear on the existing mirror network, divided into three categories :

  * 'normal' rpm packages (that people will download/install) => pushed to [mirror.centos.org](http://mirror.centos.org) (and picked up by external mirrors too)
  * debuginfo rpm packages => pushed to [debuginfo.centos.org](https://debuginfo.centos.org)
  * src.rpm packages => pushed to [Vault](https://vault.centos.org)

### CentOS stream 9 and above

Starting from CentOS Stream 9, all packages will be pushed out in one simple directory.
All packages will be appearing on [mirror.stream.centos.org](http://mirror.stream.centos.org), under the SIGs directory (separated from distro content, for a clear distinction about distro versus SIGs generated content)

This is how it would look like for Stream 9 : 

```
SIGs/9-stream/<sig_name>/
├── <architecture>
│   └── <project>-<version>
│       ├── debug
│       │   └── repodata
│       ├── Packages
│       └── repodata

```

## Consuming rpms packages through .repo definition

When packages are signed and pushed to mirror network, they are automatically (for the `release` level) checked by the mirror crawler[s] and so you don't need to point your users to either mirror.stream.centos.org or mirror.centos.org.

Instead you can point to the correct `mirrorlist` or `metalink` url instead, depending on the CentOS Linux/Stream version : 

### CentOS Linux 7 and CentOS Stream 8

You can call mirrorlist.centos.org by specifying the repo name, archictecture and centos version like this : 

```
mirrorlist=http://mirrorlist.centos.org?release=<centos_release>&arch=<arch>&repo=<sig_name>-<project>-<version>
```

Example for the configmanagement sig producing the ansible (project) 29 (version) repo : 

```
 curl 'http://mirrorlist.centos.org/?release=8&arch=x86_64&repo=configmanagement-ansible-29'
```

### CentOS Stream 9 and above

Starting from CentOS Stream 9 , mirrors are now added in [Fedora Mirrormanager](https://admin.fedoraproject.org/mirrormanager/mirrors/CentOS) so you have to call metalink= instead of mirrorlist=

As MirrorManager has already plenty of fedora/epel repositories to track, the reponame to use to query mirrormanager for metalink is more complex than for previous mirrorlist.centos.org setup.

The logic goes like this : 

```
metalink=https://mirrors.centos.org/metalink?repo=centos-<sig_name>-sig-<project>-<version>-9-stream&arch=<basearch>
```

And the whole metalink url being then (with the `infra` SIG producing the `infra` project with version `common`): 

```
curl 'https://mirrors.centos.org/metalink?repo=centos-infra-sig-infra-common-9-stream&arch=x86_64'
```

### centos-release-<sig> package

To make it convenient for end-users to add both the .repo files used by dnf/yum to automatically find new repositories, and also to ship the dedicated rpm gpg public key to verify the gpg integrity of the shipped packages, SIGs can build and ship a `centos-release-<sig>` package.

Worth knowing that such packages have to be built through specific cbs tags (see below) and not *your* SIG tag.
Indeed, SIGs content aren't "trusted" by default (at the rpm gpg level) but 8-strea/9-stream will start distributing the rpm gpg public key that will sign these specific centos-release-* packages, and so end-users will be able to `dnf install centos-release-<blah>` directly.

Once done, end-users will be able to download/consume your repositories.

To do so, you can create first an [infra ticket](https://pagure.io/centos-infra/issues) to create a project under the /rpms/ namespace on https://git.centos.org (in case it doesn't exist yet)

How should you name your 'centos-release' package ? Basically following the centos-release-<project> naming convention (see for example the [`openstack` project](https://git.centos.org/rpms/centos-release-openstack/branches), built by the Cloud SIG, and having multiple <versions> , each version for each supported centos distribution being a different branch)

At the minimum, your git project for your centos-release package should look like : 

```
├── .centos-release-<project>.metadata
├── SOURCES
│   ├── CentOS-SIG-<project>.repo
│   └── RPM-GPG-KEY-CentOS-SIG-<name>
└── SPECS
    └── centos-release-<project>.spec

```

You can then proceed as [described previously](git.md) to push to git and then [submit a build](cbs.md) against specific tags (verify through `cbs list-permissions --mine` that you can build/tag to specific 'extras' tags. If not, see with your SIG group chair/sponsor



!!! important
    Don't submit your build to your own SIG tag : instead use the dedicated `extras<8s,9s>-extras-common-release` tag, that each SIG chair will be able to build for