spotlesstofu / centos / sig-guide

Forked from centos/sig-guide a year ago
Clone

Blame docs/cbs.md

80c633
# Koji/CBS buildsystem
2d9703
2d9703
## CBS permissions
2d9703
2d9703
!!! important
2d9703
    Before being able to submit build tasks to koji/cbs, you need to have a valid [account](auth.md) and be member of a SIG group ! So be sure you followed all steps described in [doc](auth.md) before trying any other step
2d9703
2d9703
If you followed the needed steps to get your account and your workstation prepared with needed packages and TLS certs in place, we can proceed with basic validation.
2d9703
2d9703
Let's try first to confirm that we can reach CBS/koji and be authenticated (assuming that your username is `user` :
2d9703
2d9703
```
2d9703
# cbs hello
2d9703
2d9703
g'day, <user>!
2d9703
2d9703
You are using the hub at https://cbs.centos.org/kojihub/
2d9703
Authenticated via client certificate /home/<user>/.centos.cert
2d9703
2d9703
```
2d9703
2d9703
Now, if you're member of a SIG group (identified as `sig-<groupname>` , the following command should list your all your rights and you should see a corresponding `build-<groupname>` permission : 
2d9703
2d9703
```
2d9703
# cbs list-permissions --mine
2d9703
build
2d9703
livecd
2d9703
appliance
2d9703
image
55cfda
build-<sig_name>
2d9703
livemedia
2d9703
2d9703
```
2d9703
2d9703
As you can see, cbs/koji is using a 1:1 relationship between FAS/ACO group membership (sig-<groupname>) and the corresponding permissions to submit tasks against specific tags (see below) associated with that SIG.
2d9703
2d9703
!!! note
2d9703
    Just worth knowing that when you're added to a SIG group it can take just some minutes due to replication between FreeIPA servers and fasjson.fedoraproject.org to catch up, as that's the endpoint that cbs.centos.org is using to compute/modify rights. So just keep that in mind and wait for permissions to be returned as shown above before trying anything else
2d9703
2d9703
2d9703
## Understanding tags and targets
2d9703
2d9703
### Koji naming convention
2d9703
2d9703
A `build target` specifies the buildroot and destination of a package for submission. That means that as a user you ask koji to build a package within a specific build environment (build target) that is using a specific buildroot to find dependencies, etc. 
2d9703
A `Tag` is how koji is grouping packages all together, and built packages can be automatically added to a tag after a successfull build, or added manually between other tags with the `tag-build` koji task. 
2d9703
2d9703
### CBS SIGs tags structure
2d9703
2d9703
As you understood, each `build target` in CBS koji instance will have at least four tags : 
2d9703
2d9703
  * the -build one (which is where pkg is built and using the configured buildroot)
2d9703
  * three levels that will be used to promote packages after :
2d9703
    * candidate (default tag where built packages are tagged to)
2d9703
    * testing 
2d9703
    * release
2d9703
2d9703
Our koji tag structure *must* follow always the following structure (really for CBS, not the rule for koji but it's important for automation and release to mirror network steps) :
2d9703
2d9703
`<SIG><CentOS_version>-<Project>-<Version>-{candidate,testing,release}` , where:
2d9703
55cfda
  * `SIG` : SIG name , corresponding to IPA group, without the `sig-` prefix (so `sig-cloud` would become `cloud`)
55cfda
  * `CentOS Version`: .el version, so el7 means CentOS 7, el8s means CentOS Stream 8, and so on
55cfda
  * `Project` : the project that the SIG want to build (example `openstack`)
55cfda
  * `Version` : project version, can be a number or a string (example `yoga` for openstack version)
55cfda
  * `repo variant|tag`: one of the following : candidate, testing or release (see Delivery about the meaning for these levels)
2d9703
2d9703
2d9703
Let's use one example, the `Cloud` SIG would like to build Openstack (Yoga release) for CentOS Stream 9
2d9703
2d9703
The build target would be `cloud9s-openstack-yoga-el9s` and the tags would be : cloud9s-openstack-yoga-{candidate,testing,release}
2d9703
631d73
!!! note
2d9703
631d73
    When requesting a new build target via an [infra ticket](https://pagure.io/centos-infra/issues), also request the external repositories your target may need to be added (e.g epel, epel-next...).
2d9703
2d9703
### Submit a build on CBS
2d9703
f47f0b
Before being able to submit a build task in koji, one has to be the pkg owner for it for all possible tags, so if you want to become the "owner" of a pkg for your own SIG/group, you can proceed like this (and replace `ACO_name`, `pkg_name` and `basetag` with real values) :
2d9703
2d9703
```
2d9703
for level in candidate testing release ; do
55cfda
  cbs add-pkg --owner=<ACO_name> <SIG><CentOS_version>-<Project>-<Version>-${level} <pkg_name>
2d9703
done
2d9703
```
2d9703
55cfda
Now you can submit a build task again your build target by pointing to git.centos.org URL, pointing to project to build and git commit hash (assuming that you proceeded with needed steps in the [git](git.md) part :
2d9703
2d9703
```
55cfda
cbs build <SIG><CentOS_version>-<Project>-<Version>-el<CentOS_version> git+https://git.centos.org/rpms/<pkg_name>.git#<commit_hash>
2d9703
```
2d9703
2d9703
!!! note
2d9703
    you can always submit a test build by adding `--scratch` to the build command as argument : that would really build the pkg but wouldn't make it available anywhere to be used. Please avoid using --scratch though as it consumes resources on the build systems for nothing
2d9703
f47f0b
### cbs/koji common error messages
f47f0b
f47f0b
#### Build already exists
f47f0b
f47f0b
If you want to submit a build and get the following message, it means that same package/version was already built in koji by someone else :
f47f0b
f47f0b
```
f47f0b
GenericError: Build already exists
f47f0b
```
f47f0b
As a reminder, each build in koji/cbs needs to be unique and follows the ENVR (Epoch Name Version Release) RPM convention.
f47f0b
So if you need an already built package in your tags (built by another SIG), you can also `tag-build` it to your own tag. (that assumes that you already added the pkg to your tag, per instructions above)
f47f0b
f47f0b
The syntax can be a simple as (pkg_name should be the full rpm package without .rpm at the end): 
f47f0b
f47f0b
```
f47f0b
cbs tag-build <your_tag-{testing,release,candidate} <pkg_name>-1.0.1.el8 
f47f0b
```
f47f0b
f47f0b
#### Expired client TLS cert
f47f0b
f47f0b
```
f47f0b
"Error: [('SSL routines','SSL_shutdown', 'shutdown while in init')]
f47f0b
```
f47f0b
That means that your ACO cert (obtained with `centos-cert`) has expired.
f47f0b
You can always confirm that with `centos-cert -v` :
f47f0b
f47f0b
```
f47f0b
centos-cert -v
f47f0b
f47f0b
[+] 20211222-11:36 centos-cert -> Verifying if TLS cert is still valid ...
f47f0b
[+] 20211222-11:36 centos-cert -> Validating TLS cert against ~/.centos-server-ca.cert ...
f47f0b
/home/<your_username>/.centos.cert: OK
f47f0b
[+] 20211222-11:36 centos-cert -> [SUCCESS] ~/.centos.cert TLS cert verified by ~/.centos-server-ca.cert CA crt
f47f0b
[+] 20211222-11:36 centos-cert -> [SUCCESS] Your TLS cert is still valid for [61] days
f47f0b
f47f0b
```
f47f0b
f47f0b
If it expired, just use centos-cert as the first time you used it to obtain a new certificate
2d9703
2d9703
### Promoting packages to mirror network
2d9703
2d9703
See dedicated [Delivery](delivery.md) documentation