zlopez / centos / sig-guide

Forked from centos/sig-guide 2 years ago
Clone

Blame docs/git.md

fea739
# Lookaside/git
2d9703
fea739
This page documents how SIGs can upload the sources of their RPM into the lookaside
fea739
cache and how they should set-up their corresponding git repository.
2d9703
fea739
It should be noted that two model exists, the "traditional" model and the one
fea739
based on CentOS Stream and Fedora. While both model can be intertwined  in some
fea739
places, it is recommended that SIGs pick a model and follow it. This documentation
fea739
will not cover which aspect of which model is compatible with the other one.
2d9703
fea739
## New Package (from source)
2d9703
fea739
If you SIG uses [https://git.centos.org](https://git.centos.org) and the package
fea739
you want to build is not there yet, you will have to create a ticket first at:
fea739
[https://pagure.io/centos-infra/issues](https://pagure.io/centos-infra/issues)
fea739
to request that the git repo for `/rpms/<your_package>` be created.
2d9703
fea739
If your SIG uses [https://gitlab.com/CentOS](https://gitlab.com/CentOS), you,
fea739
your SIG chair or people that have been granted access by the SIG chair, will be
fea739
able to create the project for your package directly.
2d9703
fea739
In the rest of the document we will use the following example : Build the package `centpkg-minimal`,
fea739
as a member of the `sig-core` SIG group. We have to push two things:
2d9703
fea739
 * the .spec file to build the rpm, to the git repository
fea739
 * eventually an archive (.tar.gz, .tar, .xz...) of the sources, to lookaside cache
2d9703
2d9703
!!! warning
f47f0b
    To push to lookaside cache you need to have already a local check out from [https://git.centos.org/centos-git-common](https://git.centos.org/centos-git-common)
2d9703
2d9703
## Pushing first to lookaside cache
2d9703
2d9703
Let's assume that my pkg centpkg-minimal that I want to build has an archive called centpkg.minimal.tar.gz. To be able to push to lookaside cache, we need the following :
2d9703
2d9703
 * a valid TLS cert (also needed to build on cbs.centos.org) obtained through [centos-cert](../auth/) util
fea739
 * either script `lookaside_upload` or `lookaside_upload_sig` in your $PATH (coming from centos-git-common git repo, see above)
fea739
fea739
There are two path available for the lookaside cache, each is available using a different script:
fea739
fea739
- `lookaside_upload` allows uploading to the traditional CentOS Linux structure: `baseurl/pkgname/branch/hash` ([example](
fea739
https://git.centos.org/sources/kernel/c8s/0c4e10577cfd4b4f8e3d83c0406da8ab05eb775f))
fea739
- `lookaside_upload_sig` allows uploading to the same lookaside structure as the one used by CentOS Stream and Fedora: `baseurl/pkgname/tarball/hashtype/hash/tarball` ([example](
fea739
https://sources.stream.centos.org/sources/rpms/kernel/linux-5.14.0-62.el9.tar.xz/sha512/f7aeac0fe5bf594933cd35b7ecc94ea8ddcbfedc04fa769c4da298e7bf105df116375d44711d944c748c85f61f96f6149be34c76eb37f28aa1f16359a9122abf/linux-5.14.0-62.el9.tar.xz))
fea739
fea739
Both scripts need some paramters, simply use `-h` to see them.
2d9703
fea739
Back to our example.
fea739
fea739
### Pushing to the "traditional" lookaside structure
fea739
fea739
The pkg name is centpkg-minimal, file is centpkg-minimal.tar.gz and I'm member of the sig-core group, and want to build it for c7, so we'll call it like this :
2d9703
2d9703
```
fea739
lookaside_upload -f centpkg-minimal.tar.gz -n centpkg-minimal -b c7-sig-core
fea739
[+] CentOS Lookaside upload tool -> Checking if file already uploaded
fea739
[+] CentOS Lookaside upload tool -> Initialing new upload to lookaside
fea739
[+] CentOS Lookaside upload tool -> URL : https://git.centos.org
fea739
[+] CentOS Lookaside upload tool -> Source to upload : centpkg-minimal.tar.gz
fea739
[+] CentOS Lookaside upload tool -> Package name: centpkg-minimal
fea739
[+] CentOS Lookaside upload tool -> sha1sum: d6616b89617914a0dd0fd5cfa06b0afc7a4541c4
fea739
[+] CentOS Lookaside upload tool -> Remote branch: c7-sig-core
fea739
[+] CentOS Lookaside upload tool ->  ====== Trying to upload =======
2d9703
fea739
################################################################################################################ 100.0%
fea739
File centpkg-minimal.tar.gz size 15178 CHECKSUM d6616b89617914a0dd0fd5cfa06b0afc7a4541c4 stored OK
fea739
[+] CentOS Lookaside upload tool -> Validating that source was correctly uploaded ....
fea739
[+] CentOS Lookaside upload tool -> [SUCCESS] Source should be available at https://git.centos.org/sources/centpkg-minimal/c7-sig-core/d6616b89617914a0dd0fd5cfa06b0afc7a4541c4
2d9703
2d9703
```
2d9703
fea739
### Pushing to the CentOS Stream/Fedora lookaside structure
fea739
fea739
The pkg name is centpkg-minimal, file is centpkg-minimal.tar.gz, so we'll call it like this :
2d9703
2d9703
```
fea739
lookaside_upload_sig -f centpkg-minimal.tar.gz -n centpkg-minimal
2d9703
[+] CentOS Lookaside upload tool -> Checking if file already uploaded
2d9703
[+] CentOS Lookaside upload tool -> Initialing new upload to lookaside
2d9703
[+] CentOS Lookaside upload tool -> URL : https://git.centos.org
fea739
[+] CentOS Lookaside upload tool -> Source to upload : centpkg-minimal.tar.gz
2d9703
[+] CentOS Lookaside upload tool -> Package name: centpkg-minimal
2d9703
[+] CentOS Lookaside upload tool -> sha1sum: d6616b89617914a0dd0fd5cfa06b0afc7a4541c4
2d9703
[+] CentOS Lookaside upload tool -> Remote branch: c7-sig-core
2d9703
[+] CentOS Lookaside upload tool ->  ====== Trying to upload =======
2d9703
2d9703
################################################################################################################ 100.0%
2d9703
File centpkg-minimal.tar.gz size 15178 CHECKSUM d6616b89617914a0dd0fd5cfa06b0afc7a4541c4 stored OK
2d9703
[+] CentOS Lookaside upload tool -> Validating that source was correctly uploaded ....
2d9703
[+] CentOS Lookaside upload tool -> [SUCCESS] Source should be available at https://git.centos.org/sources/centpkg-minimal/c7-sig-core/d6616b89617914a0dd0fd5cfa06b0afc7a4541c4
2d9703
2d9703
```
2d9703
fea739
Now that we have uploaded to lookaside cache, we can reference it in our /rpms/centpkg-minimal git repository on git.centos.org, see below
fea739
fea739
fea739
## Pushing to git
fea739
fea739
SIGs have two options when it comes to the git repositories which can be used as part of dist-git:
2d9703
fea739
- [git.centos.org](https://git.centos.org)
fea739
- [gitlab.com/CentOS](https://gitlab.com/CentOS)
2d9703
fea739
If your SIG wants to use gitlab, they will have to follow the [instructions](gitlab.md) to request one.
fea739
Here we'll cover both workflow, assuming the git repository exists on either platform.
2d9703
fea739
### Git clone
2d9703
fea739
The first thing you'll need is a local checkout of the git repository for the pkg we'd like to work on.
fea739
In our case, the git repo url is `https://git.centos.org/rpms/centpkg-minimal` or
fea739
`https://gitlab.com/CentOS/<sig>/rpms/<repo>.git`
fea739
fea739
So the way to git clone/pull over ssh is either `ssh://git@git.centos.org/rpms/centpkg-minimal.git` or
fea739
`git@gitlab.com:CentOS/<sign>/rpms/<repo>.git`:
2d9703
2d9703
```
2d9703
git clone ssh://git@git.centos.org/rpms/centpkg-minimal.git
2d9703
Cloning into 'centpkg-minimal'...
2d9703
remote: Counting objects: 15, done.
2d9703
remote: Compressing objects: 100% (11/11), done.
2d9703
remote: Total 15 (delta 2), reused 0 (delta 0)
2d9703
Receiving objects: 100% (15/15), done.
2d9703
Resolving deltas: 100% (2/2), done.
2d9703
```
2d9703
2d9703
!!! important
fea739
    On git.centos.org you can clone over https, but then it wouldn't let you push back to it, as it needs ssh with key verification and based on group membership for acls
fea739
fea739
### Git branches
fea739
fea739
Repositories git.centos.org are used by both Red Hat and CentOS SIG, the structure of the git
fea739
repository is used to control the access. SIG members are therefore only allowed to push to branches
fea739
following the pattern: `c<version>-sig-<sig_name>*`
2d9703
fea739
That means that if I'm member of the sig-core group (in ACO), I'll be able to
fea739
commit/push to `c7-sig-core`, `c8-sig-core` or `c9s-sig-core-whatever-I-want` branches
fea739
fea739
For our example, on git.centos.org we'll have to create a new c7-sig-core branch
fea739
(with on the assumption that we want to build for c7) :
2d9703
2d9703
```
2d9703
git checkout -b c7-sig-core
2d9703
```
2d9703
fea739
Since SIGs have full control over their namespace on gitlab.com, that branching structure
fea739
is not mandatory. It will, however, be necessary if you plan on using the "traditional" lookaside
fea739
structure describe above as this relies on branch names.
fea739
2d9703
fea739
### Git repository layout
fea739
fea739
There are two possible structure possible for the repository layout:
fea739
fea739
- The exploded SRPM layout
fea739
- The flat dist-git layout
fea739
fea739
#### The exploded SRPM layout
fea739
fea739
This is the "traditional" layout used in CentOS Linux, it is articulated around two folders:
2d9703
```
fea739
├── SOURCES
fea739
└── SPECS
2d9703
```
2d9703
fea739
The `SPECS` folder is meant to receive your spec file and all other text files and
fea739
patches would go under `SOURCES`:
fea739
fea739
```
fea739
├── SOURCES
fea739
│   ├── <optional_file>
fea739
│   └── <optional_patch>
fea739
└── SPECS
fea739
    └── <pkg_name>.spec
fea739
```
fea739
fea739
Once done, we still need to link these files with the sources uploaded earlier in the
fea739
lookaside cache. See the next section.
fea739
fea739
#### The flat dist-git layout
fea739
fea739
This is the layout used in the dist-git repositories in CentOS Stream and Fedora.
fea739
All files (spec files, text files, patches...) are stored at the top level of
fea739
the repository:
fea739
fea739
```
fea739
├── <optional_file>
fea739
├── <optional_patch>
fea739
└── <pkg_name>.spec
fea739
```
fea739
fea739
### Linking to sources in the lookaside cache
fea739
fea739
Here as well, there are two ways you can link a dist-git repository to sources
fea739
uploaded to the lookaside cache:
fea739
fea739
- the "traditional" way using a `.<pkg_name>.metadata` file
fea739
- the CentOS Stream/Fedora way using a `sources` file
2d9703
15677d
!!! important
fea739
    Even if your package doesn't contain any source pushed to lookaside cache
fea739
    (like for a package just having some small files in SOURCES/ dir, you *need* to
fea739
    have either a `.<pkg_name>.metadata` or `sources` file present and pushed in
fea739
    git repository
15677d
fea739
#### Using a `.<pkg_name>.metadata` file
fea739
fea739
To use this method, simply create a `.<pkg_name>.metadata` file with the following
fea739
content:
fea739
```
fea739
<sha1 of the tarball> SOURCES/<filename>
fea739
```
fea739
fea739
For example:
fea739
```
fea739
d6616b89617914a0dd0fd5cfa06b0afc7a4541c4 SOURCES/centpkg-minimal.tar.gz
fea739
```
fea739
fea739
#### Using a `sources` file
fea739
fea739
To use this method, simply create a `sources` file. The easiest way to achieve
fea739
this is:
fea739
```
fea739
sha512sum --tag <tarball> > sources
fea739
```
fea739
fea739
Example output:
fea739
```
fea739
SHA512 (centpkg-minimal-1.1.0.tar.gz) = f7cfbc956199b1a0342f321a0e4cb055d6ac2c784a8faace43cf80772f0de34b58ede1a02a558cd03e25c14938ac6e17b3746b1919c0bbc1f5b2955472577e4f
fea739
```
fea739
fea739
Now that we have pointer to lookaside cache, and also .spec, we can push back to git and
fea739
we should be able to proceed with the "build-from-git" on cbs.centos.org. Let's commit first.
fea739
fea739
### Git push
fea739
fea739
Depending on if you picked the "traditional" structure or the CentOS Stream/Fedora
fea739
one, this is how your repository should look like:
fea739
fea739
* Traditional layout
15677d
15677d
```
15677d
.
15677d
├── .<pkg_name>.metadata
15677d
├── SOURCES
15677d
│   ├── <optional_file>
15677d
│   └── <optional_patch>
15677d
└── SPECS
15677d
    └── <pkg_name>.spec
fea739
```
15677d
fea739
* CentOS Stream/Fedora layout
15677d
15677d
```
fea739
.
fea739
├── sources
fea739
├── <optional_file>
fea739
├── <optional_patch>
fea739
└── <pkg_name>.spec
fea739
```
15677d
fea739
We can now push to to git, as usual:
15677d
2d9703
```
2d9703
# git add <files> # if needed
fea739
git commit -a
fea739
git push origin <branch> # to create the remote branch if not existing yet
2d9703
```
f47f0b
fea739
Where `<branch>` would be `c7-sig-core` for our example using `git.centos.org`.
f47f0b
a5bc48
fea739
Now that we have our sources pushed to both git.centos.org and lookaside cache,
fea739
we can now proceed with a [build in cbs/koji](cbs.md).