| |
@@ -1,19 +1,29 @@
|
| |
- # Git/lookaside
|
| |
+ # Lookaside/git
|
| |
|
| |
- Package sources checked into git.centos.org are in exploded SRPM format. This means that the package working directory should have at least a SPECS/ subdirectory.
|
| |
+ This page documents how SIGs can upload the sources of their RPM into the lookaside
|
| |
+ cache and how they should set-up their corresponding git repository.
|
| |
|
| |
- ## New Package (from source)
|
| |
+ It should be noted that two model exists, the "traditional" model and the one
|
| |
+ based on CentOS Stream and Fedora. While both model can be intertwined in some
|
| |
+ places, it is recommended that SIGs pick a model and follow it. This documentation
|
| |
+ will not cover which aspect of which model is compatible with the other one.
|
| |
|
| |
- If the package you'd like to build isn't yet on https://git.centos.org, create first a ticket on https://pagure.io/centos-infra/issues to request the git repo for /rpms/<your_package> to be created
|
| |
+ ## New Package (from source)
|
| |
|
| |
- Once it will be created, it will be possible for you to push to *specific* branches, based on the following rules: c{7,8}-<sig_name>*
|
| |
+ If you SIG uses [https://git.centos.org](https://git.centos.org) and the package
|
| |
+ you want to build is not there yet, you will have to create a ticket first at:
|
| |
+ [https://pagure.io/centos-infra/issues](https://pagure.io/centos-infra/issues)
|
| |
+ to request that the git repo for `/rpms/<your_package>` be created.
|
| |
|
| |
- So that means that if I'm member of the sig-core group (in ACO), I'll be able to commit/push to c8-sig-core or c8-sig-core-whatever-I-want branches
|
| |
+ If your SIG uses [https://gitlab.com/CentOS](https://gitlab.com/CentOS), you,
|
| |
+ your SIG chair or people that have been granted access by the SIG chair, will be
|
| |
+ able to create the project for your package directly.
|
| |
|
| |
- Let's use the following example : I need to create a branch for pkg centpkg-minimal, and I'm member of the sig-core SIG group. We have to push two things to git.centos.org :
|
| |
+ In the rest of the document we will use the following example : Build the package `centpkg-minimal`,
|
| |
+ as a member of the `sig-core` SIG group. We have to push two things:
|
| |
|
| |
- * .spec files to build a src.rpm
|
| |
- * eventually blob (.tar.gz archive) that will need to be pushed to lookaside cache
|
| |
+ * the .spec file to build the rpm, to the git repository
|
| |
+ * eventually an archive (.tar.gz, .tar, .xz...) of the sources, to lookaside cache
|
| |
|
| |
!!! warning
|
| |
To push to lookaside cache you need to have already a local check out from [https://git.centos.org/centos-git-common](https://git.centos.org/centos-git-common)
|
| |
@@ -23,29 +33,51 @@
|
| |
Let's assume that my pkg centpkg-minimal that I want to build has an archive called centpkg.minimal.tar.gz. To be able to push to lookaside cache, we need the following :
|
| |
|
| |
* a valid TLS cert (also needed to build on cbs.centos.org) obtained through [centos-cert](../auth/) util
|
| |
- * lookaside_upload script in your $PATH (coming from centos-git-common git repo, see above)
|
| |
+ * either script `lookaside_upload` or `lookaside_upload_sig` in your $PATH (coming from centos-git-common git repo, see above)
|
| |
+
|
| |
+ There are two path available for the lookaside cache, each is available using a different script:
|
| |
+
|
| |
+ - `lookaside_upload` allows uploading to the traditional CentOS Linux structure: `baseurl/pkgname/branch/hash` ([example](
|
| |
+ https://git.centos.org/sources/kernel/c8s/0c4e10577cfd4b4f8e3d83c0406da8ab05eb775f))
|
| |
+ - `lookaside_upload_sig` allows uploading to the same lookaside structure as the one used by CentOS Stream and Fedora: `baseurl/pkgname/tarball/hashtype/hash/tarball` ([example](
|
| |
+ https://sources.stream.centos.org/sources/rpms/kernel/linux-5.14.0-62.el9.tar.xz/sha512/f7aeac0fe5bf594933cd35b7ecc94ea8ddcbfedc04fa769c4da298e7bf105df116375d44711d944c748c85f61f96f6149be34c76eb37f28aa1f16359a9122abf/linux-5.14.0-62.el9.tar.xz))
|
| |
+
|
| |
+ Both scripts need some paramters, simply use `-h` to see them.
|
| |
|
| |
- This simple script would need some paramters:
|
| |
+ Back to our example.
|
| |
+
|
| |
+ ### Pushing to the "traditional" lookaside structure
|
| |
+
|
| |
+ The pkg name is centpkg-minimal, file is centpkg-minimal.tar.gz and I'm member of the sig-core group, and want to build it for c7, so we'll call it like this :
|
| |
|
| |
```
|
| |
- lookaside_upload
|
| |
+ lookaside_upload -f centpkg-minimal.tar.gz -n centpkg-minimal -b c7-sig-core
|
| |
+ [+] CentOS Lookaside upload tool -> Checking if file already uploaded
|
| |
+ [+] CentOS Lookaside upload tool -> Initialing new upload to lookaside
|
| |
+ [+] CentOS Lookaside upload tool -> URL : https://git.centos.org
|
| |
+ [+] CentOS Lookaside upload tool -> Source to upload : centpkg-minimal.tar.gz
|
| |
+ [+] CentOS Lookaside upload tool -> Package name: centpkg-minimal
|
| |
+ [+] CentOS Lookaside upload tool -> sha1sum: d6616b89617914a0dd0fd5cfa06b0afc7a4541c4
|
| |
+ [+] CentOS Lookaside upload tool -> Remote branch: c7-sig-core
|
| |
+ [+] CentOS Lookaside upload tool -> ====== Trying to upload =======
|
| |
|
| |
- You need to call the script like this : ~/bin/lookaside_upload -arguments
|
| |
+ ################################################################################################################ 100.0%
|
| |
+ File centpkg-minimal.tar.gz size 15178 CHECKSUM d6616b89617914a0dd0fd5cfa06b0afc7a4541c4 stored OK
|
| |
+ [+] CentOS Lookaside upload tool -> Validating that source was correctly uploaded ....
|
| |
+ [+] CentOS Lookaside upload tool -> [SUCCESS] Source should be available at https://git.centos.org/sources/centpkg-minimal/c7-sig-core/d6616b89617914a0dd0fd5cfa06b0afc7a4541c4
|
| |
|
| |
- -f : filename/source to upload (required, default:none)
|
| |
- -n : package name for that source (requred, default:none, example "httpd")
|
| |
- -b : "branch" where to upload to (required, default:none, example "c7-sig-core")
|
| |
- -h : display this help
|
| |
```
|
| |
|
| |
- So back to our example, the pkg name is centpkg-minimal, file is centpkg-minimal.tar.gz and I'm member of the sig-core group, and want to build it for c7, so we'll call it like this :
|
| |
+ ### Pushing to the CentOS Stream/Fedora lookaside structure
|
| |
+
|
| |
+ The pkg name is centpkg-minimal, file is centpkg-minimal.tar.gz, so we'll call it like this :
|
| |
|
| |
```
|
| |
- lookaside_upload -f centpkg-minimal.tar.gz -n centpkg-minimal -b c7-sig-core
|
| |
+ lookaside_upload_sig -f centpkg-minimal.tar.gz -n centpkg-minimal
|
| |
[+] CentOS Lookaside upload tool -> Checking if file already uploaded
|
| |
[+] CentOS Lookaside upload tool -> Initialing new upload to lookaside
|
| |
[+] CentOS Lookaside upload tool -> URL : https://git.centos.org
|
| |
- [+] CentOS Lookaside upload tool -> Source to upload : centpkg-minimal.tar.gz
|
| |
+ [+] CentOS Lookaside upload tool -> Source to upload : centpkg-minimal.tar.gz
|
| |
[+] CentOS Lookaside upload tool -> Package name: centpkg-minimal
|
| |
[+] CentOS Lookaside upload tool -> sha1sum: d6616b89617914a0dd0fd5cfa06b0afc7a4541c4
|
| |
[+] CentOS Lookaside upload tool -> Remote branch: c7-sig-core
|
| |
@@ -58,14 +90,27 @@
|
| |
|
| |
```
|
| |
|
| |
- Now that we have uploaded to lookaside cache, we can reference it in our /rpms/centpkg-minimal git repository on git.centos.org, see below
|
| |
+ Now that we have uploaded to lookaside cache, we can reference it in our /rpms/centpkg-minimal git repository on git.centos.org, see below
|
| |
+
|
| |
+
|
| |
+ ## Pushing to git
|
| |
+
|
| |
+ SIGs have two options when it comes to the git repositories which can be used as part of dist-git:
|
| |
|
| |
+ - [git.centos.org](https://git.centos.org)
|
| |
+ - [gitlab.com/CentOS](https://gitlab.com/CentOS)
|
| |
|
| |
- ## Pushing to git.centos.org
|
| |
+ If your SIG wants to use gitlab, they will have to follow the [instructions](gitlab.md) to request one.
|
| |
+ Here we'll cover both workflow, assuming the git repository exists on either platform.
|
| |
|
| |
- We should already have a local checkout of the git repository for the pkg we'd like to work on. In our case, the git repo url is https://git.centos.org/rpms/centpkg-minimal
|
| |
+ ### Git clone
|
| |
|
| |
- So the way to git clone/pull over ssh is so ssh://git@git.centos.org/rpms/centpkg-minimal.git :
|
| |
+ The first thing you'll need is a local checkout of the git repository for the pkg we'd like to work on.
|
| |
+ In our case, the git repo url is `https://git.centos.org/rpms/centpkg-minimal` or
|
| |
+ `https://gitlab.com/CentOS/<sig>/rpms/<repo>.git`
|
| |
+
|
| |
+ So the way to git clone/pull over ssh is either `ssh://git@git.centos.org/rpms/centpkg-minimal.git` or
|
| |
+ `git@gitlab.com:CentOS/<sign>/rpms/<repo>.git`:
|
| |
|
| |
```
|
| |
git clone ssh://git@git.centos.org/rpms/centpkg-minimal.git
|
| |
@@ -78,26 +123,119 @@
|
| |
```
|
| |
|
| |
!!! important
|
| |
- You can clone over https, but then it wouldn't let you push back to it, as it needs ssh with key verification and based on group membership for acls
|
| |
+ On git.centos.org you can clone over https, but then it wouldn't let you push back to it, as it needs ssh with key verification and based on group membership for acls
|
| |
+
|
| |
+ ### Git branches
|
| |
+
|
| |
+ Repositories git.centos.org are used by both Red Hat and CentOS SIG, the structure of the git
|
| |
+ repository is used to control the access. SIG members are therefore only allowed to push to branches
|
| |
+ following the pattern: `c<version>-sig-<sig_name>*`
|
| |
|
| |
- Let's create a new c7-sig-core branch (still based on the assumption that I want to build for c7) :
|
| |
+ That means that if I'm member of the sig-core group (in ACO), I'll be able to
|
| |
+ commit/push to `c7-sig-core`, `c8-sig-core` or `c9s-sig-core-whatever-I-want` branches
|
| |
+
|
| |
+ For our example, on git.centos.org we'll have to create a new c7-sig-core branch
|
| |
+ (with on the assumption that we want to build for c7) :
|
| |
|
| |
```
|
| |
git checkout -b c7-sig-core
|
| |
```
|
| |
|
| |
- You can now create your SPECS/<pkg_name>.spec and add also other text patches under SOURCES/ We still need also to point to /sources/ for the lookaside cache, so we'll use the value of the hash returned when we successfully uploaded to lookaside cache and we'll write a .`<pkg_name>`.metadata file in the root dir that will look like this in our example:
|
| |
+ Since SIGs have full control over their namespace on gitlab.com, that branching structure
|
| |
+ is not mandatory. It will, however, be necessary if you plan on using the "traditional" lookaside
|
| |
+ structure describe above as this relies on branch names.
|
| |
+
|
| |
|
| |
+ ### Git repository layout
|
| |
+
|
| |
+ There are two possible structure possible for the repository layout:
|
| |
+
|
| |
+ - The exploded SRPM layout
|
| |
+ - The flat dist-git layout
|
| |
+
|
| |
+ #### The exploded SRPM layout
|
| |
+
|
| |
+ This is the "traditional" layout used in CentOS Linux, it is articulated around two folders:
|
| |
```
|
| |
- d6616b89617914a0dd0fd5cfa06b0afc7a4541c4 SOURCES/centpkg-minimal.tar.gz
|
| |
+ ├── SOURCES
|
| |
+ └── SPECS
|
| |
```
|
| |
|
| |
- Now that we have pointer to lookaside cache, and also .spec, we can push back to git and we should be able to proceed with the "build-from-git" on cbs.centos.org. Let's commit first :
|
| |
+ The `SPECS` folder is meant to receive your spec file and all other text files and
|
| |
+ patches would go under `SOURCES`:
|
| |
+
|
| |
+ ```
|
| |
+ ├── SOURCES
|
| |
+ │ ├── <optional_file>
|
| |
+ │ └── <optional_patch>
|
| |
+ └── SPECS
|
| |
+ └── <pkg_name>.spec
|
| |
+ ```
|
| |
+
|
| |
+ Once done, we still need to link these files with the sources uploaded earlier in the
|
| |
+ lookaside cache. See the next section.
|
| |
+
|
| |
+ #### The flat dist-git layout
|
| |
+
|
| |
+ This is the layout used in the dist-git repositories in CentOS Stream and Fedora.
|
| |
+ All files (spec files, text files, patches...) are stored at the top level of
|
| |
+ the repository:
|
| |
+
|
| |
+ ```
|
| |
+ ├── <optional_file>
|
| |
+ ├── <optional_patch>
|
| |
+ └── <pkg_name>.spec
|
| |
+ ```
|
| |
+
|
| |
+ ### Linking to sources in the lookaside cache
|
| |
+
|
| |
+ Here as well, there are two ways you can link a dist-git repository to sources
|
| |
+ uploaded to the lookaside cache:
|
| |
+
|
| |
+ - the "traditional" way using a `.<pkg_name>.metadata` file
|
| |
+ - the CentOS Stream/Fedora way using a `sources` file
|
| |
|
| |
!!! important
|
| |
- Even if your package doesn't contain any source pushed to lookaside cache (like for a package just having some small files in SOURCES/ dir, you *need* to have the `.<pkg_name>.metadata` package present and pushed in git repository
|
| |
+ Even if your package doesn't contain any source pushed to lookaside cache
|
| |
+ (like for a package just having some small files in SOURCES/ dir, you *need* to
|
| |
+ have either a `.<pkg_name>.metadata` or `sources` file present and pushed in
|
| |
+ git repository
|
| |
|
| |
- Here is what the git repository for a pkg should look like :
|
| |
+ #### Using a `.<pkg_name>.metadata` file
|
| |
+
|
| |
+ To use this method, simply create a `.<pkg_name>.metadata` file with the following
|
| |
+ content:
|
| |
+ ```
|
| |
+ <sha1 of the tarball> SOURCES/<filename>
|
| |
+ ```
|
| |
+
|
| |
+ For example:
|
| |
+ ```
|
| |
+ d6616b89617914a0dd0fd5cfa06b0afc7a4541c4 SOURCES/centpkg-minimal.tar.gz
|
| |
+ ```
|
| |
+
|
| |
+ #### Using a `sources` file
|
| |
+
|
| |
+ To use this method, simply create a `sources` file. The easiest way to achieve
|
| |
+ this is:
|
| |
+ ```
|
| |
+ sha512sum --tag <tarball> > sources
|
| |
+ ```
|
| |
+
|
| |
+ Example output:
|
| |
+ ```
|
| |
+ SHA512 (centpkg-minimal-1.1.0.tar.gz) = f7cfbc956199b1a0342f321a0e4cb055d6ac2c784a8faace43cf80772f0de34b58ede1a02a558cd03e25c14938ac6e17b3746b1919c0bbc1f5b2955472577e4f
|
| |
+ ```
|
| |
+
|
| |
+ Now that we have pointer to lookaside cache, and also .spec, we can push back to git and
|
| |
+ we should be able to proceed with the "build-from-git" on cbs.centos.org. Let's commit first.
|
| |
+
|
| |
+ ### Git push
|
| |
+
|
| |
+ Depending on if you picked the "traditional" structure or the CentOS Stream/Fedora
|
| |
+ one, this is how your repository should look like:
|
| |
+
|
| |
+ * Traditional layout
|
| |
|
| |
```
|
| |
.
|
| |
@@ -107,21 +245,28 @@
|
| |
│ └── <optional_patch>
|
| |
└── SPECS
|
| |
└── <pkg_name>.spec
|
| |
+ ```
|
| |
|
| |
+ * CentOS Stream/Fedora layout
|
| |
|
| |
```
|
| |
+ .
|
| |
+ ├── sources
|
| |
+ ├── <optional_file>
|
| |
+ ├── <optional_patch>
|
| |
+ └── <pkg_name>.spec
|
| |
+ ```
|
| |
|
| |
- You can now push to to git, as usual:
|
| |
+ We can now push to to git, as usual:
|
| |
|
| |
```
|
| |
# git add <files> # if needed
|
| |
- git commit -a
|
| |
- git push origin c7-sig-core # to create the c7-sig-core branch on git.centos.org if not existing yet
|
| |
-
|
| |
+ git commit -a
|
| |
+ git push origin <branch> # to create the remote branch if not existing yet
|
| |
```
|
| |
|
| |
- Now that we have our sources pushed to both git.centos.org and lookaside cache, we can now proceed with a build in cbs/koji
|
| |
+ Where `<branch>` would be `c7-sig-core` for our example using `git.centos.org`.
|
| |
|
| |
|
| |
- !!! note
|
| |
- It's worth knowing that the documented process and structure is the default one, aka `rpmbuild style` layout for how your files/sources should be declared in your git repository/branch. But since 2022-02-21, the `flat layout` (as used also on [Fedora](https://src.fedoraproject.org) dist-git server) can also be used on git.centos.org. You still need to push tarballs to lookaside but you can use the flat layout when importing from Fedora/Epel, and it's purely an opt-in choice, and so not the default one that SIGs are using for years now
|
| |
+ Now that we have our sources pushed to both git.centos.org and lookaside cache,
|
| |
+ we can now proceed with a [build in cbs/koji](cbs.md).
|
| |
pingou
commented 2 years ago
Signed-off-by: Pierre-Yves Chibon pingou@pingoured.fr