ykarel / centos / cbs-tools

Forked from centos/cbs-tools 4 years ago
Clone

Blame lookaside/README

9ed232
CentOS Infra lookaside upload script
9ed232
9ed232
This upload script is a fork of Fedora's.  The original version of the script was taken from https://git.fedorahosted.org/cgit/fedora-infrastructure.git/tree/scripts/upload.cgi (although I think that version might be a bit on the old side, but it does the job).
9ed232
9ed232
The script has been modified somewhat (ok, quite a lot) to fit CentOS' requirements.
9ed232
9ed232
9ed232
Requirements:
9ed232
9ed232
Basic requirements from kbsingh et al:
9ed232
9ed232
- Users must be authenticated to be able to upload
9ed232
- Authentication must be done using client SSL certificates from a private CA
9ed232
- Files must be uploaded in the centos <package>/<branch>/<sha1sum> scheme, rather than the fedora scheme
9ed232
- The upload process must be able to be driven from centpkg (so ideally similar script/parameters to fedora)
9ed232
- Upload permissions must be controlled from the gitblit config
9ed232
9ed232
Some more requirements of mine:
9ed232
9ed232
- The upload system must check for revocation of client certs
9ed232
- Client cert revocation should be done quickly (immediately if possible)
9ed232
9ed232
9ed232
Assumptions:
9ed232
9ed232
- Every user who can upload has a unique username.
9ed232
- Every user has an account in git.centos.org's gitblit.
9ed232
- Every user has a client SSL certificate, issued by the CBS CA.
9ed232
- The client SSL certificate has a CN of their username as part of the certificate subject.
9ed232
- A users git username matches their koji username.
9ed232
9ed232
9ed232
Access Control:
9ed232
9ed232
The script requires a user to authenticate with their client SSL cert.  All users can run the script in check mode.  If a user tries to upload, the script checks the gitblit config to ensure that the user has permissions to the package.
9ed232
9ed232
(This access control can be disabled using the script config file, if desired)
9ed232
9ed232
9ed232
Apache SSL Configuration:
9ed232
9ed232
The following config is sufficient for the SSL client auth:
9ed232
9ed232
  # This needs to point to the CA cert that issued the client certs
9ed232
  SSLCACertificateFile /etc/pki/tls/certs/cbs-ca.crt
9ed232
9ed232
  # Don't verify client certs on the server by default
9ed232
  SSLVerifyClient none
9ed232
9ed232
  # Tell Apache the upload script is CGI
9ed232
  ScriptAlias /lookaside/upload.cgi /var/www/cgi-bin/upload.cgi
9ed232
9ed232
  # Enable client cert verification for the upload script URL
9ed232
  <Location /lookaside/upload.cgi>
9ed232
          SSLVerifyClient require
9ed232
          SSLVerifyDepth 1
9ed232
  </Location>
9ed232
9ed232
Ideally we want to check for client cert revocation.  We can check a local CRL file with the following config.  Note that if the CRL is updated, Apache needs to be restarted to pick up the changes.
9ed232
9ed232
  # check revocation of client certs against the CRL
9ed232
  SSLCARevocationCheck leaf
9ed232
  # specify the CRL file location (must be in PEM format)
9ed232
  SSLCARevocationFile /etc/pki/tls/certs/ipa.crl
9ed232
9ed232
If at some point we switch to a CA with an OCSP responder, we can use the following config (Apache 2.4 required) to do a live OCSP client cert revocation check on upload:
9ed232
9ed232
  # turn on OCSP checking of client certs
9ed232
  SSLOCSPEnable on
9ed232
  # set the URL for the OCSP responder
9ed232
  SSLOCSPDefaultResponder http://my.ca.server/ca/ocsp
9ed232
  # ignore the OCSP URL in client certs and use the one we configured
9ed232
  SSLOCSPOverrideResponder on
9ed232
9ed232
9ed232
Calling The Script:
9ed232
9ed232
Obviously, you need a valid client cert.  You then need to call the script with the right parameters.  Here's the parameters:
9ed232
9ed232
- name
9ed232
- branch
9ed232
- sha1sum
9ed232
- file
9ed232
9ed232
name, branch, and sha1sum are mandatory.  name is the package name.  branch is the name of the branch.  sha1sum is a (lowercase) hex SHA1 checksum for the file.  If only these three parameters are provided, the script checks if there is a matching file uploaded.  If the file exists, the script returns the string "Available".  If the file does not exist, the script returns the string "Missing".
9ed232
9ed232
If the parameter file is passed, this parameter must be the contents of the file.  The uploaded file will be written to a temporary file, and the checksum of the file compared to the value of the sha1sum parameter.  If the sums match, the file is moved into position.
9ed232
9ed232
The script can be called using curl commands similar to the following.  The file mycert.pem contains the client's cert (and private key).
9ed232
9ed232
To upload a file (389-ds-base-1.3.1.6.tar.bz2 in the current directory):
9ed232
9ed232
curl --cert ./mycert.pem https://git.centos.org/lookaside/upload.cgi --form "name=389-ds-base" --form "branch=c7" --form "sha1sum=ce4e6293a996e1045bc8f75533418f3172b391ee" --form "file=@389-ds-base-1.3.1.6.tar.bz2"
9ed232
File 389-ds-base-1.3.1.6.tar.bz2 size 3070988 SHA1 ce4e6293a996e1045bc8f75533418f3172b391ee stored OK
9ed232
9ed232
To check if a file exists:
9ed232
9ed232
curl --cert ./mycert.pem https://git.centos.org/lookaside/upload.cgi --form "name=389-ds-base" --form "branch=c7" --form "sha1sum=ce4e6293a996e1045bc8f75533418f3172b391ee"
9ed232
Available
9ed232
9ed232
(on a normal end client, this would be handled by centpkg)
9ed232
9ed232
(centos' curl defaults to looking in a NSS db for a client cert; force the --cert arg to be a path - even a relative one like ./mycert.pem - to stop this behaviour)
9ed232
9ed232
9ed232
Email Notification:
9ed232
9ed232
The script sends an email to a configured email address when a file is uploaded.  The script tries to send the mail via a configured mail relay.  By default, SELinux will block the script from connecting to port 25 on the configured mail server.  Set the httpd_can_network_connect boolean on to allow it.
9ed232
9ed232
9ed232
Script Config File:
9ed232
9ed232
The upload script config file is /etc/lookaside.cfg.  There are config options for all the interesting values.  The script doesn't check the config syntax is correct, so missing / malformed values are likely to case python tracebacks.