| CentOS Infra lookaside upload script |
| |
| This upload script is a fork of Fedora's. The original version of the script was taken from https://git.fedorahosted.org/cgit/fedora-infrastructure.git/tree/scripts/upload.cgi (although I think that version might be a bit on the old side, but it does the job). |
| |
| The script has been modified somewhat (ok, quite a lot) to fit CentOS' requirements. |
| |
| |
| Requirements: |
| |
| Basic requirements from kbsingh et al: |
| |
| - Users must be authenticated to be able to upload |
| - Authentication must be done using client SSL certificates from a private CA |
| - Files must be uploaded in the centos <package>/<branch>/<sha1sum> scheme, rather than the fedora scheme |
| - The upload process must be able to be driven from centpkg (so ideally similar script/parameters to fedora) |
| - Upload permissions must be controlled from the gitblit config |
| |
| Some more requirements of mine: |
| |
| - The upload system must check for revocation of client certs |
| - Client cert revocation should be done quickly (immediately if possible) |
| |
| |
| Assumptions: |
| |
| - Every user who can upload has a unique username. |
| - Every user has an account in git.centos.org's gitblit. |
| - Every user has a client SSL certificate, issued by the CBS CA. |
| - The client SSL certificate has a CN of their username as part of the certificate subject. |
| - A users git username matches their koji username. |
| |
| |
| Access Control: |
| |
| The script requires a user to authenticate with their client SSL cert. All users can run the script in check mode. If a user tries to upload, the script checks the gitblit config to ensure that the user has permissions to the package. |
| |
| (This access control can be disabled using the script config file, if desired) |
| |
| |
| Apache SSL Configuration: |
| |
| The following config is sufficient for the SSL client auth: |
| |
| # This needs to point to the CA cert that issued the client certs |
| SSLCACertificateFile /etc/pki/tls/certs/cbs-ca.crt |
| |
| # Don't verify client certs on the server by default |
| SSLVerifyClient none |
| |
| # Tell Apache the upload script is CGI |
| ScriptAlias /lookaside/upload.cgi /var/www/cgi-bin/upload.cgi |
| |
| # Enable client cert verification for the upload script URL |
| <Location /lookaside/upload.cgi> |
| SSLVerifyClient require |
| SSLVerifyDepth 1 |
| </Location> |
| |
| Ideally we want to check for client cert revocation. We can check a local CRL file with the following config. Note that if the CRL is updated, Apache needs to be restarted to pick up the changes. |
| |
| # check revocation of client certs against the CRL |
| SSLCARevocationCheck leaf |
| # specify the CRL file location (must be in PEM format) |
| SSLCARevocationFile /etc/pki/tls/certs/ipa.crl |
| |
| If at some point we switch to a CA with an OCSP responder, we can use the following config (Apache 2.4 required) to do a live OCSP client cert revocation check on upload: |
| |
| # turn on OCSP checking of client certs |
| SSLOCSPEnable on |
| # set the URL for the OCSP responder |
| SSLOCSPDefaultResponder http://my.ca.server/ca/ocsp |
| # ignore the OCSP URL in client certs and use the one we configured |
| SSLOCSPOverrideResponder on |
| |
| |
| Calling The Script: |
| |
| Obviously, you need a valid client cert. You then need to call the script with the right parameters. Here's the parameters: |
| |
| - name |
| - branch |
| - sha1sum |
| - file |
| |
| name, branch, and sha1sum are mandatory. name is the package name. branch is the name of the branch. sha1sum is a (lowercase) hex SHA1 checksum for the file. If only these three parameters are provided, the script checks if there is a matching file uploaded. If the file exists, the script returns the string "Available". If the file does not exist, the script returns the string "Missing". |
| |
| If the parameter file is passed, this parameter must be the contents of the file. The uploaded file will be written to a temporary file, and the checksum of the file compared to the value of the sha1sum parameter. If the sums match, the file is moved into position. |
| |
| The script can be called using curl commands similar to the following. The file mycert.pem contains the client's cert (and private key). |
| |
| To upload a file (389-ds-base-1.3.1.6.tar.bz2 in the current directory): |
| |
| curl --cert ./mycert.pem https://git.centos.org/lookaside/upload.cgi --form "name=389-ds-base" --form "branch=c7" --form "sha1sum=ce4e6293a996e1045bc8f75533418f3172b391ee" --form "file=@389-ds-base-1.3.1.6.tar.bz2" |
| File 389-ds-base-1.3.1.6.tar.bz2 size 3070988 SHA1 ce4e6293a996e1045bc8f75533418f3172b391ee stored OK |
| |
| To check if a file exists: |
| |
| curl --cert ./mycert.pem https://git.centos.org/lookaside/upload.cgi --form "name=389-ds-base" --form "branch=c7" --form "sha1sum=ce4e6293a996e1045bc8f75533418f3172b391ee" |
| Available |
| |
| (on a normal end client, this would be handled by centpkg) |
| |
| (centos' curl defaults to looking in a NSS db for a client cert; force the --cert arg to be a path - even a relative one like ./mycert.pem - to stop this behaviour) |
| |
| |
| Email Notification: |
| |
| The script sends an email to a configured email address when a file is uploaded. The script tries to send the mail via a configured mail relay. By default, SELinux will block the script from connecting to port 25 on the configured mail server. Set the httpd_can_network_connect boolean on to allow it. |
| |
| |
| Script Config File: |
| |
| The upload script config file is /etc/lookaside.cfg. There are config options for all the interesting values. The script doesn't check the config syntax is correct, so missing / malformed values are likely to case python tracebacks. |