3How To
3.1How to set up cgit
4.1Git web viewers
4.1.1cgit service
4.2forge nginx service
4.3ACME service
4.4Specialized application deployment services
4.4.1fcgiwrap service
4.4.2gunicorn service

1 Introduction

guix-forge is a Guix service that lets you run a complete software forge in the manner of GitHub, GitLab, etc. Unlike other free software forges such as GitLab, Gitea, etc., guix-forge is not a monolith but is an assemblage of several pieces of server software wired up to function as one. In this sense, it is a meta-service. guix-forge does for software forges what Mail-in-a-Box does for email.

guix-forge integrates the following software components:

In the future, it will also provide:

A choice of different software components may be offered provided it does not complicate the interface too much.

guix-forge is provided on a best effort basis. Its design is unstable, and open to change. We will try our best to not break your system configuration often, but it might happen.

1.1 Philosophy

In order to empower ordinary users, software should not just be free (as in freedom), but also be simple and easy to deploy, especially for small-scale deployments. guix-forge is therefore minimalistic, and does not require running large database servers such as MariaDB and PostgreSQL.

While some state is inevitable, server software should strive to be as stateless as an old analog television set. You switch it on, and it works all the time. There are no pesky software updates, and complex hidden state. guix-forge tries to be as stateless as possible. Almost all of guix-forge's state can be version controlled, and the rest are simple files that can be backed up easily.

Git is already federated and decentralized with email. guix-forge acknowledges this and prefers to support git's email driven workflow with project discussion, bug reports and patches all happening over email.

guix-forge is opinionated and will not expose all features provided by the software components underlying it. Keeping configuration options to a minimum is necessary to help casual users deploy their own forge, and to reduce the likelihood of configuration bugs.

2 Tutorial

In this tutorial, you will learn how to set up guix-forge to host continuous integration for a project. For the purposes of this tutorial, we will set up continuous integration for the guile-json project.

First, we clone the upstream guile-json repository into a local bare clone at /srv/git/guile-json.

$ git clone --bare /srv/git/guile-json
Cloning into bare repository '/srv/git/guile-json'...
remote: Enumerating objects: 1216, done.
remote: Counting objects: 100% (162/162), done.
remote: Compressing objects: 100% (107/107), done.
remote: Total 1216 (delta 96), reused 106 (delta 54), pack-reused 1054
Receiving objects: 100% (1216/1216), 276.10 KiB | 3.89 MiB/s, done.
Resolving deltas: 100% (742/742), done.

Now that we have a git repository to work with, we start writing our Guix system configuration. We begin with a bunch of use-modules statements importing all required modules.

(use-modules (gnu)
             (gnu packages autotools)
             (gnu packages gawk)
             (gnu packages guile)
             (gnu packages pkg-config)
             (gnu packages version-control)
             (gnu services ci)
             (forge forge)
             (forge laminar)
             (forge utils))

Then, we define the G-expression that will be run as a continuous integration job on every commit. This G-expression uses invoke from (guix build utils). Hence, we make it available to the G-expression using with-imported-modules. In addition, it needs a number of packages which we make available using with-packages. And finally, within the body of the G-expression, we have commands cloning the git repository, building the source and running the tests.

The attentive reader may notice what looks like (guix build utils) being referenced twice—once with with-imported-modules and again with use-modules. This is not a mistake. G-expressions are serialized into Guile scripts. with-imported-modules ensures that code for (guix build utils) is available and is in the load path. use-modules actually imports (guix build utils) when the script runs. with-imported-modules is like installing a library in your system, and use-modules is like actually importing that library in a script. Both are necessary.

(define guile-json-tests
  (with-imported-modules '((guix build utils))
    (with-packages (list autoconf automake coreutils
                         gawk git-minimal gnu-make grep
                         guile-3.0 sed pkg-config)
          (use-modules (guix build utils))
          (invoke "git" "clone" "/srv/git/guile-json" ".")
          (invoke "autoreconf" "--verbose" "--install" "--force")
          (invoke "./configure")
          (invoke "make")
          (invoke "make" "check")))))

Now, we configure a <forge-project> record that holds metadata about the project and wires up the G-expression we just defined into a continuous integration job.

(define guile-json-project
   (name "guile-json")
   (user "vetri")
   (repository "/srv/git/guile-json")
   (description "JSON module for Guile")
   (ci-jobs (list (forge-laminar-job
                   (name "guile-json")
                   (run guile-json-tests))))))

The name and description fields are hopefully self-explanatory. The user field specifies the user who will own the git repository at the path specified by repository. That user will therefore be able to push into the repository through ssh or similar. git provides various server-side hooks that trigger on various events. Of these, the post-receive hook triggers when pushed commits are received. guix-forge sets up a post-receive hook script in the repository to trigger a continuous integration run on every git push.

And finally, we put everything together in an operating-system declaration. Notice the forge service configured with guile-json-project and the laminar service configured with a port for the web interface to listen on.

  (host-name "tutorial")
  (timezone "UTC")
  (bootloader (bootloader-configuration
               (bootloader grub-bootloader)))
  (file-systems %base-file-systems)
  (users (cons* (user-account
                 (name "vetri")
                 (group "users")
                 (home-directory "/home/vetri"))
  (packages %base-packages)
  (services (cons* (service forge-service-type
                             (projects (list guile-json-project))))
                   (service laminar-service-type
                             (bind-http "localhost:8080")))

Now that we have a complete operating-system definition, let's use the following command to build a container. After a lot of building, a container script should pop out.

$ guix system container --network --share=/srv/git/guile-json tutorial.scm

The --network flag specifies that the container should share the network namespace of the host. To us, this means that all ports opened by the container will be visible on the host without any port forwarding or complicated configuration. The --share=/srv/git/guile-json option shares the git repository we cloned earlier, with the container.

To start the container, simply run the container script as root.

# /gnu/store/ilg7c2hpkxhwircxpz22qhjsqp3i9har-run-container

Now, you can see the status of laminar and running jobs through its web interface listening on http://localhost:8080. You can list and queue jobs on the command-line like so:

$ laminarc show-jobs
$ laminarc queue guile-json

That's it! You just set up your own continuous integration system and took the first steps to owning your code!

You could easily use the same configuration to configure a Guix system instead of a container. To do so, you will have to take care of defining the bootloader, file systems and other settings as per your needs. The overall configuration used in this tutorial is repeated below for your reference.

  1: (use-modules (gnu)
  2:              (gnu packages autotools)
  3:              (gnu packages gawk)
  4:              (gnu packages guile)
  5:              (gnu packages pkg-config)
  6:              (gnu packages version-control)
  7:              (gnu services ci)
  8:              (forge forge)
  9:              (forge laminar)
 10:              (forge utils))
 12: (define guile-json-tests
 13:   (with-imported-modules '((guix build utils))
 14:     (with-packages (list autoconf automake coreutils
 15:                          gawk git-minimal gnu-make grep
 16:                          guile-3.0 sed pkg-config)
 17:       #~(begin
 18:           (use-modules (guix build utils))
 19:           (invoke "git" "clone" "/srv/git/guile-json" ".")
 20:           (invoke "autoreconf" "--verbose" "--install" "--force")
 21:           (invoke "./configure")
 22:           (invoke "make")
 23:           (invoke "make" "check")))))
 25: (define guile-json-project
 26:   (forge-project
 27:    (name "guile-json")
 28:    (user "vetri")
 29:    (repository "/srv/git/guile-json")
 30:    (description "JSON module for Guile")
 31:    (ci-jobs (list (forge-laminar-job
 32:                    (name "guile-json")
 33:                    (run guile-json-tests))))))
 35: (operating-system
 36:   (host-name "tutorial")
 37:   (timezone "UTC")
 38:   (bootloader (bootloader-configuration
 39:                (bootloader grub-bootloader)))
 40:   (file-systems %base-file-systems)
 41:   (users (cons* (user-account
 42:                  (name "vetri")
 43:                  (group "users")
 44:                  (home-directory "/home/vetri"))
 45:                 %base-user-accounts))
 46:   (packages %base-packages)
 47:   (services (cons* (service forge-service-type
 48:                             (forge-configuration
 49:                              (projects (list guile-json-project))))
 50:                    (service laminar-service-type
 51:                             (laminar-configuration
 52:                              (bind-http "localhost:8080")))
 53:                    %base-services)))

3 How To

3.1 How to set up cgit

guix-forge comes with an end-to-end cgit solution that not only sets up cgit itself but also an nginx server complete with automatically renewed TLS certificates. cgit even runs in its own container for maximal security.

The cgit service uses the forge-nginx service as its web server. The forge-nginx service in turn uses the ACME service to fetch and renew TLS certificates. Here's a minimal working configuration.

(use-modules (gnu)
             ((gnu packages admin) #:select (shepherd))
             (forge acme)
             (forge cgit)
             (forge nginx)
             (forge socket))

  (host-name "forge")
  (timezone "UTC")
  (locale "en_US.utf8")
  (bootloader (bootloader-configuration
               (bootloader grub-bootloader)
               (targets (list "/dev/sdX"))))
  (file-systems %base-file-systems)
   (mixed-text-file "sudoers"
                    "@include " %sudoers-specification
                    ;; Permit the acme user to restart nginx.
                    "\nacme ALL = NOPASSWD: "
                    (file-append shepherd "/bin/herd")
                    " restart nginx\n"))
  (services (cons* (service cgit-service-type
                             (server-name "")
                             (repository-directory "/srv/git")))
                   (service forge-nginx-service-type
                             (http-listen (forge-ip-socket
                                           (ip "")
                                           (port 8080)))
                             (https-listen (forge-ip-socket
                                            (ip "")
                                            (port 4443)))))
                   (service acme-service-type
                             (email "")))

The cgit service configuration specifies the domain to serve cgit on and the /srv/git repository directory containing bare git repositories to publish. The forge nginx service configuration specifies the ports to serve HTTP and HTTPS on. The ACME service configuration specifies the email address to register an ACME account with. The sudoers file declaration is required to allow the acme user to restart the nginx server when a certificate is renewed. The configured machine will start out with self-signed certificates. Run /usr/bin/acme renew the first time to get CA-issued certificates. Thereafter, certificates will auto-renew via a cron job.

When testing your deployment, it might help to start with the Let's Encrypt staging server as shown below. This will give you dummy certificates, but will help you avoid running afoul of Let's Encrypt rate limits. Once you know everything works, delete the ACME state directory (/var/lib/acme by default) and run /usr/bin/acme renew again to get real certificates.

(service acme-service-type
          (email "")
          (acme-url %letsencrypt-staging-url)))

If you are running guix-forge in a Guix system container, do remember to mount the ACME state directory (/var/lib/acme by default) into the container from persistent storage.

4 Services

4.1 Git web viewers

4.1.1 cgit service

cgit is a web frontend to serve git repositories on the web. Our cgit service features

  • clonable URLs via the smart HTTP protocol through git-http-backend
  • syntax highlighting for a wide variety of programming languages using Pygments
  • rendering markdown, org mode, reStructuredText, man page, HTML or plain text README files in the About page
  • hiding full email addresses on cgit web pages
Note that this service is different from the cgit service of the same name in Guix upstream.

Record Type: <cgit-configuration>
cgit (Default: cgit)
cgit package to use
git (Default: git-without-safe-directory-check)
git package to use. git provides the smart HTTP protocol backend.
Domain name to serve cgit on
repository-directory (Default: "/srv/git")
Directory containing git repositories to serve
socket (Default: (forge-unix-socket (path "/var/run/fcgiwrap/cgit/socket")))
Socket that the internal cgit fcgiwrap instance listens on
readme (Default: %cgit-readme)
README file to serve as the About page of the repository. This field is a list of candidate README files looked up in the default branch of the repository. cgit will serve the first file that is found.
snapshots (Default: (list "tar.gz"))
List of strings specifying snapshot formats that cgit generates links for. Valid strings are "tar", "tar.gz", "tar.bz2", "tar.lz", "tar.xz", "tar.xst" and "zip".
about-filter (Default: (program-file "about-filter" (about-filter-gexp this-cgit-configuration)))
Script invoked to format the content of about pages
commit-filter (Default: #f)
Script invoked to format commit messages
email-filter (Default: #f)
Script invoked to format email addresses
source-filter (Default: (file-append (cgit-configuration-cgit this-cgit-configuration) "/lib/cgit/filters/"))
Script invoked to format plaintext blobs in the tree view
mimetype-file (Default: (file-append mailcap "/etc/mime.types"))
File to use for automatic mimetype lookup. This is used by the plain endpoint when serving blob content
repository-sort (Default: 'age)
Order in which repositories are sorted on the index page. Valid values are 'name (sorting by repository name) and 'age (sorting most recently updated repository first).
If #true, full email addresses will be shown. Else, they won't.
extra-options (Default: '())
Association list of additional key-value option pairs to include in the generated cgitrc configuration file

4.2 forge nginx service

The forge nginx service is a wrapper around the nginx web service in Guix upstream. It features

When using this service, you must allow the acme user to restart nginx using sudo. This is so that newly obtained certificates can be deployed to nginx. You may achieve this with the following in the sudoers-file field of your operating-system definition.


   (mixed-text-file "sudoers"
     "@include " %sudoers-specification
     "\nacme ALL = NOPASSWD: " (file-append shepherd "/bin/herd") " restart nginx\n"))

Record Type: <forge-nginx-configuration>
http-listen (Default: (forge-ip-socket (ip "") (port 80)))
Socket to listen on for HTTP requests. Socket may be a <forge-host-socket>, <forge-ip-socket>, or <forge-unix-socket> object.
https-listen (Default: (forge-ip-socket (ip "") (port 443)))
Socket to listen on for HTTPS requests. Socket may be a <forge-host-socket>, <forge-ip-socket>, or <forge-unix-socket> object.
acme-state-directory (Default: "/var/lib/acme")
State directory of the ACME service
acme-challenge-directory (Default: "/var/run/acme/acme-challenge")
Directory to serve on /.well-known/acme-challenge/ in response to ACME HTTP-01 challenges
server-blocks (Default: '())
List of <nginx-server-configuration> objects describing server blocks to add to the nginx configuration

4.3 ACME service

ACME (Automatic Certificate Management Environment) is a protocol popularized by the Let's Encrypt certificate authority for the automatic issue and renewal of TLS certificates. The guix-forge ACME service featuers

The first time the ACME service is set up or each time new certificates are configured, self-signed certificates are created so that processes (such as nginx) that depend on these certificates can start up successfully. You must replace these with certificate authority issued certificates by running /usr/bin/acme renew. /usr/bin/acme renew automatically registers an ACME account unless one already exists and renews all configured certificates. It uses parameters that were configured in the ACME service and does not need any additional command-line arguments.

The ACME service does not use certbot, the official Let's Encrypt client. It instead uses uacme. uacme is smaller, simpler, manages far less state, does no magic, and is better suited to automation. However, the choice of backend tool is an implementation detail. The ACME service is an abstract service that is largely independent of the backend tool that powers it.

By using the ACME service, you agree to the Terms of Service of your ACME server.

Record Type: <acme-configuration>
uacme (Default: uacme)
uacme package to use
Email ID to register ACME account with
acme-url (Default: %letsencrypt-production-url)
URL of the ACME server to use. This field can be set to %letsencrypt-staging-url when testing your deployment.
state-directory (Default: "/var/lib/acme")
State directory in which private keys and certificates are stored
http-01-challenge-directory (Default: "/var/run/acme/acme-challenge")
Directory served by the web server at /.well-known/acme-challenge/ in response to ACME HTTP-01 challenges
http-01-authorization-hook (Default: (program-file "acme-http-01-authorization-hook" (acme-http-01-webroot-authorization-gexp this-acme-configuration)))
Script invoked to complete a HTTP-01 challenge
http-01-cleanup-hook (Default: (program-file "acme-http-01-cleanup-hook" (acme-http-01-webroot-cleanup-gexp this-acme-configuration)))
Script invoked after the completion of a HTTP-01 challenge
key (Default: (acme-ecdsa-key))
<acme-rsa-key> or <acme-ecdsa-key> object describing the ACME account and TLS certificate keys. Changing this field does not affect keys already generated and stored on disk.
certificates (Default: '())
List of <acme-certificate> objects describing certificates to configure

The http-01-authorization-hook and http-01-cleanup-hook scripts are invoked with the following three command-line arguments.

Primary domain name on the certificate
Filename of the resource requested under /.well-known/acme-challenge/ during the HTTP-01 challenge
Authorization string expected to be at the requested resource
Record Type: <acme-certificate>
List of domain names that the certificate is valid for. Each domain name is a string.
Script invoked to deploy a new certificate after successful renewal. This script is invoked without any command-line arguments.
Record Type: <acme-rsa-key>
length (Default: 2048)
Length of the RSA key in number of bits. Must be a multiple of 8 between 2048 and 8192.
Record Type: <acme-ecdsa-key>
length (Default: 256)
Length of the ECDSA key in number of bits. Must be either 256 or 384.

4.4 Specialized application deployment services

4.4.1 fcgiwrap service

fcgiwrap is a specialized web server for CGI applications. It provides a FastCGI interface that web servers such as nginx can talk to. We run separate containerized instances of fcgiwrap for each application.

Note that this service is different from the fcgiwrap service of the same name in Guix upstream.

Record Type: <fcgiwrap-configuration>
package (Default: fcgiwrap)
fcgiwrap package to use
instances (Default: '())
List of <fcgiwrap-instance> objects describing fcgiwrap instances to run
Record Type: <fcgiwrap-instance>
Name of the fcgiwrap instance
socket (Default: (forge-unix-socket (path (string-append "/var/run/fcgiwrap/" (fcgiwrap-instance-name this-fcgiwrap-instance) "/socket"))))
Socket the fcgiwrap instance listens on. Socket may be a <forge-host-socket>, <forge-ip-socket> or <forge-unix-socket> object.
User the fcgiwrap instance should run as
Group the fcgiwrap instance should run as
processes (Default: 1)
Number of fcgiwrap worker processes
environment-variables (Default: '())
List of <environment-variable> objects describing environment variables that should be set in the execution environment
mappings (Default: '())
List of <file-system-mapping> objects describing additional directories that should be shared with the container fcgiwrap is run in

4.4.2 gunicorn service

gunicorn is a specialized web server for Python WSGI applications. We run separate containerized instances of gunicorn for each application.

Record Type: <gunicorn-configuration>
package (Default: gunicorn)
gunicorn package to use
apps (Default: '())
List of <gunicorn-app> objects describing gunicorn apps to run
Record Type: <gunicorn-app>
Name of the app
Package of the app
WSGI app module passed to gunicorn
sockets (Default: (list (forge-unix-socket (path (string-append "/var/run/gunicorn/" (gunicorn-app-name this-gunicorn-app) "/socket")))))
List of <forge-host-socket>, <forge-ip-socket> or <forge-unix-socket> objects describing sockets to listen on
workers (Default: 1)
Number of worker processes
timeout (Default: 30)
Workers silent for more than this many seconds are killed and restarted.
extra-cli-arguments (Default: '())
List of strings to pass as additional command-line arguments to gunicorn
environment-variables (Default: '())
List of <environment-variable> objects describing environment variables that should be set in the execution environment
mappings (Default: '())
List of <file-system-mapping> objects describing additional directories that should be shared with the container gunicorn is run in

5 Reference

Record Type: <environment-variable>
Name of the environment variable
Its value
Record Type: <forge-configuration>
projects (Default: '())
List of <forge-project> objects describing projects managed by guix-forge
Record Type: <forge-project>
Name of the project
Path to a local git repository, or URI to a remote git repository
user (Default: #f)
User who owns the repository if it is local. This field is disregarded if the repository is remote.
description (Default: #f)
Short one-line description of the project. It is used to set the description file in the repository and will appear in the cgit web interface. This field is disregarded if the repository is remote.
website-directory (Default: #f)
Path to the document root of the project website. The ownership of its parent directory is granted to the laminar user. The idea is that the website is built by a Guix derivation as a store item and a symbolic link to that store item is created in the parent directory.
ci-jobs (Default: '())
List of <forge-laminar-job> objects describing CI (continuous integration) jobs to configure
ci-jobs-trigger (Default: 'post-receive-hook for local repositories and 'cron for remote repositories)
One of 'post-receive-hook, 'webhook, or 'cron representing the type of trigger for continuous integration jobs.
If 'post-receive-hook is specified, the post-receive hook of the repository is configured to trigger CI jobs. This is possible only for local repositories. Note that any pre-existing post-receive hook is overwritten.
If 'webhook is specified, a webhook server is configured to trigger CI jobs when a request is received on http://hostname:port/hooks/<name> .
If 'cron is specified, a cron job triggers the CI jobs once a day.
parallel-ci-job-runs (Default: 1)
Number of CI job runs of this project to run simultaneously
repository-branch (Default: "main")
Main branch of the repository. This field is currently unused unused, and may be deprecated in the future.
Record Type: <forge-laminar-job>
Name of the job
G-expression to be run
after (Default: #f)
G-expression to be run after the main job script
trigger? (Default: #t)
If #t, this job is run on every commit. Else, it must be manually set up to run some other way.
contexts (Default: '())
List of names of contexts (strings) associated with this job
Record Type: <forge-host-socket>
hostname (Default: "localhost")
Name of the host
Port number
Record Type: <forge-ip-socket>
ip (Default: "")
IP address, either IPv4 or IPv6, as a string. The loopback address is "" and "::1" for IPv4 and IPv6 respectively. The any address is "" and "::" for IPv4 and IPv6 respectively.
Port number to listen on.
Record Type: <forge-unix-socket>
Path to socket file.
Record Type: <webhook-configuration>
package (Default: webhook)
webhook package to use
socket (Default: (forge-ip-socket (ip "") (port 9000)))
Socket, a <forge-ip-socket> object, to listen on.
log-directory (Default: "/var/log/webhook")
Directory to write log files to
hooks (Default: '())
List of <webhook-hook> objects describing hooks to configure
Record Type: <webhook-hook>
Identifier of the webhook. This hook is triggered at http://host:port/hooks/<id>.
G-expression to run when the webhook is triggered
(klaus-gunicorn-app repository-directory #:key (klaus python-klaus) (sockets (list (forge-unix-socket (path "/var/run/gunicorn/klaus/socket")))) site-name)

Return a <gunicorn-app> object to deploy klaus.

repository-directory is the path to the directory containing git repositories to serve.

klaus is the klaus package to use.

sockets is a list of <forge-ip-socket> or <forge-unix-socket> objects describing sockets to listen on.

site-name is the name of the klaus site to be displayed in the banner.

(made with skribilo)