guix-forge
1Introduction
1.1Philosophy
2Tutorial
3Reference

1 Introduction

guix-forge is a Guix service that lets you run a complete software forge in the manner of GitHub, GitLab, etc. Unlike other free software forges such as GitLab, Gitea, etc., guix-forge is not a monolith but is an assemblage of several pieces of server software wired up to function as one. In this sense, it is a meta-service. guix-forge does for software forges what Mail-in-a-Box does for email.

guix-forge integrates the following software components:

In the future, it will also provide:

A choice of different software components may be offered provided it does not complicate the interface too much.

guix-forge is provided on a best effort basis. Its design is unstable, and open to change. We will try our best to not break your system configuration often, but it might happen.

1.1 Philosophy

In order to empower ordinary users, software should not just be free (as in freedom), but also be simple and easy to deploy, especially for small-scale deployments. guix-forge is therefore minimalistic, and does not require running large database servers such as MariaDB and PostgreSQL.

While some state is inevitable, server software should strive to be as stateless as an old analog television set. You switch it on, and it works all the time. There are no pesky software updates, and complex hidden state. guix-forge tries to be as stateless as possible. Almost all of guix-forge's state can be version controlled, and the rest are simple files that can be backed up easily.

Git is already federated and decentralized with email. guix-forge acknowledges this and prefers to support git's email driven workflow with project discussion, bug reports and patches all happening over email.

guix-forge is opinionated and will not expose all features provided by the software components underlying it. Keeping configuration options to a minimum is necessary to help casual users deploy their own forge, and to reduce the likelihood of configuration bugs.



2 Tutorial

In this tutorial, you will learn how to set up guix-forge to host continuous integration for a project. For the purposes of this tutorial, we will set up continuous integration for the guile-json project.

First, we clone the upstream guile-json repository into a local bare clone at /srv/git/guile-json.

$ git clone --bare https://github.com/aconchillo/guile-json /srv/git/guile-json
Cloning into bare repository '/srv/git/guile-json'...
remote: Enumerating objects: 1216, done.
remote: Counting objects: 100% (162/162), done.
remote: Compressing objects: 100% (107/107), done.
remote: Total 1216 (delta 96), reused 106 (delta 54), pack-reused 1054
Receiving objects: 100% (1216/1216), 276.10 KiB | 3.89 MiB/s, done.
Resolving deltas: 100% (742/742), done.

Now that we have a git repository to work with, we start writing our Guix system configuration. We begin with a bunch of use-modules statements importing all required modules.

(use-modules (gnu)
             (gnu packages autotools)
             (gnu packages gawk)
             (gnu packages guile)
             (gnu packages pkg-config)
             (gnu packages version-control)
             (gnu services ci)
             (forge forge)
             (forge laminar)
             (forge utils))

Then, we define the G-expression that will be run as a continuous integration job on every commit. This G-expression uses invoke from (guix build utils). Hence, we make it available to the G-expression using with-imported-modules. In addition, it needs a number of packages which we make available using with-packages. And finally, within the body of the G-expression, we have commands cloning the git repository, building the source and running the tests.

The attentive reader may notice what looks like (guix build utils) being referenced twice—once with with-imported-modules and again with use-modules. This is not a mistake. G-expressions are serialized into Guile scripts. with-imported-modules ensures that code for (guix build utils) is available and is in the load path. use-modules actually imports (guix build utils) when the script runs.

(define guile-json-tests
  (with-imported-modules '((guix build utils))
    (with-packages (list autoconf automake coreutils
                         gawk git-minimal gnu-make grep
                         guile-3.0 sed pkg-config)
      #~(begin
          (use-modules (guix build utils))
          (invoke "git" "clone" "/srv/git/guile-json" ".")
          (invoke "autoreconf" "--verbose" "--install" "--force")
          (invoke "./configure")
          (invoke "make")
          (invoke "make" "check")))))

Now, we configure a <forge-project> record that holds metadata about the project and wires up the G-expression we just defined into a continuous integration job.

(define guile-json-project
  (forge-project
   (name "guile-json")
   (user "vetri")
   (repository "/srv/git/guile-json")
   (description "JSON module for Guile")
   (ci-jobs (list (forge-laminar-job
                   (name "guile-json")
                   (run guile-json-tests))))))

The name and description fields are hopefully self-explanatory. The user field specifies the user who will own the git repository at the path specified by repository. That user will therefore be able to push into the repository through ssh or similar. git provides various server-side hooks that trigger on various events. Of these, the post-receive hook triggers when pushed commits are received. guix-forge sets up a post-receive hook script in the repository to trigger a continuous integration run on every git push.

And finally, we put everything together in an operating-system declaration. Notice the forge service configured with guile-json-project and the laminar service configured with a port for the web interface to listen on.

(operating-system
  (host-name "tutorial")
  (timezone "UTC")
  (bootloader (bootloader-configuration
               (bootloader grub-bootloader)))
  (file-systems %base-file-systems)
  (users (cons* (user-account
                 (name "vetri")
                 (group "users")
                 (home-directory "/home/vetri"))
                %base-user-accounts))
  (packages %base-packages)
  (services (cons* (service forge-service-type
                            (forge-configuration
                             (projects (list guile-json-project))))
                   (service laminar-service-type
                            (laminar-configuration
                             (bind-http "localhost:8080")))
                   %base-services)))

Now that we have a complete operating-system definition, let's use the following command to build a container. After a lot of building, a container script should pop out.

$ guix system container --network --share=/srv/git/guile-json tutorial.scm
/gnu/store/ilg7c2hpkxhwircxpz22qhjsqp3i9har-run-container

The --network flag specifies that the container should share the network namespace of the host. To us, this means that all ports opened by the container will be visible on the host without any port forwarding or complicated configuration. The --share=/srv/git/guile-json option shares the git repository we cloned earlier, with the container.

To start the container, simply run the container script as root.

# /gnu/store/ilg7c2hpkxhwircxpz22qhjsqp3i9har-run-container

Now, you can see the status of laminar and running jobs through its web interface listening on http://localhost:8080. You can list and queue jobs on the command-line like so:

$ laminarc show-jobs
guile-json
$ laminarc queue guile-json
guile-json:1

That's it! You just set up your own continuous integration system and took the first steps to owning your code!

You could easily use the same configuration to configure a Guix system instead of a container. To do so, you will have to take care of defining the bootloader, file systems and other settings as per your needs. The overall configuration used in this tutorial is repeated below for your reference.

  1: (use-modules (gnu)
  2:              (gnu packages autotools)
  3:              (gnu packages gawk)
  4:              (gnu packages guile)
  5:              (gnu packages pkg-config)
  6:              (gnu packages version-control)
  7:              (gnu services ci)
  8:              (forge forge)
  9:              (forge laminar)
 10:              (forge utils))
 11: 
 12: (define guile-json-tests
 13:   (with-imported-modules '((guix build utils))
 14:     (with-packages (list autoconf automake coreutils
 15:                          gawk git-minimal gnu-make grep
 16:                          guile-3.0 sed pkg-config)
 17:       #~(begin
 18:           (use-modules (guix build utils))
 19:           (invoke "git" "clone" "/srv/git/guile-json" ".")
 20:           (invoke "autoreconf" "--verbose" "--install" "--force")
 21:           (invoke "./configure")
 22:           (invoke "make")
 23:           (invoke "make" "check")))))
 24: 
 25: (define guile-json-project
 26:   (forge-project
 27:    (name "guile-json")
 28:    (user "vetri")
 29:    (repository "/srv/git/guile-json")
 30:    (description "JSON module for Guile")
 31:    (ci-jobs (list (forge-laminar-job
 32:                    (name "guile-json")
 33:                    (run guile-json-tests))))))
 34: 
 35: (operating-system
 36:   (host-name "tutorial")
 37:   (timezone "UTC")
 38:   (bootloader (bootloader-configuration
 39:                (bootloader grub-bootloader)))
 40:   (file-systems %base-file-systems)
 41:   (users (cons* (user-account
 42:                  (name "vetri")
 43:                  (group "users")
 44:                  (home-directory "/home/vetri"))
 45:                 %base-user-accounts))
 46:   (packages %base-packages)
 47:   (services (cons* (service forge-service-type
 48:                             (forge-configuration
 49:                              (projects (list guile-json-project))))
 50:                    (service laminar-service-type
 51:                             (laminar-configuration
 52:                              (bind-http "localhost:8080")))
 53:                    %base-services)))

3 Reference

Record Type: <forge-configuration>
projects (Default: '())
List of [?mark <forge-project>: doc/forge.skb:187:20:] objects describing projects managed by guix-forge
Record Type: <forge-project>
name
Name of the project
repository
Path to a local git repository, or URI to a remote git repository
user (Default: #f)
User who owns the repository if it is local. This field is disregarded if the repository is remote.
description (Default: #f)
Short one-line description of the project. It is used to set the description file in the repository and will appear in the cgit web interface.
website-directory (Default: #f)
Path to the document root of the project website. The ownership of its parent directory is granted to the laminar user. The idea is that the website is built by a Guix derivation as a store item and a symbolic link to that store item is created in the parent directory.
ci-jobs (Default: '())
List of [?mark <forge-laminar-job>: doc/forge.skb:210:20:] objects describing CI (continuous integration) jobs to configure
ci-jobs-trigger (Default: 'post-receive-hook for local repositories and 'cron for remote repositories)
One of 'post-receive-hook, 'webhook, or 'cron representing the type of trigger for continuous integration jobs.
'post-receive-hook
If 'post-receive-hook is specified, the post-receive hook of the repository is configured to trigger CI jobs. This is possible only for local repositories. Note that any pre-existing post-receive hook is overwritten.
'webhook
If 'webhook is specified, a webhook server is configured to trigger CI jobs when a request is received on http://hostname:port/hooks/<name> .
'cron
If 'cron is specified, a cron job triggers the CI jobs once a day.
repository-branch (Default: "main")
Main branch of the repository. This field is currently unused unused, and may be deprecated in the future.
Record Type: <forge-laminar-job>
name
Name of the job
run
G-expression to be run
after (Default: #f)
G-expression to be run after the main job script
trigger? (Default: #t)
If #t, this job is run on every commit. Else, it must be manually set up to run some other way.
Record Type: <forge-ip-socket>
ip (Default: "127.0.0.1")
IP address, either IPv4 or IPv6, as a string. The loopback address is "127.0.0.1" and "::1" for IPv4 and IPv6 respectively. The any address is "0.0.0.0" and "::" for IPv4 and IPv6 respectively.
port
Port number to listen on.
Record Type: <forge-unix-socket>
path
Path to socket file.
Record Type: <webhook-configuration>
package (Default: webhook)
webhook package to use
socket (Default: (forge-ip-socket (ip "127.0.0.1") (port 9000)))
Socket, a [?mark <forge-ip-socket>: doc/forge.skb:258:22:] object, to listen on.
log-directory (Default: "/var/log/webhook")
Directory to write log files to
hooks (Default: '())
List of [?mark <webhook-hook>: doc/forge.skb:263:20:] objects describing hooks to configure
Record Type: <webhook-hook>
id
Identifier of the webhook. This hook is triggered at http://host:port/hooks/<id>.
run
G-expression to run when the webhook is triggered

(made with skribilo)