1 Site Configuration Modules
A build farm is normally run via the installers, site , or snapshot-site target of the Racket repository’s top-level makefile. The latter two targets chain to the installers target, which expects a CONFIG=... argument to specify a configuration module file (or uses "build/site.rkt" as the default).
A site configuration module starts #lang distro-build/config and uses keywords to specify various options for the configuration. This format is described is detail in Site Configuration Language. For now, it’s enough to know that there are various options, each of which is associated with a keyword.
The machine where make installers is run is the server machine. The server machine first prepares packages for installation on client machines. The site configuration’s top-level entry is consulted for a #:pkgs and/or #:doc-search option, which overrides any PKGS and/or DOC_SEARCH configuration from the makefile. A top-level #:test-pkgs entry in the configuration is added to #:pkgs to determine the packages that are prepared by the server.
When building installers that include the 'cs variant, make installer SETUP_MACHINE_FLAGS=-M is recommended, because it prepares the packages in a machine-independent format that can be quickly recompiled on clients.
The site configuration file otherwise describes and configures client machines hierarchically, where configuration options propagate down the hierarchy when they are not overridden more locally.
Each client is normally built by running commands via ssh, where the client’s host configured with #:host (with and optional #:port and/or #:user) indicate the ssh target. Each client machine must be set up with a public-key authentication, because a direct ssh is expected to work without a password prompt. An exception is when the host is "localhost" and user is #f, in which case a shell is used directly instead of ssh. When ssh is used, -R is also used to create a tunnel back to the server, and the client by default uses that tunnel for all communication, so the server by default accepts only connections via "localhost".
On the client machine, all work is performed at a specified directory as specified by #:dir. The directory defaults to "build/plt" (Unix or Mac OS) or "build\plt" (Windows), except when the host is "localhost" and the client is #f, in which case the current directory (i.e., the server’s directory) is used.
Normally, the client directory is a Git clone:
If the directory exists already on a client machine (and the machine is not configured for “clean” mode), then if the directory contains a ".git" subdirectory, it is assumed to be a git clone and updated with git pull. The git pull operation can be disabled by specifying #:pull? as #f, and it defaults to #f in the case that #:dir is not specified, the host is "localhost", and the user is #f.
If the directory does not exist, a Git repository is cloned. The repository can be specified with #:repo. By default, the server is used as the source Git repository (so that the server and client are in sync), which means that the server’s directory must be a Git clone.
Note that neither ssh nor git turn out to be needed when the host is "localhost", the user is #f, and the directory is not specified (which corresponds to the defaults in all cases).
If a build fails for a machine, building continues on other machines. Success for a given machine means that its installer ends up in "build/installers" (and failure for a machine means no installer) as recorded in the "table.rktd" file.
To use the site makefile target, the configuration file must at least provide a #:dist-base-url value, which is a URL at which the site will be made available. To use the snapshot-site makefile target, then #:site-dest will need to be specified, normally as a path that ends with the value produced by (current-stamp).
Hint: When developing a configuration file, use an empty set of packages to a configuration that works as quickly as possible. Then, change the list of packages to the ones that you actually want in the installers.