Schematics — An Introduction

Go to the profile of Hans
Hans

Schematics is a workflow tool for the modern web; it can apply transforms to your project, such as create a new component, or updating your code to fix breaking changes in a dependency. Or maybe you want to add a new configuration option or framework to an existing project.

Goals

The mission of the Angular CLI is to improve your development productivity. We came to a point where we needed a more powerful and generic facility to support the CLI scaffolding, and we settled on 4 primary goals:

  1. Ease of use and development. It had to have simple concepts for developers that were intuitive and feel natural. Also, the code developed needed to be synchronous to make it easier to develop.
  2. Extensibility and Reusability. By keeping reusability in mind, we were able to design a simple but powerful pipeable interface. Schematics can be added as the input, or the output of other Schematics. For example, an application can be created using components and modules schematics.
  3. Atomicity. We had many errors in the CLI blueprints that were the direct result of side effects by our blueprints. When we created Schematics, we decided to remove side effects entirely from our code. All the changes are recorded in memory, and only applied once they’re confirmed to be valid. For example, creating a file that already exist is an error, and would discard all the other changes applied so far.
  4. Asynchronicity. Many workflow are asynchronous in nature (e.g. accessing web servers), and so Schematics had to support those use cases. This seems in contradiction with the first goal of making the debugging process synchronous, but we came to a design that made everything work together. The input of a Schematics is synchronous, but the output can be asynchronous, and the library will wait for everything to be done before starting the next step. This way developers can reuse without even knowing that a Schematics is asynchronous.

All the Schematics design decisions turned out around these 4 major goals. Schematics is the combined efforts to build a better workflow tool.

Understanding Schematics

In a schematic, you don’t actually perform any direct actions on the filesystem. Rather, you describe what transformation you would like to apply to a Tree. This allows us to support features like dry runs (or patch runs) without adding special support from the schematics themselves. It also makes schematics hermetic which ensures reusability and safety.

The Tree is a data structure that contains a base (a set of files that already exists) and a staging area (a list of changes to be applied to the base). When making modifications, you don’t actually change the base, but add those modifications to the staging area. This is really powerful but can be tricky and will be further explored in a separate medium post.

The Tree that a schematic will receive can be anything. The Angular CLI will use a Tree representing the project on the drive to the first schematic it calls, but composed schematics could receive any Trees. The good news is that it doesn’t matter; the Tree represents your starting point.

Creating your first Schematics

First, make sure you have Node 6.9 or above installed. Next, install the Schematics command line tool globally:

npm install -g @angular-devkit/schematics-cli

This will install a schematics executable, which you can use to create a blank Schematics project:

schematics blank --name=my-component

Et voilà. The blank schematics either create a new project, or add a blank schematic to an existing project (it can be used for both). You can then cd into your new project, install your npm dependencies, and open your new collection using your favorite editor of choice:

cd my-component
npm install
code . # or atom, webstorm, vi, ...

Collections

Schematics Collections are sets of named schematics, that are published and installed by users. For example, the Angular team publishes and maintains the official @schematics/angular collection, which contains schematics like component, module and application.

In our case, our collection will only include the my-component schematic. You can look at the src/collection.json file, which contains the description of our collection:

{
  "$schema": "../node_modules/@angular-devkit/schematics/collection-schema.json",
  "schematics": {
    "my-component": {
      "description": "A blank schematic.",
      "factory": "./my-component/index#myComponent"
    }
  }
}

The $schema key points to the JSON Schema defining this format. It is used by IDEs to do auto completion, tools for validation, and is entirely optional.

The important key is "schematics", which describes the schematics included in this collection. In our example, we describe one schematic: my-component, it has a simple description and a factory field. The factory field uses a string reference to point to a JavaScript function; in our case the exported function myComponent in the file my-component/index.js. It represents the RuleFactory.

Rules, Trees and Files

A Rule is a function that takes a Tree and returns another Tree. Rules are the core of Schematics; they are the ones making changes to your projects, calling external tools, and implementing logic. RuleFactory, as the name implies, are functions that create a Rule.

Here’s the blank RuleFactory created so far:

This factory takes an options argument and returns a Rule that takes a Tree and returns it unchanged.

The options argument is an object that can be seen as the input of the factory. From the CLI, it is the command line arguments the user passed. From another schematic, it’s the options that were passed in by that schematic. A GUI tool could construct an options object from user or project inputs, for example.

In any case, this is always an object and can be typed as any. It can also be validated with a JSON Schema to make sure that the inputs have appropriate default and types. JSON Schemas will be looked at more closely in a later post.

In the meantime, let’s do something more interesting with our rule:

With this new line, we’re creating a file in the root of the schematic’s Tree, named either after the name option (or 'hello' by default), containing the string world. This might seem trivial for now, but there’s a lot going on here under the hood.

A Tree contains the files that your schematics should be applied on. It has a list of files, and contains metadata associated with the changes you want to apply. In our case, the only change being made is to create a new file. Trees are more complex than just being a filesystem equivalent, and will be explored more deeply in a later post, but for the moment you can see them as a collection of files and changes.

By default, the Angular CLI will pass the root of your Angular project as the Tree, but any schematic can pass in a different Tree to other schematics. You can create empty trees, scope a Tree to a directory of a parent Tree, merge two trees, or branch them (making a copy of it).

There are four methods that directly create a change in a Tree; create, delete, rename, and overwrite.

Running Your New Schematics

To run our example, you first need to build it, then use the schematics command line tool with the path to our schematic project’s directory as the collection. From the root of our project:

npm run build
# ... wait for build to finish
schematics .:my-component --name=test
# ... see that a file is created in the root.

Before looking further into what happens here, a word of warning; don’t worry, this time you did not actually create a file on your filesystem. This is because the schematics tool is in debug mode when using a path as the collection it should use. When in debugging (which can also be used with --debug=true), the default is also to run in dry run mode, which prevents the tool from actually creating files.

This can be changed using the argument --dry-run=false. But beware, this means that the changes will really happen on the filesystem. If you delete or overwrite a file, you might lose content you don’t want to. We suggest to be in a separate temporary directory when debugging schematics, and to disable dry runs only when necessary.

You can also start npm run build -- -w in a separate terminal so it automatically rebuild your schematic project when a file changes.

Debugging

In order to debug your schematics, you need to run with node in debugging mode:

node --inspect-brk $(which schematics) .:myComponent --name=test

Another advantage of running in debug mode is that the schematics command line tool puts a break point directly before running your own schematic.

Calling Another Schematic

One of the great advantage of Schematics is in how easy they are to compose together. In our example, we’ll call the component schematic from the Angular collection to add a component to your application, then add a header to every TypeScript files added by the Schematic.

Don’t forget to add @schematics/angular to your dependencies in your package.json!

A few things to note here. First, we are calling and returning chain() directly. chain() is a RuleFactory provided by the Schematics library that chain multiple rules together, waiting in between for the Rule to finish. There are other rule factories like this provided by the Schematic library, and we’ll go over them at a later moment.

Second, we’re using another RuleFactory called externalSchematic (it also has a sister factory called schematic). Schematics being rules, you might be tempted to simply import a schematic’s rule factory and create the rule yourself, then call it directly (or pass it to chain directly). Do not call other Schematics as Rules directly. There is more logic to the externalSchematic (and schematic) rule factory than importing the schematic and running it. For example, validating the Schema and filling default values.

Finally, there are no good way for now to list files that were created or overwritten in a tree. Because Schematics was built to be hermetic, the Tree that you receive does not have local changes. Because of this, we have to go through all files.

Using Angular CLI

The best usage of Schematics for your users is currently through the Angular CLI. This means you should probably give it a try before publishing this to NPM. Here we will try to use our new myComponent schematic through the Angular CLI.

First, create an empty project with the CLI:

ng new my-project

Then in your new project, link the Schematics we just built:

npm link $PATH_TO_SCHEMATIC_PROJECT

Replace $PATH_TO_SCHEMATIC_PROJECT with the path to your project’s root. Note that users will install instead of linking, this is just to iterate faster locally while developing.

Once your schematic project is linked, you can use ng generate to call your schematics:

ng generate my-component:my-component someName

By default, if the schematic takes a name argument, the second argument of the generate command will be set to that name.

Voilà! This should be enough to get your users started. Please note that there’s also a default collection in the CLI configuration that you can set. For more information on the configuration, see the CLI wiki.

Conclusion

Just to recap what we learnt so far:

  1. How to use the Schematics CLI tool to create a new project.
  2. How to use the Schematics CLI tool to debug our own schematic project.
  3. How to use chain and externalSchematic rule factories to compose rules and call other schematics.
  4. How to use the Angular CLI to call our schematics (for both debugging and usage purposes).

In the next blog post, I’ll visit the Tree data structure a little deeper, as well as look at tasks, which can be used to call external processes in a smart and safe way.

Schematics is the first part of a greater effort with the Angular DevKit, which will comprise many other libraries in the future, and those efforts will be described in a separate post.

Cheers!