Internals

This document describes some of the internal workings of Nanoc. The audience of this document is potential contributors to Nanoc. You do not need to read or understand this document to use Nanoc.

Data

Nanoc’s data is divided into two kinds: entities and views. Entities contain data, while views wrap entities and provide useful APIs. When using Nanoc in the usual way, you will always deal with views, never with entities.

Entities include Item, Layout, their common superclass Document, ItemRep, …. They reside in lib/nanoc/base/entities. Views include MutableItemView, CompilationItemView, PostCompileItemRepView, …. They reside in lib/nanoc/base/views.

Views are, unlike entities, context-specific. The Item entity, for example, has MutableItemView, CompilationItemView and PostCompileItemView. MutableItemView is used during preprocessing and allows mutating properties of the item, while the other view classes do not. PostCompileItemView is used during preprocessing and provides #modified_reps, which is not available in the other item views.

Operations performed on an entity will typically yield another entity or a primitive, such as a string or a number. Similarly, operations performed on a view will typically yield another view. Converting from a view to an entity happens by calling #_unwrap, and wrapping an entity in a view happens by creating an instance of a view class that references the entity.

Outdatedness checking

Nanoc is a smart static-site generator; it avoids recompiling items unless necessary.

Nanoc will recompile an item if it is deemed to be outdated. The class responsible for determining whether or not an item is outdated is the OutdatednessChecker. An item is considered as outdated if any of the following conditions hold:

• The item’s content has changed
• The item’s attributes have changed
• The site configuration has changed
• The rule for this item has changed
• Some output files for this item do not exist
• New items or layouts have been added
• A dependent item or layout is considered as outdated

The last condition is necessary, because an item might use data of another item or layout.

Dependency tracking

If an item uses data from another entity, Nanoc will record a dependency from the former item onto the latter entity. This type of dependency is used to infer the outdatedness of an item, and is called a soft dependency. Nanoc records a dependency in the following cases:

• The raw content of an item or layout is accessed
• The compiled content of an item is accessed
• An attribute of an item or layout is accessed
• An attribute of the site configuration is accessed
• An item representation’s path is accessed
• The collection of items or layouts is accessed

For example, the following code snippet will generate dependencies:

<!-- Creates a dependency on the /about.* page -->
See the <%= link_to @items['/about.*'] %> page.

<!-- Creates a dependency on every item -->
See the <%= link_to @items.select { |i| i[:title] =~ /About/ }.first %> page.

<!-- Creates a dependency on any item whose identifier matches /gallery/*  -->
<% @items.find_all('/gallery/*').each do |i| %>
  <li><%= link_to i %></li>
<% end %>

For every dependency, Nanoc remembers which of the following properties are affected by the dependency:

• the raw content
• the attributes, and which
• the compiled content
• the path

An item will only be considered as outdated due to dependencies when the dependencies’ properties match the outdatedness reason of the targets of the dependency.

For example, if an item B is marked as outdated due to an attribute title being changed, and another item A has a dependency on the title attribute of former item B, then the item A will be considered as outdated:

If item B’s tagline attribute were changed, rather than the title attribute, then the item A would not be considered as outdated:

If the dependency were a dependency on the raw content rather than the attributes, the source of the dependency would not be considered as outdated:

Similarly, an item that depends on a certain configuration attribute will be considered as outdated when this configuration attribute has changed.

Compilation order

The order in which item representations are compiled, depends on the dependencies on compiled content between individual item representations.

Dependencies on compiled content

An item representation can depend on the compiled content of another item representation. For example, blog archive pages typically tend to contain (snippets of) the content of individual blog posts. An item representation which contains the compiled content of another item representation cannot be compiled until the compiled content of the dependent item representation is available. If this compiled content is not available and Nanoc tries to compile the item representation, then the compilation is suspended.

It is normal for the compilation for an item to be suspended. It merely indicates that the dependent item representation should be compiled first. The dependencies of an item representation are not known until Nanoc has finished compiling the item representation, and so Nanoc cannot make an accurate estimation of what a proper order of compilation is. For that reason, compilation suspensions are not only expected, but commonplace.

A dependency from one item representation onto another item representation’s compiled content is called a hard dependency, as opposed to a soft dependency, which is used in outdatedness checking. A compilation suspension always indicates a hard dependency.

Compilation selection

Once Nanoc has selected the items representations that it deems are outdated (see the Outdatedness checking section), it will compile these item reps in the best possible order. The class responsible for determining this order is the ItemRepSelector.

Nanoc will attempt to compile every outdated item representation sequentially. If an item representation cannot be compiled due to a compilation suspension, Nanoc will attempt to compile the item rep that is depended on. If the item rep that is depended on is also suspended, Nanoc will raise an error, informing the user of a dependency cycle.

Action providers

An action provider is a plugin that describes the actions that need to be taken in order to process each item. The rules DSL is the default (and at the moment of writing, only) action provider; it reads actions from the Rules file.

An action provider needs to support the following methods:

rep_names_for(item)

Given an item, returns the representation names for that item (e.g. [:default]).

action_sequence_for(rep)

Returns the sequence of actions to be taken in order to process the item representation rep.

An action sequence is an instance of Nanoc::Int::ActionSequence, and it contains zero or more of the following actions, in order (all in the Nanoc::Int::ProcessingActions namespace):

Filter.new(filter_name, params)

Apply the filter with the given filter_name (a Symbol), with optional parameters.

Layout.new(layout_identifier, params)

Apply the layout with the given identifier (a String), with optional parameters.

Snapshot.new(snapshot_names, paths)

Create a snapshot with one or more names (Symbols), and zero or more paths (Strings).

Compilation stages and phases

The compilation process is divided into a sequence of stages. Each of these is run only once, in sequence:

Preprocess

Run the preprocessor, defined in the Rules file, if any.

BuildReps

For each item, determine which representations it has, along with the processing instructions (“action sequences”) that go with it.

LoadStores

Read state information from the previous compilation (if any) in memory. This includes previous checksums, previous outdatedness information, etc.

CalculateChecksums

Calculate checksums for every item, layout, …

DetermineOutdatedness

For every item representation, determine whether or not it is outdated and therefore needs to be recompiled.

ForgetOutdatedDependencies

For every outdated item representation, remove dependencies that originate from it. The dependency information is not useful, as it is outdated.

StorePreCompilationState

Write out the current checksums along with the action sequences for each item representation.

Prune

Remove all files in the output directory that do not correspond to item representations.

CompileReps

Loop through all item representations and compile them. (See below for details.)

StorePostCompilationState

Write out all dependency information.

Postprocess

Run the postprocessor, defined in the Rules file, if any.

Cleanup

Remove all intermediate files that are no longer used after compilation is finished.

The CompileReps will loop through all item representations and compile them. Each item representation will go through each of the following phases, in sequence:

MarkDone

Yields to the next phase, and then marks the item representation as compiled.

Write

Yields to the next phase, and then writes out all files for this item representation.

Resume

Suspends compilation of the item representation when dependency errors occur, and resumes compilation when the dependency errors are resolves.

Cache

If possible, loads the already-compiled content for this item representation from the cache.

Recalculate

Executes all actions needed to calculate the compiled content for this item representation.