Extend Deploy

How to extend Umbraco Deploy to synchronize custom data

Umbraco Deploy supports the deployment of CMS schema information and definitions from the HQ's Forms package, along with managed content and media. Additionally, it can be extended by package or custom solution developers. This allows the deployment of custom data, such as that stored in your own database tables.

As a package or solution developer, you can hook into the disk-based serialization and deployment. It is similar to that used for Umbraco Document Types and Data Types. It's also possible to provide the ability for editors to deploy custom data via the Backoffice. In the same way that Umbraco content and media can be queued for transfer and restored.

Concepts and Examples

Entities

Entities are what you may be looking to transfer between two websites using Deploy. Within Umbraco, they are the Document Types, Data Type, documents etc. In a custom solution or a package, there may be representations of some other data that's being stored separately from Umbraco content. These can still be managed in the Backoffice using custom trees and editors.

For the purposes of subsequent code samples, we'll consider an example entity as a Plain Old Class Object (POCO) with a few properties.

The entity has no dependency on Umbraco or Umbraco Deploy; it can be constructed and managed however makes sense for the package or solution. The only requirement is that it has an ID that will be consistent across the environments (normally a Guid) and a name.

public class Example
{
    public Guid Id { get; set; }

    public string Name { get; set; }

    public string Description { get; set; }
}

Artifacts

Every entity Deploy works with, whether from Umbraco core or custom data, needs to have an artifact representation. You can consider an artifact as a container capable of knowing everything there is to know about a particular entity is defined. They are used as a transport object for communicating between Deploy environments.

They can also be serialized to JSON representations. These are visible in the .uda files seen on disk in the /data/revisions/ folder for schema transfers. It is also used when transferring content between different environments over HTTP requests.

Artifact classes must inherit from DeployArtifactBase.

The following example shows an artifact representing the entity and it's single property for transfer:

public class ExampleArtifact : DeployArtifactBase<GuidUdi>
{
    public ExampleArtifact(GuidUdi udi, IEnumerable<ArtifactDependency> dependencies = null)
        : base(udi, dependencies)
    { }

    public string Description { get; set; }
}

Control Over Serialization

In most cases the default settings Umbraco Deploy uses for serialization will be appropriate. For example, it ensures that culture specific values such as dates and decimal numbers are rendered using an invariant culture. This ensures that any differences in regional settings between source and destination servers are not a concern.

If you do need more control, attributes can be applied to the artifact properties.

For example, to ensure a decimal value is serialized to a consistent number of decimal places you can use the following. RoundingDecimalJsonConverter is found in the Umbraco.Deploy.Serialization namespace

[JsonConverter(typeof(RoundingDecimalJsonConverter), 2)]
public decimal Amount { get; set; }

Service Connectors

Service connectors are responsible for knowing how to handle the mapping between artifacts and entities. They know how to gather all the data required for the type of entity they correspond to, including figuring out what dependencies are needed. For example, in Umbraco, how a Document Type depends on a Data Type. They are responsible for packaging an entity as an artifact and for extracting an entity from an artifact and persisting it in a destination site.

Service connectors inherit from ServiceConnectorBase and are constructed with the artifact and entity as generic type arguments.

The class is decorated with a UdiDefinition via which the name of the entity type is provided. This needs to be unique across all entities so it's likely worth prefixing with something specific to your package or application.

The following example shows a service connector, responsible for handling the artifact shown above and it's related entity. There are no dependencies to consider here. More complex examples involve collating the dependencies and potentially handling extraction in more than one pass to ensure updates are made in the correct order.

An illustrative data service is provided via dependency injection. This will be whatever is appropriate for to use for Create, Read, Update and Delete (CRUD) operations around reading and writing of entities.

In Deploy 9.5/10.1, to improve performance on deploy operations, we introduced a cache. This change required the addition of new methods to interfaces, allowing the passing in of a cache parameter. In order to introduce this without breaking changes, we created some new interfaces and base classes.

In the example below, if instead we inherited from ServiceConnectorBase2, which has a type parameter of IServiceConnector2, we would be able to implement IArtifact? IServiceConnector2.GetArtifact(Udi udi, IContextCache contextCache). This would allow the connector to read and write to the cache and remove the use of the obsolete methods.

There's no harm in what is listed below though. It's only that the connectors won't be able to use the cache for any look-ups that are repeated in deploy operations. The obsolete methods won't be removed until Deploy 11. In that version we plan to return back to the original interface and class names. We also plan to introduce the new method overloads which will be a documented breaking change.

[UdiDefinition("mypackage-example", UdiType.GuidUdi)]
public class ExampleServiceConnector : ServiceConnectorBase<ExampleArtifact, GuidUdi, ArtifactDeployState<ExampleArtifact, Example>>
{
    private readonly IExampleDataService _exampleDataService;

    public ExampleServiceConnector(IExampleDataService exampleDataService) => _exampleDataService = exampleDataService;

    public override ExampleArtifact GetArtifact(object o)
    {
        var entity = o as Example;
        if (entity == null)
        {
            throw new InvalidEntityTypeException($"Unexpected entity type \"{o.GetType().FullName}\".");
        }

        return GetArtifact(entity.GetUdi(), entity);
    }

    public override ExampleArtifact GetArtifact(GuidUdi udi)
    {
        EnsureType(udi);
        var entity = _exampleDataService.GetExampleById(udi.Guid);

        return GetArtifact(udi, entity);
    }

    private ExampleArtifact GetArtifact(GuidUdi udi, Example entity)
    {
        if (entity == null)
        {
            return null;
        }

        var dependencies = new ArtifactDependencyCollection();
        var artifact = Map(udi, entity, dependencies);
        artifact.Dependencies = dependencies;

        return artifact;
    }

    private ExampleArtifact Map(GuidUdi udi, Example entity, ICollection<ArtifactDependency> dependencies)
    {
        var artifact = new ExampleArtifact(udi);
        artifact.Description = example.Description;
        return artifact;
    }

    private string[] ValidOpenSelectors => new[]
    {
        DeploySelector.This,
        DeploySelector.ThisAndDescendants,
        DeploySelector.DescendantsOfThis
    };
    private const string OpenUdiName = "All Examples";

    public override void Explode(UdiRange range, List<Udi> udis)
    {
        EnsureType(range.Udi);

        if (range.Udi.IsRoot)
        {
            EnsureSelector(range, ValidOpenSelectors);
            udis.AddRange(_exampleDataService.GetExamples().Select(e => e.GetUdi()));
        }
        else
        {
            var entity = _exampleDataService.GetExampleById(((GuidUdi)range.Udi).Guid);
            if (entity == null)
            {
                return;
            }

            EnsureSelector(range.Selector, DeploySelector.This);
            udis.Add(entity.GetUdi());
        }
    }

    public override NamedUdiRange GetRange(string entityType, string sid, string selector)
    {
        if (sid == "-1")
        {
            EnsureSelector(selector, ValidOpenSelectors);
            return new NamedUdiRange(Udi.Create("mypackage-example"), OpenUdiName, selector);
        }

        if (!Guid.TryParse(sid, out Guid id))
        {
            throw new ArgumentException("Invalid identifier.", nameof(sid));
        }

        var entity = _exampleDataService.GetExampleById(id);
        if (entity == null)
        {
            throw new ArgumentException("Could not find an entity with the specified identifier.", nameof(sid));
        }

        return GetRange(entity, selector);
    }

    public override NamedUdiRange GetRange(GuidUdi udi, string selector)
    {
        EnsureType(udi);

        if (udi.IsRoot)
        {
            EnsureSelector(selector, ValidOpenSelectors);
            return new NamedUdiRange(udi, OpenUdiName, selector);
        }

        var entity = _exampleDataService.GetExampleById(udi.Guid);
        if (entity == null)
        {
            throw new ArgumentException("Could not find an entity with the specified identifier.", nameof(udi));
        }

        return GetRange(entity, selector);
    }

    private static NamedUdiRange GetRange(Example e, string selector)