How to import content and schema while migrating them into newer alternatives
As well as importing the content and schema directly, we also provide support for modifying the items as part of the process.
For example, you may have taken an export from an Umbraco 8 site, and are looking to import it into a newer major version. In this situation, most content and schema will carry over without issue. However, you may have some items that are no longer compatible. Usually this is due to a property editor - either a built-in Umbraco one or one provided by a package. These may no longer be available in the new version.
Often though there is a similar replacement. Using Deploy's import feature we can transform the exported content for the obsolete property into that used by the new one during the import. The migration to a content set compatible with the new versions can then be completed.
For example, we can migrate from a Nested Content property in Umbraco 8 to a Block List in Umbraco 13.
We provide the necessary migration hooks for this to happen, divided into two types - artifact migrators and property migrators.
Artifact migrators work by transforming the serialized artifact of any imported artifact, via two interfaces:
IArtifactMigrator
- where the migration occurs at the artifact property level
IArtifactJsonMigrator
- where the migration occurs at the lower level of transforming the serialized JSON itself.
Implementations to handle common migrations of Data Types from obsoleted property editors are available:
ReplaceMediaPickerDataTypeArtifactMigrator
- migrates a Data Type from using the legacy Media Picker to the current version of this property editor
ReplaceNestedContentDataTypeArtifactMigrator
- migrates a Data Type based on the obsolete Nested Content property editor to the Block List
ReplaceGridDataTypeArtifactMigrator
- migrates a Data Type based on the legacy Grid layout into the Block Grid
ReplaceUnknownEditorDataTypeArtifactMigrator
- replaces any unknown editor alias with a label
We've also made available base implementations that you can use to build your own migrations. You may need to handle the transfer of information between other obsolete and replacement property editors that you have in your Umbraco application.
ArtifactMigratorBase<TArtifact>
- migrates the artifact of the specified type
DataTypeArtifactMigratorBase
- migrates Data Type artifacts
ReplaceDataTypeArtifactMigratorBase
- migrates a Data Type from one property editor to another
ArtifactJsonMigratorBase<TArtifact>
- migrates the JSON of the specified artifact type
ReplaceGridDataTypeArtifactMigratorBase
- migrates a Data Type based on the legacy Grid layout into the Block Grid
Property migrators work to transform the content property data itself. They are used in the Deploy content connectors (documents, media and members) when the property editor is changed during an import:
Again we have an interface:
IPropertyTypeMigrator
Implementations for common migrations:
MediaPickerPropertyTypeMigrator
NestedContentPropertyTypeMigrator
GridPropertyTypeMigrator
And a base type to help you build your own migrations:
PropertyTypeMigratorBase
GridPropertyTypeMigratorBase
Property editor changes are determined by comparing the PropertyEditorAliases
dictionary stored in the content artifact to the current Content Type/Data Type configuration. The dictionary contains editor aliases for each content property.
Migrators will run if you have registered them, so you can enable only the ones needed for your solution.
You can do this via a composer, as in the following example. Here we register two of the migrators shipped with Umbraco Deploy:
When an import is started, the following happens:
Artifact signatures are read from the import provider (using IArtifactImportProvider.GetArtifactSignatures()
).
The artifact signatures are sorted based on dependencies with Ordering
enabled (ensuring dependent items are processed in the correct order, like parent items before children and data types before document types).
For each artifact signature:
Check whether the entity type is allowed to be imported.
Publish an ArtifactImportingNotification
(cancelling will skip importing the artifact).
Publish a ValidateArtifactImportNotification
:
Deploy adds a default handler (ValidateArtifactImportDependenciesNotificationHandler
) to validate whether all dependencies are either in the import or already present in the current environment. It emits warnings for missing content artifacts, missing or different artifact checksums and errors for missing schema artifacts.
The import fails on validation errors or 'soft' fails on warnings if the WarningsAsErrors
import option is set.
Create a Deploy scope and context (containing the 'owner' user for auditing purposes and cultures to import, in case the user doesn't have edit permissions for all languages).
For each artifact signature:
Create a (readonly) Stream
for the artifact (using IArtifactImportProvider.CreateArtifactReadStream(Udi)
).
Deserialize the artifact into a generic JSON object (JsonNode
).
Parse the __version
and __type
properties and resolve the artifact type (using IArtifactTypeResolver
).
Migrate the JSON object (using IArtifactJsonMigrator
).
Deserialize the JSON object into the artifact type.
Migrate the artifact (using IArtifactMigrator
).
Initialize artifact processing (using IServiceConnector.ProcessInit(...)
) and track deploy state with next passes.
For each next process pass (starting at the lowest initial next pass):
Process artifact (using IServiceConnector.Process(...)
).
During processing: service connectors for content, media and members migrate property type values if a property editor alias has changed (using IPropertyTypeMigrator
).
When no next pass is required (the deploy state returns -1 as next pass):
Publish an ArtifactImportedNotification
.
Report the import process (using IProgress.Report(...)
).
The Deploy scope is completed, causing all scoped notifications to be published to handlers implementing IDistributedCacheNotificationHandler
) and completing the import.
Umbraco Deploy ships with migrators to handle the conversion of core property editors as they have changed, been removed or replaced between versions.
Open source migrators may be built by HQ or the community for property editors found in community packages. They will be made available for use and review via the Umbraco.Deploy.Contrib
package.
The Grid editor introduced in Umbraco 7 has been removed from Umbraco 14. Its functionality is replaced with the Block Grid.
With Deploy migrators we have support for migrating Data Type configuration and property data between these property editors.
Deploy adds the ReplaceGridDataTypeArtifactMigrator
and GridPropertyTypeMigrator
migrators by default, so using a custom migrator requires replacing the default ones:
The project you are importing into needs to know about any custom legacy Grid editor configurations to migrate to the Block Grid editor correctly. Make sure to either copy the grid.editors.config.js
and package.manifest
(containing grid editors) files or override the GetGridEditors()
method of the artifact migrator to provide this.
These implementations make use of the following conventions to migrate the data:
ReplaceGridDataTypeArtifactMigrator
:
Grid layouts are migrated to an existing or new element type with an alias based on the layout name, prefixed with gridLayout_
(this can be customized by overriding MigrateGridTemplate()
);
Row configurations are migrated to an existing or new element type with an alias based on the row name, prefixed with gridRow_
(this can be customized by overriding MigrateGridLayout()
);
Similarly, grid editors are migrated to an existing or new element type with an alias based on the editor alias, prefixed with gridEditor_
(this can be customized by overriding MigrateGridEditor()
). The default editors used in version 13 are returned by GetGridEditors()
and you can override this method to include your custom editors. Each migrated grid editor will have the following property types added to the element type:
The media
grid editor is migrated to multiple properties: the value
property contains the selected media item (using Media Picker v3), altText
the alternate text (using a Textbox) and caption
the caption (also using a Textbox);
The remaining grid editors create a single value
property that uses the following editors:
rte
- the default 'Rich Text Editor', falling back to the first Umbraco.TinyMCE
editor.
headline
- the default 'Textstring', falling back to the first Umbraco.TextBox
editor.
macro
and embed
grid editors are converted into rich text editors.
quote
or any other - use falling back to the first Umbraco.TextArea
editor.
The block label is also updated for the built-in grid editors, ensuring a nice preview is available (the WYSIWYG style previews are incompatible between these editors, so the custom views are not migrated);
Grid settings config and styles are migrated to a new element type with a random alias, prefixed with gridSettings_
(this can be customized by overriding MigrateGridSettings()
). This is because the migration only has context about the Data Type configuration (not the actual Data Type) and multiple Data Type can potentially use the same configuration (for config and styles), so there's no predictable way to create a unique alias. The migrated settings element type will have the property types added for the config and styles:
Each config setting is migrated to a property with an alias based on the key, prefixed with setting_
and added below a 'Settings' property group;
Similarly, each style is migrated to a property with an alias based on the key, prefixed with style_
and added below a 'Styles' property group;
The following property editors are used for these properties based on the config/style view:
radiobuttonlist
- a new 'Radio Button List' Data Type that uses the pre-values;
multivalues
- a new 'Checkbox List' Data Type that uses the pre-values;
textstring
- the default 'Textstring', falling back to the first Umbraco.TextBox
editor.
mediapicker
and imagepicker
- the default 'Media Picker' (v3, single image), falling back to the first Umbraco.MediaPicker3
editor.
boolean
- the default 'Checkbox', falling back to the first Umbraco.TrueFalse
editor.
number
- the default 'Numeric', falling back to the first Umbraco.Integer
editor.
treepicker
, treesource
, textarea
or any other - the default 'Textarea', falling back to the first Umbraco.TextArea
editor.
GridPropertyTypeMigrator
:
Gets the grid layout and row configuration element types based on the alias prefix/name convention used by the Data Type artifact migrator;
The grid editor values are migrated to the respective properties:
The media
grid editor converts the value to a media item with crops (based on the UDI or media path), including the focal point (although this needs to be enabled on the Data Type), alternate text and caption;
All other values are converted to a text value or otherwise to a JSON string;
If a row or cell contains settings config or styles and the corresponding block has a settings element type configured, the settings config and styles are migrated to their respective properties in a similar way, based on the property editor alias:
Umbraco.MediaPicker3
- removes url('
from the beginning and ')
from the end of the value (commonly used as a modifier and added to the stored value), before trying to get the media item by a path.
All other values are returned as-is.
Given the flexibility of the grid editor and Block Grid you may want to take further control over the migration. You can do that by creating your own migrator classes, that make use of our provided base classes. You would then register your own migrators instead of the ones shipped with Umbraco Deploy in your composer.
The base classes provide the following functionality. Methods you should look to override to amend the default behavior have been noted above.
ReplaceGridDataTypeArtifactMigratorBase
- replaces the Umbraco.Grid
Data Type editor alias with Umbraco.BlockGrid
and migrates the configuration:
The number of columns is copied over.
Grid layouts, row configurations and grid editors are migrated to blocks:
If multiple grid layouts are configured or if at least one contains multiple sections or isn't the full width, each grid layout will be migrated to a 'layout block' (an element type without properties).
If multiple row configurations are configured or if at least one contains areas that don't allow all grid editors or has a maximum amount of items set, each row configuration is migrated to a block (this is also always done when there are multiple grid layouts, as each layout can configure allowed row configurations).
All grid editors are migrated to blocks (allowing a single grid editor to be migrated to multiple blocks to support DocTypeGridEditor, as that allows selecting different element types).
The settings config and styles are migrated to a single element type (even though each setting can define whether it's supported for rows and/or cells) and used on the blocks that are allowed.
Block groups are added for Layout and Content and used on the corresponding block types.
GridPropertyTypeMigratorBase
- migrates the property data from the GridValue
into the BlockValue
(using the Umbraco.BlockGrid
layout):
The related Data Type is retrieved to get the configured blocks.
All grid control values are first migrated into their content blocks.
Settings config and styles for 'grid cells' are stored on the area within a row, but areas in the Block Grid can't have settings, so this is migrated into the first migrated grid control content block instead.
If a layout block can be found for the row configuration name, all grid controls are wrapped into that block.
Similarly, if a layout block can be found for the grid layout name, all items are wrapped into that block.
The JSON serialized BlockValue
is returned.
Doc Type Grid Editor was a community package commonly used with the legacy grid editor. If you are using this with Umbraco 7 and up, you can export and migrate into the Block Grid on Umbraco 13 or above.
Ensure you are running the latest version of Umbraco.Deploy.Contrib
compatible with your Umbraco major version.
In your new project, register the following migrators to add support for the import from Doc Type Grid Editor grids:
The migrators add the following behavior:
ReplaceDocTypeGridEditorDataTypeArtifactMigrator
extends ReplaceGridDataTypeArtifactMigrator
and ensures any DocTypeGridEditor is migrated to blocks using the allowed element types. If the element types aren't found the default implementation will migrate to new element types.
DocTypeGridEditorPropertyTypeMigrator
extends GridPropertyTypeMigrator
and ensures the Doc Type Grid Editor values are mapped one-to-one to the block item data.
The artifact migrator adds the default DocTypeGridEditor configuration (with alias docType
). This can be disabled by setting the AddDefaultDocTypeGridEditor
property to false
in a custom/inherited class. Similar to the base migrator, any custom DocTypeGridEditor configurations must be available to migrate to the Block Grid editor correctly.
Matryoshka was an Umbraco package that added tab support for document types in Umbraco. The feature was subsequently added to the product itself.
We provide a migrator for this package in Umbraco.Deploy.Contrib
.
This adds support for migrating Matryoshka Group Separators into native property groups. It removes the Matryoshka Data Types during import and migrates the document, media and member types. Native property groups are also changed into tabs, similarly to how they were displayed with Matryoshka installed.
To use, you register the migrators:
As described above, the nested content to block list migration will occur register the corresponding migrator with your application.
To help write your own migrations, we share the source code of an example that ships with Umbraco Deploy. This migration converts Nested Content to Block List.
First we have the artifact migrator that handles the conversion of the configuration stored with a datatype:
And secondly we have the property migrator that handles restructuring the content property data:
How to export content and schema from Umbraco 7 and import into a newer version
The import and export features are available for Umbraco Deploy supporting Umbraco 8 and above. It's not been ported back to Umbraco 7, hence you cannot trigger an export from the backoffice or use the service.
However, you can still use this feature to help migrate from Umbraco 7 to a supported major version. This requires adding additional logic to your Umbraco 7 project to create an export ZIP archive similar to newer versions.
You can generate an export archive in the same format as the import/export feature. This is done by adding the Umbraco.Deploy.Contrib.Export
assembly to your Umbraco 7 project (that already has Deploy and Deploy Contrib installed, see below). This archive can be imported into a newer Umbraco version by configuring the legacy import migrators. You can also apply additional migrators to update obsolete data types and property data into newer equivalents.
This is possible via code, by temporarily applying a composer to an Umbraco 7 project to generate the export archive on start-up:
To import this archive into a newer Umbraco project, you need to install either of these packages:
UmbracoDeploy.Contrib
4.3 for Umbraco 8
Umbraco.Deploy.Contrib
for Umbraco 10.2, 12.1, 13.1 or later
Then you need to configure the legacy artifact type resolver and migratory.
Artifact type resolvers allow resolving changes in the type that's stored in the __type
JSON property of the artifact. This is in case it moved to a different assembly or namespace (or got renamed) in a newer version. The legacy migrators handle the following changes:
Moving the pre-values of data types to the configuration property;
Moving the invariant release and expiration dates of content to the (culture variant) schedule property;
Moving the 'allowed at root' and 'allowed child content types' of content/media/member types to the permissions property;
Migrating the Data Type configuration from pre-values to the correct configuration objects and new editor aliases for:
Umbraco.CheckBoxList
(pre-values to value list)
Umbraco.ColorPickerAlias
to Umbraco.ColorPicker
(pre-values to value list)
Umbraco.ContentPicker2
to Umbraco.ContentPicker
(removes invalid start node ID)
Umbraco.ContentPickerAlias
to Umbraco.ContentPicker
(removes invalid start node ID)
Umbraco.Date
to Umbraco.DateTime
Umbraco.DropDown
to Umbraco.DropDownListFlexible
(pre-values to value list, single item select)
Umbraco.DropDownListFlexible
(pre-values to value list, defaults to multiple item select)
Umbraco.DropdownlistMultiplePublishKeys
to Umbraco.DropDownListFlexible
(pre-values to value list, defaults to multiple item select)
Umbraco.DropdownlistPublishingKeys
to Umbraco.DropDownListFlexible
(pre-values to value list, defaults to single item select)
Umbraco.DropDownMultiple
to Umbraco.DropDownListFlexible
(pre-values to value list, defaults to multiple item select)
Umbraco.MediaPicker2
to Umbraco.MediaPicker
(removes invalid start node ID, defaults to single item select)
Umbraco.MediaPicker
(removes invalid start node ID)
Umbraco.MemberPicker2
to Umbraco.MemberPicker
Umbraco.MultiNodeTreePicker2
to Umbraco.MultiNodeTreePicker
(removes invalid start node ID)
Umbraco.MultiNodeTreePicker
(removes invalid start node ID)
Umbraco.MultipleMediaPicker
to Umbraco.MediaPicker
(removes invalid start node ID, defaults to multiple item select)
Umbraco.NoEdit
to Umbraco.Label
Umbraco.RadioButtonList
(pre-values to value list, change database type from integer to nvarchar)
Umbraco.RelatedLinks2
to Umbraco.MultiUrlPicker
Umbraco.RelatedLinks
to Umbraco.MultiUrlPicker
Umbraco.Textbox
to Umbraco.TextBox
Umbraco.TextboxMultiple
to Umbraco.TextArea
Umbraco.TinyMCEv3
to Umbraco.TinyMCE
Migrating pre-value property values for:
Umbraco.CheckBoxList
Umbraco.DropDown.Flexible
Umbraco.RadioButtonList
The following composer adds the required legacy artifact type resolver and migrators. It also adds a custom resolver that marks the specified Document Type alias testElement
as the element type. Element types are a concept added in Umbraco 8 and are required for Document Types that are used in Nested Content.
It is recommended to start by importing only the schema and schema files (by deselecting 'Content' and 'Content files' in the dialog). Then, you can proceed with importing all content and schema together. The order in which the artifacts are imported depends on their dependencies. By importing the schema first, we ensure that the schema is updated before any content is processed.
Umbraco Deploy for Umbraco 7 is no longer supported and was only available on Umbraco Cloud. It was not released for use on-premise.
As such if you are looking to migrate from an Umbraco Cloud project running on Umbraco 7, you already have Umbraco Deploy installed.
If you have an Umbraco 7 on-premises website, you can use this guide to migrate from on-premises to Umbraco Cloud. Or to upgrade to a newer Deploy On-premises version. You need to obtain and install Umbraco Deploy for Umbraco 7 into your project, solely to use the export feature.
The export feature can be used without a license.
A license is required for the Umbraco project you are importing into. This can be a license that comes as part of an Umbraco Cloud subscription, or a Deploy On-premises one.
Use this guide to migrate from on-premises to Umbraco Cloud or to upgrade to a newer Deploy On-premises version.
Download the required dll
files for Umbraco Deploy for Umbraco 7 from the following links:
Umbraco Deploy v2.1.6: Latest Deploy Version 2 release for Umbraco CMS Version 7 (officially for use on Cloud)
Umbraco Deploy Contrib v2.0.0: Latest/only Deploy Contrib Version 2
Umbraco Deploy Export v2.0.0: For exporting all content/schema in Version 7
Install Umbraco Deploy with the Contrib and Export extensions.
Install Umbraco Deploy
, Deploy.Contrib
, and Deploy.Export
by copying the downloaded .dll
files into your Umbraco 7 site.
When copying the files over from Umbraco Deploy
you should not overwrite the following files (if you already had Umbraco Deploy installed):
Run the project to make sure it runs without any errors
Update the web.config
file with the required references for Umbraco Deploy:
Export Content.
Export your content, schema, and files to zip.
How to import and export content and schema between Umbraco environments and projects
The import and export feature of Umbraco Deploy allows you to transfer content and schema between Umbraco environments. Exports are made from one environment to a .zip
file. And this file is imported into another environment to update the Umbraco data there.
Umbraco Deploy provides two primary workflows for managing different types of Umbraco data:
Umbraco schema (such as document types and data types) are transferred as .uda
files serialized to disk. They are deployed to refresh the schema information in a destination environment along with code and template updates.
Umbraco content (such as content and media) are transferred by editors using backoffice operations.
We recommend using these approaches for day-to-day editorial and developer activities.
Import and export is intended more for larger transfer options, project upgrades, or one-off tasks when setting up new environments.
As import and export is a two-step process, it doesn't require inter-environment communication. This allows us to process much larger batches of information without running into hard limits imposed by Cloud hosting platforms.
We are also able provide hooks to allow for migrations of artifacts (such as data types) and property data when importing. This should allow you to migrate your Umbraco data from one Umbraco major version to a newer one.
To export content and schema, you can select either a specific item of content, or a whole tree or workspace.
When exporting, you can choose to include associated media files. Bear in mind that including media files for a large site can lead to a big zip file. So even with this option, you might want to consider a different method for transferring large amounts of media. For example using direct transfer between Cloud storage accounts or File Transfer Protocol (FTP).
If your account has access to the Settings section, you can also choose to include the schema information and related files as well.
Umbraco Deploy will then serialize all the selected items to individual files, archive them into a zip file and make that available for download. You can download the file using the Download button.
After the download, you should also delete the archive file from the server. You can do this immediately via the Delete button available in the dialog.
If you miss doing this, you can also clean up archive files from the Umbraco Deploy dashboard in the Settings section.
The exported archive files are saved to the Umbraco temp folder in the Deploy\Export
sub-directory. This is a temporary (non-persistent) location, local to the backoffice server and therefore shouldn't be used for long-term storage of exports. You can also only download the file from the export dialog in the backoffice.
Having previously exported content and schema to a zip file, you can import this into a new environment.
You can upload the file via the browser.
Similar to when exporting, you can choose to import everything from the archive file, or only content, schema or files.
Deploy does not touch the default maximum upload size, but you can configure this yourself by following the CMS documentation. On Umbraco Cloud, the upload size limit is 500 MB.
We validate the file before importing. Schema items that content depends on must either be in the upload itself or already exist on the target environment with the same details. If there are any issues that mean the import cannot proceed, it will be reported. You may also be given warnings for review. You can choose to ignore these and proceed if they aren't relevant to the action you are carrying out.
The import then proceeds, processing all the items provided in the zip file.
Once complete or on close of the dialog, the imported file will be deleted from the server. If this is missed, perhaps via a closed browser, you can also delete archive files from the Umbraco Deploy dashboard in the Settings section.
It is possible to migrate schema and content whilst importing. For example, to change Data Type using Nested Content to Block List and ensure content data is imported to the correct Block Editor format.
Deploy contains base classes and implementations to handle common migrations that need to be registered in code, as explained in Import with migrations.
The import and export feature is not available in Deploy 2 for Umbraco 7. We have though released a package to allow creating an export. This needs to be done in code and requires additional legacy migrators to be able to import into a newer version. This is explained in Migrating from Umbraco 7.
Underlying the functionality of import/export with Deploy is the import/export service, defined by the IArtifactImportExportService
.
You may have need to make use of this service directly if building something custom with the feature. For example you might want to import from or export to some custom storage.
The service interface defines two methods:
ExportArtifactsAsync
- takes a collection of artifacts and a storage provider defined by the IArtifactExportProvider
interface. The artifacts are serialized and exported to storage.
IArtifactExportProvider
defines methods for creating streams for writing serialized artifacts or files handled by Deploy (media, templates, stylesheets etc.).
ImportArtifactsAsync
- takes storage provider containing an import defined by the IArtifactImportProvider
interface. The artifacts from storage are imported into Umbraco.
IArtifactImportProvider
defines methods for creating streams for reading serialized artifacts or files handled by Deploy (media, templates, stylesheets etc.).
Implementations for IArtifactExportProvider
and IArtifactImportProvider
are provided for:
A physical directory.
An Umbraco file system.
A zip file.
These are all accessible for use via extension methods available on IArtifactImportExportService
found in the Umbraco.Deploy.Infrastructure.Extensions
namespace.
The following example shows this service in use, importing and exporting from a zip file on startup: