1 of 6

Examine

Examine uses Lucene as its search and index engine. Searching using Examine with Lucene can be powerful and fast.

What is Examine?

The Examine documentation is found here https://shazwazza.github.io/Examine/ and the source code repository for Examine is here https://github.com/Shazwazza/Examine.

Examine allows you to index and search data quickly. Examine is a library that sits on top of Lucene.Net, a high performance search engine library. Examine provides APIs to make searching and indexing as straight forward as possible. Umbraco provides a further layer on top, UmbracoExamine. This opens up the Umbraco-specific APIs for indexing and searching content and media out of the box.

Examine is provider based so it is extensible and allows you to configure your own custom indexes if required. The backoffice search in Umbraco also uses this same search engine, so you can trust that you're in good hands.

Quick start

Get up and running with Examine straight away with this quick start guide.

Customizing indexes

Learn how to customize the built in Umbraco indexes and how to create your own Lucene indexes using Examine in Umbraco.

PDF indexing and multisearchers

Learn how to index PDF files in Examine and how to create a multisearcher that searches through both the External Index and the Pdf Index.

Examine Management in the backoffice

Provides an overview of the available Examine functionality available directly within the Umbraco backoffice.

Details about subscribing to Examine events which can provide a way to modify the data being indexed.

API - Examine Manager

Describes the singleton object which exposes all of the index and search providers which are registered in the configuration of the Umbraco application.

Examine Management

Provides an overview of the available Examine functionality available directly within the Umbraco backoffice

Overview

The Umbraco backoffice allows you to view details about your Examine indexes and searchers - all in one place. You can see which fields are being indexed, rebuild the indexes if there's a problem, and test keywords to see what results would be returned.

The Examine Management dashboard, accessible from within the Settings section, is split into two sections: Indexers and Searchers.

Indexers

From the Indexers section, you can view details about each Examine index currently configured within your Umbraco installation. Clicking any of these indexes will show you additional options, each discussed below.

Index info

This section displays properties of the selected index, including the number of stored documents and fields.

Within the Indexers it displays the details for the index provider as well.

This can be useful to confirm the configuration that Umbraco is using and to ensure it is working as expected. This section also displays the full file path of the index itself.

This section also provides the ability to rebuild the index, should this be required. Depending on how much content your website has, rebuilding the search indexes could take a while and affect the site performance temporarily. It is not recommended to do this while the website is under high load.

Fields

From here, you can see the default system fields that are stored for each document within the search index. That includes the number of fields document, and the score which is calculated by Examine depending on how closely the individual results matched the search term.

Searchers

From the Searchers section, you can view details about each Examine searcher currently configured within your Umbraco installation. Clicking any of these searchers will take you to a search page, where you can test out your search terms.

You can see an example here how to configure an Examine searcher in the Examine Multisearcher documentation.

Search field

The search field allows you to enter a search term and receive results back from the searcher in question. You can confirm if your query is working as expected. Matching results are returned in their raw format, with the score, document ID and Name being returned. The score is calculated by Examine depending on how closely the individual results matched the search term.

Examine Manager

Accessing the singleton can be done by using dependency injection.

In a class you can inject the IExamineManager interface:

using Examine;

namespace MyCustomUmbracoSolution;

public class MyClass
{
    private readonly IExamineManager _examineManager;
    public MyClass(IExamineManager examineManager)
    {
        _examineManager = examineManager;
    }
}

In a view the IExamineManager can be injected as well:

@inject IExamineManager ExamineManager;

This returns an active instance of the ExamineManager which exposes operations such as:

Default index & search providers
Full collection of index & search providers
All indexing and searching methods

Searching

Important to note that the Search methods on the ExamineManager will call the Search methods of the default search provider specified in config. If you want to search using a specific provider, there are generally two approaches for this.

If you want to use the searcher of a specific index, you should get the the searcher via the index:

@inherits UmbracoViewPage
@using Examine
@using Umbraco.Cms.Core
@inject IExamineManager ExamineManager
@{

    // Get the search text from the query string
    string query = Context.Request.Query["query"];

    // Try to get the "ExternalIndex" from the Examine manager
    if (ExamineManager.TryGetIndex(Constants.UmbracoIndexes.ExternalIndexName, out IIndex index))
    {

        // Search via the searcher of the index
        ISearchResults searchResults = index.Searcher.Search(query);

        // Check whether the search revealed any results
        if (searchResults.Any())
        {
            <ul>
                @foreach (ISearchResult result in searchResults)
                {

                    // Skip the result if the ID is null
                    if (result.Id is null) continue;

                    // Skip the result if not found in the content cache
                    if (Umbraco.Content(result.Id) is not {} node) continue;

                    <li>
                        <a href="@node.Url()">@node.Name</a>
                    </li>

                }
            </ul>
        }

    }

}

If you have configured a custom searcher that you wish to use instead, you can access the searcher directly via the IExamineManager instance:

bool canGetSearcher = ExamineManager.TryGetSearcher("MyCustomSearcher", out ISearcher searcher);

An example using a custom searcher is below:

@inherits UmbracoViewPage
@using Examine
@inject IExamineManager ExamineManager
@{

    // Get the search text from the query string
    string query = Context.Request.Query["query"];

    // Try to get the "MyCustomSearcher" searcher
    if (ExamineManager.TryGetSearcher("MyCustomSearcher", out ISearcher searcher))
    {

        // Search via the searcher
        ISearchResults searchResults = searcher.Search(query);

        // Check whether the search revealed any results
        if (searchResults.Any())
        {
            <ul>
                @foreach (ISearchResult result in searchResults)
                {

                    // Skip the result if the ID is null
                    if (result.Id is null) continue;

                    // Skip the result if not found in the content cache
                    if (Umbraco.Content(result.Id) is not {} node) continue;

                    <li>
                        <a href="@node.Url()">@node.Name</a>
                    </li>

                }
            </ul>
        }

    }

}

Indexing

When you wanna populate an index, you will need to use the IExamineManager and get the specific index. The build-in index names are all available as constants from the Umbraco.Cms.Core.Constants.UmbracoIndexes namespace

if (_examineManager.TryGetIndex(Umbraco.Cms.Core.Constants.UmbracoIndexes.ExternalIndexName, out IIndex index))
{
   // Use index here
}

The indexing methods available on a single index are:

void DeleteFromIndex(IEnumerable<string> itemIds);
void DeleteFromIndex(this IIndex index, string itemId);
void IndexExists();
void IndexItems(IEnumerable<ValueSet> values);

Custom indexing

Learn how to build and customize the indexes that comes with your Umbraco website.

Customizing the built in indexes

You can modify the built-in indexes in the following ways:

- giving you control over exactly what data goes into them and how the fields are configured
Changing the field value types to change how values are stored in the index
Changing the IValueSetValidator to change what goes into the index
Take control of the entire index creation pipeline to change the implementation

We can do all this by using the ConfigureNamedOptions pattern.

Creating a ConfigureOptions class

We will start by creating a ConfigureExamineOptions class, that derives from IConfigureNamedOptions<LuceneDirectoryIndexOptions>:

In this sample we are altering the external index and thus we name the class ConfigureExternalIndexOptions. If you are altering multiple indexes, it is recommended to have separate classes for each index - i.e. ConfigureExternalIndexOptions for the external index, ConfigureInternalIndexOptions for the internal index and so on.

When using the ConfigureNamedOptions pattern, we have to register this in a composer for it to configure our indexes, this can be done like this:

Changing field value types

By default, Examine will store values into the Lucene index as "Full Text" fields, meaning the values will be indexed and analyzed for a textual search. However, if a field value is numerical, date/time, or another non-textual value type, you might want to change how the value is stored in the index. This will let you take advantage of some value type-specific search features such as numerical or date range.

The easiest way to modify how a field is configured is using the ConfigureNamedOptions pattern like so:

This will ensure that the price field in the index is treated as a double type (if the price field does not exist in the index, it is added).

Changing IValueSetValidator

An IValueSetValidator is responsible for validating a ValueSet to see if it should be included in the index. For example, by default the validation process for the ExternalIndex checks if a ValueSet has a category type of either "media" or "content" (not member). If a ValueSet was passed to the ExternalIndex and it did not pass this requirement it would be ignored.

The IValueSetValidator is also responsible for filtering the data in the ValueSet. For example, by default the validator for the MemberIndex will validate on all the default member properties, so an extra property "PhoneNumber", would not pass validation, and therefore not be included.

The IValueSetValidator implementation for the built-in indexes, can be changed like this:

Remember to register ConfigureMemberIndexOptions in your composer.

Creating your own index

The following example will show how to create an index that will only include nodes based on the document type product.

We always recommend that you use the existing built in ExternalIndex. You should then query based on the NodeTypeAlias instead of creating a new separate index based on that particular node type. However, should the need arise, the example below will show you how to do it.

To create this index we need five things:

An UmbracoExamineIndex implementation that defines the index.
An IConfigureNamedOptions implementation that configures the index fields and options.
An IValueSetBuilder implementation that builds index value sets a piece of content.
An IndexPopulator implementation that populates the index with the value sets for all applicable content.
An INotificationHandler implementation that updates the index when content changes.
A composer that adds all these services to the runtime.

ProductIndex

ConfigureProductIndexOptions

ProductIndexValueSetBuilder

ProductIndexPopulator

This is only an example of how you could do indexing. In this example, we're indexing all content, both published and unpublished.

ProductIndexingNotificationHandler

The index will only update its content when you manually trigger an index rebuild in the Examine dashboard. This is not always the desired behavior for a custom index.

To update your index when content changes, you can use notification handlers.

ExamineComposer

The order of these registrations matters. It is important to register your index with AddExamineLuceneIndex before calling ConfigureOptions.

Result

PDF indexes and multisearchers

If you want to index PDF files and search for them you will need to use the UmbracoExamine.Pdf extension package.

Installation

Install with NuGet: dotnet add package Umbraco.ExaminePDF

This will create a new Examine index called "PDFIndex", which will appear in "Examine Management" dashboard under the "Settings" section. Using this index you can start searching the contents of any PDF files uploaded to the media section.

Multi-index searchers

A multi-index searcher is a searcher that can search multiple indexes. This can be helpful when you for example want to search both the external and internal indexes. You can register a multi-index searcher with the ExamineManager on startup like:

using Examine;
using Umbraco.Cms.Core;
using Umbraco.Cms.Core.Composing;
using Umbraco.Cms.Core.DependencyInjection;
using UmbracoExamine.PDF;

namespace MySite.MyCustomIndex;

[ComposeAfter(typeof(ExaminePdfComposer))]
public class ExamineComposer : IComposer
{
    public void Compose(IUmbracoBuilder builder)
    {
        builder.Services.AddExamineLuceneMultiSearcher("MultiSearcher", new[] {Constants.UmbracoIndexes.ExternalIndexName, PdfIndexConstants.PdfIndexName});
    }
}

With this approach, the multi-index searcher will show up in the "Examine Management" dashboard.

The multi-index searcher can be resolved in code from the ExamineManager like this:

if (_examineManager.TryGetSearcher("MultiSearcher", out var searcher))
{
    //TODO: use the `searcher` to search
}

The implementation of IPdfTextExtractor is PdfSharpTextExtractor in this library, which uses PDFSharp to extract the bytes to convert to text. That implementation doesn't deal well with Unicode text which means when some PDF files are read, the result will be 'junk' strings.

It is certainly possible to replace the IPdfTextExtractor using your own composer like

composition.RegisterUnique<IPdfTextExtractor, MyCustomSharpTextExtractor>();

The iTextSharp library deals with Unicode in a better way but is a paid for license. If you wish to use iTextSharp or another PDF library you can swap out the IPdfTextExtractor with your own implementation.

Quick-start

This guide will help you get set up quickly using Examine with minimal configuration options. Umbraco ships Examine with 3 indexes: internal, external, and members. The internal index should not be used for searching when returning results on a public website because it includes content that has not been published yet. Instead, you can use the external index to get up and running.

Performing a search

In the coming examples, the has been used, as it provides some example content that can be searched. Therefore, some of the examples below may require 'the setting up of templates, etc' if you follow the guide on your existing site.

The starter kit comes with some Templates, Document Types, and content nodes created already. We will use some of these to set up a basic search system. This is a 'Quick Start' guide, as many more complex searches are possible with Examine.

We will make it possible to 'search' on the People page, by adding a search bar to the template page: people.cshtml - add the following form at the top of the template, but underneath the <nav> element:

This will create a basic input field at the top of the page and make it post to the same people page when submitted along with the search term.

Handling the search request

Let's start by creating a PeopleController that derives from RenderController and add an Index method.

It is important to name our controller by the convention NameOfViewController. In our case the view is named People, so the controller is named PeopleController.

Adding a Service that handles our search logic

To search anything from our controller, we first need to create a service that handles the actual search logic. We'll start by creating an interface for our service.

Now create a default implementation of the service interface.

And finally register the service in Startup.

Examine Search Index

To perform the search we will first need to get a reference to the particular Examine index that we want to search. Then we will use this index to access its corresponding Searcher. We use the Searcher to construct the query logic to execute and search the index.

Umbraco ships with three indexes:

ExternalIndex - available to use for indexing published unprotected content.
InternalIndex - which Umbraco's backoffice search uses.
MembersIndex - which Umbraco's Membership implementation uses.

The service IExamineManager is used to retrieve an Examine index by its 'alias', so we need to inject that service into our SearchService.

Creating the Search Query

With the IExamineManager injected in our SearchService, we can implement the SearchContentNames method. We do this using the Searcher for the Examine index 'ExternalIndex'.

We reference the External index by its alias "ExternalIndex". Umbraco has a set of 'Constants' that refer to the indexes that can be more convenient to use Constants.UmbracoIndexes. So, in the example here we could have used Constants.UmbracoIndexes.ExternalIndexName instead of "ExternalIndex".

The Searcher has a CreateQuery method, where you can choose to search content, media or members eg:

From here you can see how we can chain together the logic to perform the search. In the example, we are searching all content using the person Document Type, where the nodeName is equal to the search term that was typed in the input bar.

Calling .Execute() at the end of the query logic triggers the search and returns a set of matching search results, which we can loop through to get the IDs of the resulting content items.

Getting the content

We want to retrieve the actual content from the IDs. For that, we need the UmbracoHelper, which must be injected into our service as well. The final implementation of SearchService then looks like this.

After getting the ids from our search, we then loop through the list and return the content.

Creating a custom view model

We will now need a custom view model so that we can pass our search results to the view. Our view model needs to inherit from PublishedContentWrapped because our People view is expecting a model that is content. We then wrap the content and add the search data, all in a convenient view model.

Using the service and view model in the controller

Now that we've created our service to handle the actual search logic, and our view model to pass the search results to the view, let's look at using them in the controller. We will want to update the Index() method to get out the query string from the request, then create a view model and populate the SearchResults property by using our service.

Updating the view to use the viewmodel

The final thing we need to do is update the view to use our new view model. We do that by changing the @inherits line in the view.

Let's now use the view model to display the search results. We'll place them directly under the form we created earlier.

Different ways to query

Examine has a lot of different ways to query data. Building upon the example from before, here are a few other searches that can be done to get different data:

Search through all nodes

Let's say you want to search through all content nodes by their file names. You could amend the query from before like this:

Search using Lucene queries

To do the search like above, but only use Lucene to query, amend the query from before like this:

Search children of a specific node

To search through all child nodes of a specific node by their bodyText property, amend the query from before like this:

Quick-start

Performing a search

This will create a basic input field at the top of the page and make it post to the same people page when submitted along with the search term.

Handling the search request

The best practice for POST requests is to encapsulate the request handling in a controller. To do this we will leverage the concept of .

Let's start by creating a PeopleController that derives from RenderController and add an Index method.

It is important to name our controller by the convention NameOfViewController. In our case the view is named People, so the controller is named PeopleController.

using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.ViewEngines;
using Microsoft.Extensions.Logging;
using Umbraco.Cms.Core.Web;
using Umbraco.Cms.Web.Common.Controllers;

namespace MyStarterKitSite.Controllers;

public class PeopleController : RenderController
{
    public PeopleController(
    ILogger<RenderController> logger,
    ICompositeViewEngine compositeViewEngine,
    IUmbracoContextAccessor umbracoContextAccessor)
    : base(
        logger,
        compositeViewEngine,
        umbracoContextAccessor)
    {
    }

    public override IActionResult Index()
    {
        return CurrentTemplate(CurrentPage);
    }
}

Adding a Service that handles our search logic

To search anything from our controller, we first need to create a service that handles the actual search logic. We'll start by creating an interface for our service.

using System.Collections.Generic;
using Umbraco.Cms.Core.Models.PublishedContent;

namespace MyStarterKitSite.Services;

public interface ISearchService
{
    IEnumerable<IPublishedContent> SearchContentNames(string query);
}

Now create a default implementation of the service interface.

using Umbraco.Cms.Core.Models.PublishedContent;
using Umbraco.Cms.Web.UI.Services;

namespace MyStarterKitSite.Services;

public class SearchService : ISearchService
{
    public IEnumerable<IPublishedContent> SearchContentNames(string query) => throw new NotImplementedException();
}

And finally register the service in Startup.

// ... (removed for abbreviation)
builder.Services.AddTransient<ISearchService, SearchService>();

Examine Search Index

Umbraco ships with three indexes:

ExternalIndex - available to use for indexing published unprotected content.
InternalIndex - which Umbraco's backoffice search uses.
MembersIndex - which Umbraco's Membership implementation uses.

if you need to analyse text in a different language for example.

The service IExamineManager is used to retrieve an Examine index by its 'alias', so we need to inject that service into our SearchService.

using System.Collections.Generic;
using Examine;
using Umbraco.Cms.Core.Models.PublishedContent;

namespace MyStarterKitSite.Services;

public class SearchService : ISearchService
{
    private readonly IExamineManager _examineManager;
    public SearchService(IExamineManager examineManager)
    {
        _examineManager = examineManager;
    }
    public IEnumerable<IPublishedContent> SearchContentNames(string query) => throw new NotImplementedException();
}

Creating the Search Query

With the IExamineManager injected in our SearchService, we can implement the SearchContentNames method. We do this using the Searcher for the Examine index 'ExternalIndex'.

IEnumerable<string> ids = Array.Empty<string>();
if (!string.IsNullOrEmpty(query) && _examineManager.TryGetIndex("ExternalIndex", out IIndex? index))
{
    ids = index
        .Searcher
        .CreateQuery("content")
        .NodeTypeAlias("person")
        .And()
        .Field("nodeName", query)
        .Execute()
        .Select(x => x.Id);
}

The Searcher has a CreateQuery method, where you can choose to search content, media or members eg:

Searcher.CreateQuery("content")

Searcher.CreateQuery("content").NodeTypeAlias("person").And().Field("nodeName", searchTerm)

Calling .Execute() at the end of the query logic triggers the search and returns a set of matching search results, which we can loop through to get the IDs of the resulting content items.

Getting the content

using System;
using System.Collections.Generic;
using System.Linq;
using Examine;
using Umbraco.Cms.Core.Models.PublishedContent;
using Umbraco.Cms.Web.Common;
using Umbraco.Extensions;

namespace MyStarterKitSite.Services;

public class SearchService : ISearchService
{
    private readonly IExamineManager _examineManager;
    private readonly UmbracoHelper _umbracoHelper;

    public SearchService(IExamineManager examineManager, UmbracoHelper umbracoHelper)
    {
        _examineManager = examineManager;
        _umbracoHelper = umbracoHelper;
    }

    public IEnumerable<IPublishedContent> SearchContentNames(string query)
    {
        IEnumerable<string> ids = Array.Empty<string>();
        if (!string.IsNullOrEmpty(query) && _examineManager.TryGetIndex("ExternalIndex", out IIndex? index))
        {
            ids = index
                .Searcher
                .CreateQuery("content")
                .NodeTypeAlias("person")
                .And()
                .Field("nodeName", query)
                .Execute()
                .Select(x => x.Id);
        }

        foreach (var id in ids)
        {
            yield return _umbracoHelper.Content(id);
        }
    }
}

After getting the ids from our search, we then loop through the list and return the content.

Creating a custom view model

using System.Collections.Generic;
using System.Linq;
using Umbraco.Cms.Core.Models.PublishedContent;

namespace MyStarterKitSite.Models;

public class SearchViewModel : PublishedContentWrapped
{
    public SearchViewModel(IPublishedContent content, IPublishedValueFallback publishedValueFallback)
        : base(content, publishedValueFallback)
    {
    }

    public IEnumerable<IPublishedContent> SearchResults { get; set; } = Enumerable.Empty<IPublishedContent>();
    public bool HasSearched { get; set; }
}

Using the service and view model in the controller

using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.ViewEngines;
using Microsoft.Extensions.Logging;
using MyStarterKitSite.Models;
using MyStarterKitSite.Services;
using Umbraco.Cms.Core.Models.PublishedContent;
using Umbraco.Cms.Core.Web;
using Umbraco.Cms.Web.Common.Controllers;

namespace MyStarterKitSite.Controllers;

public class PeopleController : RenderController
{
    private readonly IPublishedValueFallback _publishedValueFallback;
    private readonly ISearchService _searchService;

    public PeopleController(
        ILogger<RenderController> logger,
        ICompositeViewEngine compositeViewEngine,
        IUmbracoContextAccessor umbracoContextAccessor,
        IPublishedValueFallback publishedValueFallback,
        ISearchService searchService)
        : base(logger,
            compositeViewEngine,
            umbracoContextAccessor)
    {
        _publishedValueFallback = publishedValueFallback;
        _searchService = searchService;
    }

    public override IActionResult Index()
    {
        // Get the queryString from the request
        string queryString = HttpContext.Request.Query["query"];

        // Create the view model and pass it to the view
        SearchViewModel viewModel = new(CurrentPage!, _publishedValueFallback)
        {
            SearchResults = _searchService.SearchContentNames(queryString),
            HasSearched = !string.IsNullOrEmpty(queryString),
        };

        return CurrentTemplate(viewModel);
    }
}

Updating the view to use the viewmodel

The final thing we need to do is update the view to use our new view model. We do that by changing the @inherits line in the view.

@inherits Umbraco.Cms.Web.Common.Views.UmbracoViewPage<MyStarterKitSite.Models.SearchViewModel>

Let's now use the view model to display the search results. We'll place them directly under the form we created earlier.

<div>
    @if (Model.SearchResults.Any())
    {
        <ul>
            @foreach (var content in Model.SearchResults)
            {
                <li>
                    <a href="@content.Url()">@content.Name</a>
                </li>
            }
        </ul>
    }
    else if(Model.HasSearched)
    {
        <p>No results found</p>
    }
</div>

Different ways to query

Examine has a lot of different ways to query data. Building upon the example from before, here are a few other searches that can be done to get different data:

Search through all nodes

Let's say you want to search through all content nodes by their file names. You could amend the query from before like this:

Searcher.CreateQuery("content").Field("nodeName", searchTerm).Execute();

Search using Lucene queries

To do the search like above, but only use Lucene to query, amend the query from before like this:

Searcher.CreateQuery().NativeQuery("+__IndexType:content +nodeName:" + searchTerm).Execute();

Search children of a specific node

To search through all child nodes of a specific node by their bodyText property, amend the query from before like this:

Searcher.CreateQuery("content").ParentId(1105).And().Field("bodyText", searchTerm).Execute();

Custom indexing

Learn how to build and customize the indexes that comes with your Umbraco website.

Customizing the built in indexes

You can modify the built-in indexes in the following ways:

- giving you control over exactly what data goes into them and how the fields are configured
Changing the field value types to change how values are stored in the index
Changing the IValueSetValidator to change what goes into the index
Take control of the entire index creation pipeline to change the implementation

We can do all this by using the ConfigureNamedOptions pattern.

Creating a ConfigureOptions class

We will start by creating a ConfigureExamineOptions class, that derives from IConfigureNamedOptions<LuceneDirectoryIndexOptions>:

When using the ConfigureNamedOptions pattern, we have to register this in a composer for it to configure our indexes, this can be done like this:

using Microsoft.Extensions.DependencyInjection;
using Umbraco.Cms.Core.Composing;
using Umbraco.Cms.Core.DependencyInjection;

namespace Umbraco.Docs.Samples.Web.CustomIndexing;

public class ExamineComposer : IComposer
{
    public void Compose(IUmbracoBuilder builder)
    {
        builder.Services.ConfigureOptions<ConfigureExternalIndexOptions>();
    }
}

Changing field value types

There is some documentation about this in the .

The easiest way to modify how a field is configured is using the ConfigureNamedOptions pattern like so:

using Examine;
using Examine.Lucene;
using Microsoft.Extensions.Options;
using Umbraco.Cms.Core;

namespace Umbraco.Docs.Samples.Web.CustomIndexing;

public class ConfigureExternalIndexOptions : IConfigureNamedOptions<LuceneDirectoryIndexOptions>
{
    public void Configure(string name, LuceneDirectoryIndexOptions options)
    {
        if (name.Equals(Constants.UmbracoIndexes.ExternalIndexName))
        {
            options.FieldDefinitions.AddOrUpdate(new FieldDefinition("price", FieldDefinitionTypes.Double));
        }
    }

    // Part of the interface, but does not need to be implemented for this.
    public void Configure(LuceneDirectoryIndexOptions options)
    {
        throw new System.NotImplementedException();
    }
}

This will ensure that the price field in the index is treated as a double type (if the price field does not exist in the index, it is added).

Changing IValueSetValidator

The IValueSetValidator implementation for the built-in indexes, can be changed like this:

using Examine.Lucene;
using Microsoft.Extensions.Options;
using Umbraco.Cms.Core;
using Umbraco.Cms.Infrastructure.Examine;

namespace Umbraco.Docs.Samples.Web.CustomIndexing;

public class ConfigureMemberIndexOptions : IConfigureNamedOptions<LuceneDirectoryIndexOptions>
{
    public void Configure(string name, LuceneDirectoryIndexOptions options)
    {
        if (name.Equals(Constants.UmbracoIndexes.MembersIndexName))
        {
            options.Validator = new MemberValueSetValidator(null, null, new[] {"email"}, null);
        }
    }

    // Part of the interface, but does not need to be implemented for this.
    public void Configure(LuceneDirectoryIndexOptions options)
    {
        throw new System.NotImplementedException();
    }
}

Remember to register ConfigureMemberIndexOptions in your composer.

Creating your own index

The following example will show how to create an index that will only include nodes based on the document type product.

Take a look at our to see some examples of how to search the ExternalIndex.

To create this index we need five things:

An UmbracoExamineIndex implementation that defines the index.
An IConfigureNamedOptions implementation that configures the index fields and options.
An IValueSetBuilder implementation that builds index value sets a piece of content.
An IndexPopulator implementation that populates the index with the value sets for all applicable content.
An INotificationHandler implementation that updates the index when content changes.
A composer that adds all these services to the runtime.

ProductIndex

using Examine.Lucene;
using Microsoft.Extensions.Options;
using Umbraco.Cms.Core.Services;
using Umbraco.Cms.Infrastructure.Examine;
using IHostingEnvironment = Umbraco.Cms.Core.Hosting.IHostingEnvironment;

namespace Umbraco.Docs.Samples.Web.CustomIndexing;

public class ProductIndex : UmbracoExamineIndex
{
    public ProductIndex(
        ILoggerFactory loggerFactory,
        string name,
        IOptionsMonitor<LuceneDirectoryIndexOptions> indexOptions,
        IHostingEnvironment hostingEnvironment,
        IRuntimeState runtimeState)
        : base(loggerFactory,
        name,
        indexOptions,
        hostingEnvironment,
        runtimeState)
    {
    }
}

ConfigureProductIndexOptions

using Examine;
using Examine.Lucene;
using Lucene.Net.Analysis.Standard;
using Lucene.Net.Index;
using Lucene.Net.Util;
using Microsoft.Extensions.Options;
using Umbraco.Cms.Core.Configuration.Models;

namespace Umbraco.Docs.Samples.Web.CustomIndexing;

public class ConfigureProductIndexOptions : IConfigureNamedOptions<LuceneDirectoryIndexOptions>
{
    private readonly IOptions<IndexCreatorSettings> _settings;

    public ConfigureProductIndexOptions(IOptions<IndexCreatorSettings> settings)
        => _settings = settings;

    public void Configure(string? name, LuceneDirectoryIndexOptions options)
    {
        if (name?.Equals("ProductIndex") is false)
        {
            return;
        }

        options.Analyzer = new StandardAnalyzer(LuceneVersion.LUCENE_48);

        options.FieldDefinitions = new(
            new("id", FieldDefinitionTypes.Integer),
            new("name", FieldDefinitionTypes.FullText)
        );

        options.UnlockIndex = true;

        if (_settings.Value.LuceneDirectoryFactory == LuceneDirectoryFactory.SyncedTempFileSystemDirectoryFactory)
        {
            // if this directory factory is enabled then a snapshot deletion policy is required
            options.IndexDeletionPolicy = new SnapshotDeletionPolicy(new KeepOnlyLastCommitDeletionPolicy());
        }
    }

    // not used
    public void Configure(LuceneDirectoryIndexOptions options) => throw new NotImplementedException();
}

ProductIndexValueSetBuilder

using Examine;
using Umbraco.Cms.Core.Models;
using Umbraco.Cms.Infrastructure.Examine

namespace Umbraco.Docs.Samples.Web.CustomIndexing;

public class ProductIndexValueSetBuilder : IValueSetBuilder<IContent>
{
    public IEnumerable<ValueSet> GetValueSets(params IContent[] contents)
    {
        foreach (IContent content in contents.Where(CanAddToIndex))
        {
            var indexValues = new Dictionary<string, object>
            {
                // this is a special field used to display the content name in the Examine dashboard
                [UmbracoExamineFieldNames.NodeNameFieldName] = content.Name!,
                ["name"] = content.Name!,
                // add the fields you want in the index
                ["nodeName"] = content.Name!,
                ["id"] = content.Id,
            };

            yield return new ValueSet(content.Id.ToString(), IndexTypes.Content, content.ContentType.Alias, indexValues);
        }
    }

    // filter out all content types except "product"
    private bool CanAddToIndex(IContent content) => content.ContentType.Alias == "product";
}

ProductIndexPopulator

using Examine;
using Umbraco.Cms.Core.Models;
using Umbraco.Cms.Core.Services;
using Umbraco.Cms.Infrastructure.Examine;

namespace Umbraco.Docs.Samples.Web.CustomIndexing;

public class ProductIndexPopulator : IndexPopulator
{
    private readonly IContentService _contentService;
    private readonly ProductIndexValueSetBuilder _productIndexValueSetBuilder;

    public ProductIndexPopulator(IContentService contentService, ProductIndexValueSetBuilder productIndexValueSetBuilder)
    {
        _contentService = contentService;
        _productIndexValueSetBuilder = productIndexValueSetBuilder;
        RegisterIndex("ProductIndex");
    }

    protected override void PopulateIndexes(IReadOnlyList<IIndex> indexes)
    {
        foreach (IIndex index in indexes)
        {
            IContent[] roots = _contentService.GetRootContent().ToArray();
            index.IndexItems(_productIndexValueSetBuilder.GetValueSets(roots));

            foreach (IContent root in roots)
            {
                const int pageSize = 10000;
                var pageIndex = 0;
                IContent[] descendants;
                do
                {
                    descendants = _contentService.GetPagedDescendants(root.Id, pageIndex, pageSize, out _).ToArray();
                    IEnumerable<ValueSet> valueSets = _productIndexValueSetBuilder.GetValueSets(descendants);
                    index.IndexItems(valueSets);

                    pageIndex++;
                }
                while (descendants.Length == pageSize);
            }
        }
    }
}

This is only an example of how you could do indexing. In this example, we're indexing all content, both published and unpublished.

In certain scenarios only published content should be added to the index. To achieve that, you will need to implement your own logic to filter out unpublished content. This can be somewhat tricky as the published state can vary throughout an entire structure of content nodes in the content tree. For inspiration on how to go about such filtering, you can look at the .

ProductIndexingNotificationHandler

The index will only update its content when you manually trigger an index rebuild in the Examine dashboard. This is not always the desired behavior for a custom index.

To update your index when content changes, you can use notification handlers.

The following handler class does not automatically update the descendant items of the modified content nodes, such as removing descendants of deleted content. If changes to the parent content item can affect its children or descendant items in your setup, please refer to the . Such logic should be applied when both removing and reindexing content items of type product.

using Examine;
using Umbraco.Cms.Core;
using Umbraco.Cms.Core.Cache;
using Umbraco.Cms.Core.Events;
using Umbraco.Cms.Core.Models;
using Umbraco.Cms.Core.Notifications;
using Umbraco.Cms.Core.Services;
using Umbraco.Cms.Core.Services.Changes;
using Umbraco.Cms.Core.Sync;
using Umbraco.Cms.Infrastructure;
using Umbraco.Cms.Infrastructure.Search;

namespace Umbraco.Docs.Samples.Web.CustomIndexing;

public class ProductIndexingNotificationHandler : INotificationHandler<ContentCacheRefresherNotification>
{
    private readonly IRuntimeState _runtimeState;
    private readonly IUmbracoIndexingHandler _umbracoIndexingHandler;
    private readonly IExamineManager _examineManager;
    private readonly IContentService _contentService;
    private readonly ProductIndexValueSetBuilder _productIndexValueSetBuilder;

    public ProductIndexingNotificationHandler(
        IRuntimeState runtimeState,
        IUmbracoIndexingHandler umbracoIndexingHandler,
        IExamineManager examineManager,
        IContentService contentService,
        ProductIndexValueSetBuilder productIndexValueSetBuilder)
    {
        _runtimeState = runtimeState;
        _umbracoIndexingHandler = umbracoIndexingHandler;
        _examineManager = examineManager;
        _contentService = contentService;
        _productIndexValueSetBuilder = productIndexValueSetBuilder;
    }

    /// <summary>
    ///     Updates the index based on content changes.
    /// </summary>
    public void Handle(ContentCacheRefresherNotification notification)
    {
        if (NotificationHandlingIsDisabled())
        {
            return;
        }

        if (!_examineManager.TryGetIndex("ProductIndex", out IIndex? index))
        {
            throw new InvalidOperationException("Could not obtain the product index");
        }

        ContentCacheRefresher.JsonPayload[] payloads = GetNotificationPayloads(notification);

        foreach (ContentCacheRefresher.JsonPayload payload in payloads)
        {
            // Remove
            if (payload.ChangeTypes.HasType(TreeChangeTypes.Remove))
            {
                index.DeleteFromIndex(payload.Id.ToString());
            }
            // Reindex
            else if (payload.ChangeTypes.HasType(TreeChangeTypes.RefreshNode) ||
                     payload.ChangeTypes.HasType(TreeChangeTypes.RefreshBranch))
            {
                IContent? content = _contentService.GetById(payload.Id);
                if (content == null || content.Trashed)
                {
                    index.DeleteFromIndex(payload.Id.ToString());
                    continue;
                }

                IEnumerable<ValueSet> valueSets = _productIndexValueSetBuilder.GetValueSets(content);
                index.IndexItems(valueSets);
            }
        }
    }

    private bool NotificationHandlingIsDisabled()
    {
        // Only handle events when the site is running.
        if (_runtimeState.Level != RuntimeLevel.Run)
        {
            return true;
        }

        if (_umbracoIndexingHandler.Enabled == false)
        {
            return true;
        }

        if (Suspendable.ExamineEvents.CanIndex == false)
        {
            return true;
        }

        return false;
    }

    private ContentCacheRefresher.JsonPayload[] GetNotificationPayloads(CacheRefresherNotification notification)
    {
        if (notification.MessageType != MessageType.RefreshByPayload ||
            notification.MessageObject is not ContentCacheRefresher.JsonPayload[] payloads)
        {
            throw new NotSupportedException();
        }

        return payloads;
    }
}

You can find further inspiration for implementing notification handlers (for example, for media updates) in the .

ExamineComposer

using Examine;
using Umbraco.Cms.Core.Composing;
using Umbraco.Cms.Core.Notifications;
using Umbraco.Cms.Infrastructure.Examine;

namespace Umbraco.Docs.Samples.Web.CustomIndexing;

public class ExamineComposer : IComposer
{
    public void Compose(IUmbracoBuilder builder)
    {
        builder.Services.AddExamineLuceneIndex<ProductIndex, ConfigurationEnabledDirectoryFactory>("ProductIndex");

        builder.Services.ConfigureOptions<ConfigureProductIndexOptions>();

        builder.Services.AddSingleton<ProductIndexValueSetBuilder>();

        builder.Services.AddSingleton<IIndexPopulator, ProductIndexPopulator>();

        builder.AddNotificationHandler<ContentCacheRefresherNotification, ProductIndexingNotificationHandler>();
    }
}

The order of these registrations matters. It is important to register your index with AddExamineLuceneIndex before calling ConfigureOptions.