Using SkipToken for Paging in Asp.Net OData and Asp.Net Core OData

Loading large data can be slow. Services often rely on pagination to load the data incrementally to improve the response times and the user experience. Paging can be server-driven or client-driven:

Client-driven paging

In client-driven paging, the client decides how many records it wants to load and asks the server for that many records. That is achieved by using $skip and $top query options in conjunction. For instance, if a client needs to request 10 records from 71-80, it can send a similar request as below:

GET ~/Products/$skip=70&$top=10

However, this is problematic if the data is susceptible to change. In case of a deletion in between two requests for consecutive pages, a record will be served in both the requests.

Server-driven paging

In server-driven paging, the client asks for a collection of entities and the server sends back partial results as well as a nextlink to use to retrieve more results. The nextlink is an opaque link which may use $skiptoken to store state about the request, such as the last read entity.

Default NextLink Generation

Skiptoken is now available with Asp.NetCore OData >= 7.2.0 and Asp.Net OData >= 7.2.0. It can be enabled by calling SkipToken() extension method on HttpConfiguration.

configuration.MaxTop(10).Expand().Filter().OrderBy().SkipToken();

The default implementation of the skiptoken handling is encapsulated by a new class – DefaultSkipTokenHandler. This class implements the abstract methods of the base class SkipTokenHandler, which basically determines the format of the skiptoken value and how that value gets used while applying the SkipTokenQueryOption to the IQueryable.

Format of the nextlink

The nextlink may contain $skiptoken if the result needs to be paginated. In the default implementation, $skiptoken value will be a list of pairs, where the pair consists of a property name and property value separated by a delimiter(:). The orderby property and value pairs will be followed by key property and value pairs in the value for $skiptoken. Each property and value pair will be comma separated.

~/Products?$skiptoken=Id:27
~/Books?$skiptoken=ISBN:978-2-121-87758-1,CopyNumber:11
~/Products?$skiptoken=Id:25&$top=40
~/Products?$orderby=Name&$skiptoken=Name:'KitKat',Id:25&$top=40
~/Cars(id)/Colors?$skip=4

Applying SkipToken Query Option

The skiptoken value is parsed into a dictionary of property name and property-value pairs.  For each pair, we compose a predicate on top of the IQueryable to ensure that the resources returned have greater (or lesser in case of desc orderby) values than the last object encoded in the nextlink.

Custom NextLink Generation

The library provides you with a way to specify your own custom nextlink generation through dependency injection. The code below is delegating the responsibility of handling nextlink to a new class named SkipTopNextLinkGenerator by calling into the container builder extension method in MapODataServiceRoute.

configuration.MaxTop(10).Expand().Filter().OrderBy().SkipToken();

 configuration.MapODataServiceRoute("customskiptoken", "customskiptoken", builder =>
     builder.AddService(ServiceLifetime.Singleton, sp => EdmModel.GetEdmModel(configuration))
            .AddService<IEnumerable<IODataRoutingConvention>>(ServiceLifetime.Singleton, sp =>
               ODataRoutingConventions.CreateDefaultWithAttributeRouting("customskiptoken", configuration))
            .AddService<SkipTokenHandler, SkipTopNextLinkGenerator>(ServiceLifetime.Singleton));

Generating the NextLink

A nextlink can be generated by implementing the GenerateNextPageLink method in a derived class of SkipTokenHandler. The instance passed to this method will be the last object being serialized in the collection.

/// <summary>
        /// Returns the URI for NextPageLink
        /// </summary>
        /// <param name="baseUri">BaseUri for nextlink.</param>
        /// <param name="pageSize">Maximum number of records in the set of partial results for a resource.</param>
        /// <param name="instance">Instance based on which SkipToken value will be generated.</param>
        /// <param name="context">Serializer context</param>
        /// <returns>URI for the NextPageLink.</returns>
        public abstract Uri GenerateNextPageLink(Uri baseUri, int pageSize, Object instance, ODataSerializerContext context);

However, if your paging strategy does not align with this approach for all your use cases. You can set the nextlink in your controller method by returning a paged result. This will override the nextlink that is generated by your implementation of the SkipTokenHandler.

return new PageResult<Product>(https://myservice/odata/Entity?$skiptoken=myValue,
        results as IEnumerable<Product>, 
        yourCustomNextLink, 
        inLineCount);

Applying the SkipToken

In your custom nextlink generation, you are free to use the skiptoken in the nextlink to encode additional information that you may require. However, you will also have to implement how to use the SkipToken query option by implementing your own ApplyTo methods.

/// <summary>
        /// Apply the $skiptoken query to the given IQueryable.
        /// </summary>
        /// <param name="query">The original <see cref="IQueryable"/>.</param>
        /// <param name="skipTokenQueryOption">The query option that contains all the relevant information for applying skiptoken.</param>
        /// <returns>The new <see cref="IQueryable"/> after the skiptoken query has been applied to.</returns>
        public abstract IQueryable<T> ApplyTo<T>(IQueryable<T> query, SkipTokenQueryOption skipTokenQueryOption);

To implement your own ApplyTo, it may be useful to look at the DefaultSkipTokenHandler’s implementation.

The post Using SkipToken for Paging in Asp.Net OData and Asp.Net Core OData appeared first on OData.