Wednesday, 5 November 2014

How to fetch more than 5000 entities from CRM

Let's start with simple example to get all entities from CRM

public EntityCollection GetEntities(string entityName)
{
    var config = ServerConnection.GetServerConfiguration();
    var proxy = ServerConnection.GetOrganizationProxy(config);

    string request = string.Format(@"<fetch mapping ='logical'>
               <entity name = '{0}'></entity></fetch>", entityName);
    FetchExpression expression = new FetchExpression(request);
    var mult = proxy.RetrieveMultiple(expression);

    return mult;           
}
This will work, but will return maximum of 5000 elements in mult.Entities.
Let's add paging now and put that in loop until mult.MoreRecords is false.

string request = string.Format(@"<fetch
                count='5000' page='{1}' mapping ='logical'>
                <entity name = '{0}'></entity></fetch>"
, entityName, page++);
But now Paging cookie is required when trying to retrieve a set of records on any high pages.
Paging cookie is returned in EntityCollection.PagingCookie property. Also, XML tags must be encoded, because this is xml-in-xml.

pagingCookie = string.Format("paging-cookie='{0}'",
    System.Web.
HttpUtility.HtmlEncode(mult.PagingCookie));

Now, complete method looks like this:

public IList<Entity> GetEntitiesNoLimit(string entityName)
{
    var config = ServerConnection.GetServerConfiguration();
    IOrganizationService proxy = ServerConnection.GetOrganizationProxy(config);

    var entities = new List<Entity>();
    int page = 1;
    string pagingCookie = string.Empty;
    while (true)
    {
        string request = string.Format(@"<fetch {2} count='5000' page='{1}'
                         mapping ='logical'><entity name = '{0}'></entity></fetch>",
                         entityName, page++, pagingCookie);
        FetchExpression expression = new FetchExpression(request);
        var mult = proxy.RetrieveMultiple(expression);
        entities.AddRange(mult.Entities);
        if (mult.MoreRecords)
        {
            pagingCookie = string.Format("paging-cookie='{0}'",
                        System.Web.HttpUtility.HtmlEncode(mult.PagingCookie));
        }
        else
        {
            break;
        }
    }

    return entities;
}

Tuesday, 4 November 2014