Chapter 6. LINQ to SharePoint

When you need to develop server-side solutions, one of the most significant features of Microsoft SharePoint 2013 is its support for LINQ (Language-Integrated Query). Supported since SharePoint 2010, LINQ is a satisfying alternative to the classic object model discussed in Chapter 5 This chapter will begin with a quick overview of LINQ, just in case you’re not familiar with it, and then show you how to work with it. Next, you’ll learn about LINQ to SharePoint as a LINQ query provider implementation, which is useful for querying and managing items in SharePoint lists using the LINQ data access model. If you already know about LINQ, you can skip the next section and move directly to the section titled “Introducing LINQ to SharePoint.”

More Info

To learn more about LINQ, read Programming Microsoft LINQ in .NET 4.0, by Paolo Pialorsi and Marco Russo (Microsoft Press, 2010).

LINQ overview

LINQ is a programming model that introduces queries as a first-class concept into any Microsoft .NET language. Complete support for LINQ, however, requires some extensions in the language that you are using. These extensions boost developer productivity, thereby providing a shorter, more meaningful, and expressive syntax with which to manipulate data.

LINQ provides a methodology that simplifies and unifies the implementation of any kind of data access. LINQ does not force you to use a specific architecture; it facilitates the implementation of several existing architectures for accessing data, such as the following:

  • RAD/prototype

  • Client/server

  • N-tier

  • Smart client

The architecture of LINQ is based on the idea of having a set of LINQ providers, each able to target a different kind of data source. Figure 6-1 shows a schema of the main LINQ providers available in .NET Framework 4.5. Aside from SharePoint and out of the box with .NET Framework 4.5 and Microsoft Visual Studio 2012, LINQ includes many providers suitable for accessing several different types of data sources, including the following:

  • LINQ to Objects. This is used to query in-memory data and object graphs.

  • LINQ to SQL. This was specifically designed to query and manage data stored in a Microsoft SQL Server database, using a lightweight, simplified object-relational mapper (O/RM) that maps entities to tables with a one-to-one relationship. LINQ to SQL can be considered a discontinued library. Nevertheless, you can still use it.

  • LINQ to Entities. The first-class O/RM offered by Microsoft to design solutions based on the domain model, with a real abstraction from the underlying persistence storage. LINQ to Entities is based on the Entity Framework. The .NET Framework 4.5 ships with the Entity Framework version 5.

  • LINQ to DataSet. This is a LINQ implementation targeting old-style ADO.NET DataSet and DataTable types. It is mainly offered for backward compatibility reasons.

  • LINQ to XML. This is a LINQ implementation targeting XML contents, useful to query, manage and navigate across XML nodes.

In your SharePoint solutions, you can use any of these LINQ providers, as long as you access objects, SQL Server, DBMSs, DataSet objects, or XML. If you need to access SharePoint data, however, you cannot use any of these providers; but read on for suggested solutions.

A diagram listing the .NET Framework 4.5 LINQ providers for Objects in memory, DataSets, Microsoft SQL Server, Entity Framework 5, and XML. These are positioned to pass data between C# or Visual Basic (shown on the top level of the diagram) and objects, graphs, relational data, and XML (shown on the bottom).

Figure 6-1. A graphical representation of the main LINQ providers available in .NET Framework 4.5.

The goal of LINQ

The goal of LINQ is to provide a set of tools that improve code implementation by adapting to several different architectures, rather than changing application architectures.

Today, data managed by a program can originate from many and varied data sources, such as an array, an object graph, an XML document, a database, a text file, a registry key, an e-mail message, Simple Object Access Protocol (SOAP) message content, a Microsoft Office Excel file, and so forth. The list is extensive. LINQ makes it easier to access all these various kinds of data, providing a unified programming model. In fact, each data source has its own specific data-access model. When you need to query a database, you typically use SQL. You navigate XML data by using the Document Object Model (DOM) or XPath/XQuery. You iterate an array and build algorithms to navigate an object graph. You use specific application programming interfaces (APIs) to access other data sources, such as an Excel file, an e-mail message, or the Windows registry. Put briefly, you use different programming models to access different data sources.

The unification of data access techniques into a single comprehensive model has been attempted in many ways. For example, Open Database Connectivity (ODBC) providers allow you to query an Excel file as you would a Windows Management Instrumentation (WMI) repository. With ODBC, you use an SQL-like language to access data represented through a relational model. Sometimes, however, data is represented more effectively in a hierarchical or network model instead of a relational one. Moreover, if a data model is not tied to a specific language, you probably need to manage different type systems. All these differences create an “impedance mismatch” between data and code.

LINQ addresses these issues by offering a uniform method to access and manage data without forcing the adoption of a “one size fits all” model. LINQ makes use of common capabilities in the operations in different data models instead of flattening the different structures between them. In other words, by using LINQ, you keep existing heterogeneous data structures, such as classes or tables, but you gain a uniform syntax to query all these data types, regardless of their physical representation. Think about the differences between a graph of in-memory objects and relational tables with proper relationships. With LINQ, you can use the same query syntax on both models.

Here is a simple LINQ query for a typical software solution that returns the names of customers in Italy:

var query =
    from   c in Customers
    where  c.Country == "Italy"
    select c.CompanyName;

The result of this query is a list of strings. You can enumerate these values with a foreach loop in C#:

foreach ( string name in query ) {
    Console.WriteLine( name );
}

Both the query definition and the foreach loop are regular C# statements, valid for C# 3.0 or higher, but what is Customers? At this point, you might be wondering what it is we are querying. Is this query a new form of Embedded SQL? Not at all. You can apply the same query (and the foreach loop) to the following:

  • An SQL database using LINQ to SQL

  • A third-party DBMS using LINQ to Entities

  • A DataSet object using LINQ to DataSet

  • An array of objects in memory using LINQ to Objects

  • A remote service

In fact, you can apply the query to many other kinds of data, as well, so long as you use each kind’s specific LINQ provider. For example, Customers could be a collection of objects:

Customer[] Customers;

Customers also could be an entity class that describes a physical table in a relational database:

DataContext db = new DataContext( ConnectionString );
Table<Customer> Customers = db.GetTable<Customer>();

Or, Customers could be an entity class that describes a conceptual model mapped to a relational database:

NorthwindModel dataModel = new NorthwindModel();
ObjectSet<Customer> Customers = dataModel.Customers;

And in SharePoint 2013, Customers could be an entity class that describes a collection of SPListItem types retrieved from an SPList of customers stored in SharePoint:

MySiteContext sp = new MySiteContext ( siteUri );
EntityList<Customer> Customers = sp.GetList<Customer>("Customers");

These examples highlight that the main goal of LINQ is to provide a unified querying and programming model—fully integrated with programming languages—that abstracts code from the underlying infrastructure.

LINQ under the hood

Now you know that a LINQ query can target any kind of data source supported by a LINQ provider. But how does LINQ work? This section gives you a tour of what’s under its hood.

Suppose you write the following code that uses LINQ:

Customer[] Customers = GetCustomers();
var query =
    from   c in Customers
    where  c.Country == "Italy"
    select c;

From that query, the compiler generates this code:

Customer[] Customers = GetCustomers();
IEnumerable<Customer> query =
        Customers
        .Where( c => c.Country == "Italy" );

When the query becomes more complex, as you can see in the next example…

Note

From now on, the examples will skip the Customers declaration for the sake of brevity.

var query =
    from    c in Customers
    where   c.Country == "Italy"
    orderby c.Name
    select  new { c.Name, c.City };

The generated code is more complex, too:

var query =
        Customers
        .Where( c => c.Country == "Italy" )
        .OrderBy( c => c.Name )
        .Select( c => new { c.Name, c.City } );

The code calls instance members on the object returned from the previous call. It calls Where on Customers; OrderBy on the object returned by Where; and finally, Select on the object returned by OrderBy. This behavior is regulated by what are known as extension methods in the host language (C# in this case). The implementation of the Where, OrderBy, and Select methods—called by the sample query—depends on the type of Customers and on namespaces specified in relevant using statements. Extension methods are a fundamental syntax feature used by LINQ so that it can maintain the same syntax across different data sources.

The basic concept behind LINQ is that queries target objects that implement either the IEnumerable<T> interface for in-memory data or the IQueryable<T> interface for data retrieved from an external store. Here’s the definition of the IEnumerable<T> interface:

public interface IEnumerable<T> : IEnumerable {
    IEnumerator<T> GetEnumerator();
}

And here’s the definition of the IQueryable<T> interface, together with its base interface, IQueryable:

public interface IQueryable<T> : IEnumerable<T>, IQueryable, IEnumerable {
}
public interface IQueryable : IEnumerable {
    Type ElementType { get; }
    Expression Expression { get; }
    IQueryProvider Provider { get; }
}

Whenever you browse for (enumerate) the results of a query—for example, by using a foreach statement—the compiler invokes the GetEnumerator method of the IEnumerable<T> interface, and at that point the query is effectively executed.

When the target object of your query implements only the IEnumerable<T> interface, the extension methods targeting that type will work against in-memory objects. For example, LINQ to Objects and LINQ to XML both work in this way.

However, when the query target object implements IQueryable<T>, the extension methods construct an expression tree, which describes the query from a provider-independent point of view. The expression tree is then processed by the IQueryable implementation of the query target object, invoking the IQueryProvider object published by the IQueryable.Provider property. The query provider visits the expression tree, using an expression visitor, and produces a query syntax that targets the concrete persistence storage.

For example, as shown in Figure 6-2, for a LINQ to SQL query engine, the query provider will generate a T-SQL query that corresponds to the LINQ query you defined in your .NET code. Similarly, when using LINQ to SharePoint, the query provider generates a CAML (Collaborative Application Markup Language) query that will be executed against the target SPList using the standard Server Object Model querying syntax.

A diagram that illustrates how the same LINQ query can produce different results, depending on the target LINQ query provider. In the depicted example, LINQ to SQL produces a T-SQL query targeting a SQL Server instance, whereas LINQ to SharePoint generates the CAML needed to query a target SharePoint list.

Figure 6-2. A graphical representation of how LINQ providers work.

Note

CAML is an XML-based querying language that is useful for retrieving, manipulating, sorting, and grouping SharePoint data.

Introducing LINQ to SharePoint

With a better understanding of LINQ and generally how it works, you can dive into LINQ to SharePoint, which is just another LINQ query provider that targets SharePoint data. Figure 6-3 shows the data access model architecture of SharePoint 2013, illustrating the role of LINQ to SharePoint compared to other data access technologies available in SharePoint 2013.

A diagram illustrating the relationships among all the available options in the SharePoint 2013 data access model architecture. On the server side are the Server Object Model, CAML, and LINQ to SharePoint. On the client side are the Client-Side Object Model, REST/OData, the Web API, and mobile support.

Figure 6-3. The SharePoint 2013 data access model architecture and the role of LINQ to SharePoint.

The key feature of LINQ to SharePoint is that it can query SharePoint data with a fully typed approach, using a common querying language (LINQ) and retrieving typed entities.

Modeling with SPMetal.exe

The first and main task when developing solutions that make use of LINQ to SharePoint is to model the typed entities. You can define these manually, but it is generally more useful to use SPMetal.exe, which can automatically generate entities for you. You can find the SPMetal.exe utility in the SharePoint15_RootBIN folder. SPMetal.exe is a command-line tool that accepts the wide range of arguments listed in Table 6-1.

Table 6-1. Arguments that you can provide to SPMetal.exe

Argument

Description

/web:<url>

Specifies the absolute URL of the target website. The host address can be local, in which case, the tool uses the Server Object Model to connect to the server.

/useremoteapi

Specifies that the website URL is remote. You might not use this option if any of the lists on the website contain lookup fields. Secondary lookups are not supported by the Client Object Model.

/user:<name>

Specifies the logon username (or domain).

/password:<password>

Specifies the logon password.

/parameters:<file>

Specifies an XML file with code generation parameters.

/code:<file>

Specifies the output location for generated code (default: console).

/language:<language>

Specifies the source code language. Valid options are csharp and vb (default: inferred from source code file name extension).

/namespace:<namespace>

Specifies a namespace used for autogenerated code (default: no namespace).

/serialization:<type>

Specifies a serialization type. Valid options are none and unidirectional (default: none). The entities serialization topic will be discussed in the “Disconnected entities” section.

Note that the default behavior of SPMetal.exe is to output autogenerated code to the console. That’s not terribly useful except for testing, so you should generally provide a /code argument to instruct the tool to generate a code file instead. The resulting code file should be included in your Visual Studio project manually. Next, you need to provide the target website URL by using the /web argument, and then instruct the tool to use the Client Object Model (/useremoteapi) if the site is remote. It’s common to also provide a namespace by using the /namespace argument to make the generated code part of the same namespace of your target project. Here’s a typical command-line invocation of the tool:

SPMETAL.EXE /web:http://devbook.sp2013.local /code:devbook.cs /namespace:DevLeap.SP2013.Linq2SP

By default, SPMetal.exe creates a full model for the target site, defining a class for almost every supported content type and a list for every list instance, except for hidden lists. The tool will also create a class named {WebSiteName}DataContext, where {WebSiteName} is the name of the target website (without spaces, in case the site name has spaces in its content). This class represents the entry point for using LINQ to SharePoint, and it inherits from the Microsoft.SharePoint.Linq.DataContext base class.

Quite often, you do not really need to model each and every content type and list instance of the target site. Usually, you need to model only some custom data structures that you plan to query and manage with LINQ to SharePoint. The /parameters command-line argument is provided for this purpose. In fact, by using this argument, you can provide SPMetal.exe with an XML file that instructs the tool about what to skip and what to include in the autogenerated model. A sample XML parameters file suitable for SPMetal.exe shows a sample XML parameters file that excludes all the common team site default contents, but includes all other lists and content types. Notice that you cannot use both an ExcludeList and a List element targeting the same list.

A sample XML parameters file suitable for SPMetal.exe shows that the XML file is based on a custom XML namespace. Table 6-2 describes the supported elements that you can use to define such a file.

Table 6-2. The elements available for defining an XML parameters file

Element name

Description and purpose

Web

The root element of the schema. This tag defines the name of the DataContext generated class, configuring the class attribute. It also defines the access modifier used for autogenerated types. By default, SPMetal.exe uses a public access modifier.

List

Instructs SPMetal.exe to include a specified list definition. It is useful for including hidden lists. It also allows you to override the name of the list in the autogenerated code. The tag requires a Name attribute whose value is the list name.

ExcludeList

Excludes the generation of the specified target list from the autogenerated code. This tag requires a Name attribute whose value is the list name.

ExcludeOtherLists

Instructs SPMetal.exe to avoid generating any list definition except those that you explicitly define using a List element.

IncludeHiddenLists

Requests that SPMetal.exe generate list definitions for hidden lists. You cannot use this element together with the ExcludeOtherLists element.

ContentType

Forces SPMetal.exe to generate the code for a specific content type, referenced by Name, using a specific attribute. You can use this tag, for example, to include a hidden content type. This element can be a child of the Web or List elements.

Column

Instructs SPMetal.exe to output a property representing a field (site column) that it would not model by default. For example, you can use this element to include a hidden field. It requires a Name attribute, whose value is the name of the field to include.

ExcludeColumn

Excludes a field from code generation. This element requires a Name attribute, whose value is the name of the field to include.

ExcludeOtherColumns

Configures the tool to block code generation for columns that are not explicitly referenced by a Column element.

IncludeHiddenColumns

Causes SPMetal.exe to generate code for hidden column. This element cannot be used together with the ExcludeOtherColumns element.

ExcludeContentType

Blocks code generation for the content type specified by the value of a Name attribute. This element can be a child of Web or List elements.

ExcludeOtherContentTypes

Configures SPMetal.exe to block code generation for SharePoint apps not explicitly referenced by a ContentType element.

IncludeHiddenContentTypes

Requests that SPMetal.exe also generate code for any hidden content type. This element cannot be used together with the ExcludeOtherContentTypes element.

Now suppose that you have a website with a couple of custom lists: a standard document library named Invoices and a custom list of items named DevLeap Contacts in which each item can be of type DevLeapCustomer or DevLeapSupplier. Both types share a base content type called DevLeapContact.

Note

To provision these content types and lists, see the code samples in Chapter 3

Sample XML parameters file for SPMetal.exe shows another XML parameters file that includes these custom content types and lists and excludes all other content types and lists.

Assume you’ve executed this with SPMetal.exe; now you’ll examine the resulting autogenerated code. First, you have a DevbookDataContext class that provides entry points to access the content lists of the target site. The DevbookDataContext class, autogenerated using the XML parameters file from Sample XML parameters file for SPMetal.exe shows the definition of this DataContext-inherited class.

The class has a constructor that accepts the URL of the target website as its only argument. Internally, it invokes a partial method (OnCreated), which you can use to customize the context initialization. Next, there are a couple of public properties that correspond to the two modeled lists (Invoices and DevLeap Contacts). It is interesting to see that both of these properties are decorated with the ListAttribute attribute, stating the name of the underlying SharePoint list. Also, both of these properties are of type EntityList<T>, which is the type LINQ to SharePoint uses to represent a collection of typed items.

Internally, these properties invoke the DataContext.GetList<T> method. If you have any experience with LINQ to SQL, you will find many similarities between LINQ to SharePoint and LINQ to SQL. The Invoices list is made up of a set of Document instances, where Document is the typed entity autogenerated by SPMetal.exe that describes a SharePoint document from a conceptual viewpoint. The DevLeapContacts list is composed of items of type DevLeapContact, which is the typed entity corresponding to the base content type DevLeapContact.

One last thing to consider about the DataContext type is that it implements IDisposable, because internally it uses some types that exploit unmanaged resources such as the SPSite and SPWeb types. Therefore, you should always call Dispose whenever you create an instance.

More Info

See the “Resource disposal” section in Chapter 5 to better understand the reasons for disposing of unmanaged resources.

Figure 6-4 shows the class diagram of the generated types.

A diagram illustrating the inheritance of types generated by SPMetal.exe. The root type is Item. The DevLeapContact type is inherited from Item. Both the DevLeapCustomer and DevLeapSupplier types inherit from the DevLeapContact type. The Document class inherits from the Item class and the DevLeapInvoice class inherits from Document.

Figure 6-4. Class diagram of typed entities generated by SPMetal.exe.

Specifically, Figure 6-4 shows that the SPMetal.exe tool generated an Item base class, which internally implements some infrastructural interfaces for data management tracking (ITrackEntityState, ITrackOriginalValues) and for data binding (INotifyPropertyChanged, INotifyPropertyChanging), as well as some properties that correspond to the common data of every SharePoint list item (Id, Path, Title, and Version). The Document entity inherits from Item and adds some document-specific properties (DocumentCreatedBy, DocumentModifiedBy, and Name). The most interesting part of the model is how it defines entities that map to custom content types. In fact, SPMetal.exe modeled a DevLeapContact class, which inherits from Item and is the base class for the types DevLeapCustomer and DevLeapSupplier. This is challenging behavior; SPMetal.exe modeled the content types and lists of SharePoint, mapping them to an object-oriented model of entities, with full inheritance support.

Important

Because of the need for a set of typed entities that model the content types defined in the target SharePoint site, it is good practice to use LINQ to SharePoint only against sites that have a well-defined and stable structure. To learn how to correctly provision data structures in SharePoint, see Chapter 3. Similarly, it is not as useful to use LINQ to SharePoint on sites that frequently change their structure, because you would need to refresh the typed model frequently as well. Instead, you should access and query sites with a high change frequency using the standard Server Object Model and the untyped approach, eventually using CAML at a low level.

The Item type code autogenerated by SPMetal.exe shows a portion of the code corresponding to the base Item type.

It is interesting to see the class attribute decorations—which are specific for LINQ to SharePoint—that instruct the engine about the content type ID (ID=0x01) behind the Item class, as well as about the types that inherit from this base class. You can see that the base Item type, and thus every typed entity in the model, provides an EntityState property related to the ITrackEntityState interface implementation, and an OriginalValues property of type Dictionary, related to the ITrackOriginalValues interface implementation. You’ll see these properties used for tracking entities’ states and changes in the “Managing data” section later in the chapter. In addition, the entity offers two public properties useful for accessing the current item ID and title. These properties are marked with the ColumnAttribute attribute, which defines the underlying storage field and the corresponding SharePoint column. Lastly, the class provides three partial methods that you can implement to add custom behaviors to the type when loading (OnLoaded), validating (OnValidate), and creating (OnCreated) a type instance.

Starting from this base type, the tool arranges inheritance for all the entity types corresponding to the content types. The autogenerated code for the custom types contains an excerpt of the DevLeapContact, DevLeapCustomer, and DevLeapSupplier types.

The autogenerated code for the custom types shows that the classes are fully connected with the original SharePoint types because each class refers to its corresponding content type using its ID, just as the Item base type did in The Item type code autogenerated by SPMetal.exe. Additionally, when you have a Choice field on the SharePoint side (for example, DevLeapContact.Country and DevLeapCustomer.CustomerLevel), the tool generates an enum type, giving you strongly typed access to the choice values.

Of course, you could write all this code manually and get the same results, but that’s not recommended because you would be wasting your time.

Querying data

Now that you have seen how to model your data with SPMetal.exe, and what the model is, you can start querying the site for content. The key feature of this new query provider lies in its ability to query SharePoint content using LINQ queries. As an example, A code excerpt using a LINQ to SharePoint query to find documents in the Invoices list created by a specific user contains a code excerpt with a query that fetches the titles of documents in the Invoices library with the Title property starting with a specific word.

Note

To execute the custom code illustrated in this section and in those that follow, you need to reference the Microsoft.SharePoint.Linq.dll assembly, which is available in the SharePoint15_RootISAPI folder of every SharePoint server. In addition, you should declare a couple of using statements for the namespaces Microsoft.SharePoint.Linq and System.Linq in your code.

A code excerpt using a LINQ to SharePoint query to find documents in the Invoices list created by a specific user creates a new instance of the DataContext class, passing in the URL of the target site. The target site can be the URL of any SharePoint site with a data structure that is compatible with the site from which you generated the model. Of course, in real code, the URL should not be hard coded, and you should refer to a configurable parameter. It also employs the using keyword to dispose of unmanaged resources expediently. Then it simply queries the Invoices collection provided by the current context, just as with any other LINQ query. Under the hood, the query engine creates a CAML query and sends it to the Invoices list using an SPQuery instance, invoking the SPList.GetItems method. If you want to see the autogenerated CAML query, you can set the Log property of the DataContext instance to a TextWriter object (for example, Console.Out if you are working with a Console application). Here’s the syntax:

spContext.Log = Console.Out;

And here’s the CAML code generated for the query in A code excerpt using a LINQ to SharePoint query to find documents in the Invoices list created by a specific user:

<View>
  <Query>
    <Where>
      <And>
        <BeginsWith><FieldRef Name="ContentTypeId" />
          <Value Type="ContentTypeId">0x0101</Value>
        </BeginsWith>
        <BeginsWith>
          <FieldRef Name="Title"/><Value Type="Text">Invoice</Value> 
        </BeginsWith>
      </And>
    </Where>
  </Query>
  <ViewFields>
    <FieldRef Name="Title" />
  </ViewFields>
  <RowLimit Paged="TRUE">2147483647</RowLimit>
</View>

The LINQ to SharePoint query engine allows you to define many kinds of queries, with partitioning (where), projection (select), and under some circumstances, relationships (join). Imagine that the Invoices list of documents is made of a custom content type named DevLeapInvoice that has a lookup field that accepts a DevLeapContact object from the DevLeap Contacts custom list. If you refresh the model (via SPMetal.exe) after adding such a lookup field in the Invoices list, you will see a new class that inherits from the original Document type, as shown in The definition of the DevLeapInvoice type.

This new type has a property named DevLeapContact, of type DevLeapContact, which internally works with a private storage field of type EntityRef<DevLeapContact>. In addition, the type constructor automatically creates an instance of that field and registers some event handlers to manage the synchronization of the association between the DevLeapInvoice object and its corresponding DevLeapContact instance.

On the other side, the DevLeapContact type has been changed, too. In fact, now it supports a public property of type Microsoft.SharePoint.Linq.EntitySet<DevLeapInvoice>, which represents a reference to all the invoices for the current contact.

Now comes the nice part of the story: you can define a LINQ query that joins these entities. In addition, you can use deferred loading of entities when dynamically browsing related items. Deferred loading allows you to dynamically load data related to the entities you are querying, whenever you need them and without having to explicitly and manually load them. In fact, the LINQ to SharePoint provider will take care of that for you. A LINQ to SharePoint query that uses a join between contacts and invoices shows a code excerpt of a sample query with a join syntax.

The output of this query will be a set of new anonymous types that expose the properties ContactID, Title, and InvoiceTitle. The CAML query sent to the SharePoint is as follows:

<View>
  <Query>
    <Where>
      <And>
        <BeginsWith>
          <FieldRef Name="ContentTypeId" />
          <Value Type="ContentTypeId">0x0101000B231F0B244C41F59EB1467059EA59E8</Value>
        </BeginsWith>
        <BeginsWith>
          <FieldRef Name="DevLeap_x0020_ContactContentTypeId" />
          <Value Type="Lookup">0x010025836A76187A4B49892A35CB80CC5232</Value>
        </BeginsWith>
      </And>
    </Where>
    <OrderBy Override="TRUE" />
  </Query>
  <ViewFields>
    <FieldRef Name="DevLeap_x0020_ContactDevLeapContactID" />
    <FieldRef Name="DevLeap_x0020_ContactTitle" />
    <FieldRef Name="Title" />
  </ViewFields>
  <ProjectedFields>
    <Field Name="DevLeap_x0020_ContactDevLeapContactID" Type="Lookup"
        List="DevLeap_x0020_Contact" ShowField="DevLeapContactID" />
    <Field Name="DevLeap_x0020_ContactTitle" Type="Lookup"
        List="DevLeap_x0020_Contact" ShowField="Title" />
    <Field Name="DevLeap_x0020_ContactContentTypeId" Type="Lookup"
        List="DevLeap_x0020_Contact" ShowField="ContentTypeId" />
  </ProjectedFields>
  <Joins>
    <Join Type="INNER" ListAlias="DevLeap_x0020_Contact">
      <!--List Name: DevLeap Contacts-->
      <Eq>
        <FieldRef Name="DevLeap_x0020_Contact" RefType="ID" />
        <FieldRef List="DevLeap_x0020_Contact" Name="ID" />
      </Eq>
    </Join>
  </Joins>
  <RowLimit Paged="TRUE">2147483647</RowLimit>
</View>

Notice the elements ProjectedFields and Joins in the CAML code. In A LINQ to SharePoint Query using deferred loading code excerpt illustrates deferred loading in action.

In A LINQ to SharePoint Query using deferred loading, the first LINQ query is converted into CAML and executed against SharePoint within the first and external foreach block. Then, when the inner foreach block browses for the InvoicesDocument collection of the current contact, the LINQ to SharePoint engine automatically executes a CAML query to retrieve the invoices belonging to the current contact. This is the default behavior, which you can change by setting the DeferredLoadingEnabled property of the DataContext to false, as shown here:

spContext.DeferredLoadingEnabled = false;

If you’re familiar with LINQ, you probably use hierarchical grouped queries, making use of the join into (also known as group join) clause, which avoids the need to execute a separate query to retrieve the invoices for every single contact. However, the LINQ to SharePoint query provider has limitations due to its use of CAML queries under the covers. For example, with CAML, you cannot query more than one list at a time, so you can’t use a group join. An unsupported LINQ to SharePoint query syntax presents a code excerpt that declares an unsupported group join query.

When you try to execute a query like this, the LINQ to SharePoint query provider throws an exception similar to this:

Unhandled Exception: System.InvalidOperationException: The query uses unsupported elements,
such as references to more than one list, or the projection of a complete entity by usingEntityRef/EntitySet.

Moreover, LINQ to SharePoint does not support multifetch queries that query across multiple lists or join clauses on fields other than Lookup fields. For multifetch queries, you can consider using specific controls instead, such as the Content by Query Web Part or the SPSiteDataQuery class, which allows querying multiple lists using CAML queries. Also, you cannot define queries across multiple websites or that query different DataContext instances. Finally, you cannot use mathematical functions because CAML does not support them. Overall, LINQ to SharePoint does not support queries that cannot be translated into CAML syntax.

More Info

For a complete list of unsupported syntax and commands, please refer to the article “Unsupported LINQ Queries and Two-stage Queries,” on MSDN, at http://msdn.microsoft.com/en-us/library/ee536585.aspx.

Managing data

The previous section showed that LINQ to SharePoint provides a convenient syntax for executing CAML queries with a fully typed approach. Even if this is sufficient for your needs, the story becomes more interesting when you consider that LINQ to SharePoint gives you access to data using a kind of SharePoint-specific O/RM, meaning you can also manage (insert, update, delete) data using LINQ to SharePoint, and it’s a fully typed approach.

Here’s a quick initial example. The code in Using LINQ to SharePoint to change an entity queries for a specific contact in the DevLeap Contacts list, using a LINQ to SharePoint query, and then changes the Country property of the retrieved item.

As Using LINQ to SharePoint to change an entity demonstrates, the process is both simple and intuitive. You just need to retrieve the object, change its properties, and then confirm the changes by invoking the SubmitChanges method of the DataContext. You should consider SubmitChanges the counterpart of the Update method in the standard Server Object Model. In fact, just as with the Server Object Model, whenever you change an instance of an entity that models an item in a SharePoint list, you are changing the in-memory copy of that data, not the SharePoint content database. Behind the scenes, the LINQ to SharePoint engine tracks this change, so you can apply it on the real content database when you invoke the DataContext.SubmitChanges method.

Internally, the DataContext base class provides an object tracker (an internal EntityTracker class) that tracks any changes you make to in-memory copies of typed entities. Furthermore, as you have already seen in the previous section, the base Item class that every LINQ to SharePoint entity inherits implements the ITrackEntityState interface, which provides an EntityState property that can assume one of the following values:

  • Unchanged. The entity has not been changed.

  • ToBeInserted. The entity is new and will be inserted into its parent list when you call SubmitChanges.

  • ToBeUpdated. The entity has been changed and will be updated in the content database when you call SubmitChanges.

  • ToBeDeleted. The entity has been deleted and will be permanently removed from the content database when you call SubmitChanges.

  • ToBeRecycled. The entity has been deleted and will be moved to the Recycle Bin when you call SubmitChanges.

  • Deleted. The entity has been deleted or recycled.

For example, if you test the EntityState property of the contact in Using LINQ to SharePoint to change an entity, you will see that the entity is in the Unchanged state just after retrieval. As soon as you change the Country property, its state becomes ToBeUpdated. Finally, just after you invoke the SubmitChanges method, the state returns to Unchanged, because the entity has been synchronized with the content database.

This tracking behavior is provided transparently by default whenever you create a DataContext instance and retrieve modeled entities. Note that tracking does not work on anonymous types, which are types that you get through LINQ queries that use custom projection. For example, the code illustrated in A LINQ to SharePoint query that uses a join between contacts and invoices uses projection to extract an anonymous type made only of ContactID, Title, and InvoiceTitle properties. The type resulting from that query will be read-only and thus will not have tracking support.

The tracking behavior, however, has an impact on performance and resource consumption. Therefore, if you don’t need to manage data (such as when you need to query and render contents in a read-only fashion), you can disable the entity tracking service by setting the ObjectTrackingEnabled property of the DataContext class to false:

spContext.ObjectTrackingEnabled = false;

In the next few pages, you will see how to manage data, taking advantage of the LINQ to SharePoint tracking engine through some concrete examples. You’ve already seen an example of updating an item in Using LINQ to SharePoint to change an entity, so that operation will not be repeated.

Inserting a new item

To insert a new item into a list, you first create the item instance, just as you would with any .NET object. Next, you need to configure its properties, and finally, you need to add the new item to its parent list and submit changes to the content database. The code in Inserting a new item in a list using LINQ to SharePoint illustrates this process.

The key point of this example, aside from the SubmitChanges method invocation that you have already seen, is the call to the InsertOnSubmit method of the EntityList<T> class that lies behind the DevLeapContacts property of the DataContext. The InsertOnSubmit method accepts an item to be inserted into the target list as soon as you invoke SubmitChanges. The entity passed to the method will acquire a state of ToBeInserted. Note that the InsertOnSubmit method is fully typed according to the generic type T of the EntityList<T> class. Thus, in Inserting a new item in a list using LINQ to SharePoint you can invoke this method by providing a class of type DevLeapContact or any type inherited from DevLeapContact, such as DevLeapCustomer or DevLeapSupplier.

The EntityList<T> class also provides an InsertAllOnSubmit method, which lets you insert a group of entities instead of a single entity. This last method requires an argument of type IEnumerable<T>, representing the collection of items to insert.

Deleting or recycling an existing item

Deleting an item is much like inserting a new item. The EntityList<T> class provides a DeleteOnSubmit method, as well as a DeleteAllOnSubmit method, similar to the methods presented in the preceding section. The former accepts a single item to delete, whereas the latter accepts a collection of type IEnumerable<T>, representing the items to delete. Both of these methods permanently delete the target items from the content database when you confirm the action by invoking SubmitChanges. SharePoint provides a Recycle Bin feature, so the EntityList<T> class also provides a couple of methods specifically intended to move items into the Recycle Bin, instead of permanently deleting them. These methods are RecycleOnSubmit and RecycleAllOnSubmit. Deleting or recycling an item from a list using LINQ to SharePoint shows a code excerpt that illustrates how to delete or recycle an item.

Advanced topics

In this section, you’ll see some more advanced topics about using LINQ to SharePoint. These topics include managing concurrency conflicts, working with the identity management services, handling disconnected entities, supporting versioning, and extending the entity model.

Handling concurrency conflicts

Whenever you have a data management infrastructure that works when disconnected from the source repository, you will inevitably face concurrency conflicts. In fact, every single time you insert, update, or delete/recycle any data, you are working with an in-memory copy of the content; therefore, you have no guarantee that your changes will be effectively confirmed by the back-end store when you invoke SubmitChanges. For example, when you retrieve an item from a list to change its properties, someone else might change that same item concurrently. Moreover, your code can be executed multiple times concurrently when you are in a high-traffic solution. Thus, when you try to apply your changes to the back-end repository, it will throw a concurrency conflict exception.

Fortunately, LINQ to SharePoint has established and complete support for concurrency conflicts. In fact, the SubmitChanges method has three overloads:

public void SubmitChanges();
public void SubmitChanges(ConflictMode failureMode);
public void SubmitChanges(ConflictMode failureMode, bool systemUpdate);

At this point, the first overload should be familiar (you have seen it in many of the previous code listings). Both the second and the third overloads accept an argument of type ConflictMode, which is an enum defined in the following excerpt:

public enum ConflictMode {
    ContinueOnConflict,
    FailOnFirstConflict
}

The names of the available values reveal their purposes:

  • ContinueOnConflict. When any concurrency conflict occurs, the DataContext object will skip the conflicting items, but it will continue to submit changes for all nonconflicting items. When the SubmitChanges method completes with conflicts, it throws a ChangeConflictException, so you will have the opportunity to evaluate conflicts and decide what to do.

  • FailOnFirstConflict. This stops processing the SubmitChanges method as soon as any concurrency conflict occurs. This overload also throws a ChangeConflictException so that you can evaluate the conflict and decide what to do. Any modifications submitted before the first conflict will be persisted to the content database.

Note

The third overload also accepts a Boolean argument with the name systemUpdate, which is not directly related to handling concurrency conflicts, but simply allows you to update the content database without incrementing the version number of the changed items. By default, the SubmitChanges overload without arguments uses a ConflictMode argument with a value of FailOnFirstConflict and a systemUpdate argument with a value of false.

When you submit changes to the content database and a change conflict occurs, you can catch a ChangeConflictException, which contains a description tightly bound to SharePoint’s typical web scenario. As an example, here’s the Message property for a concurrency conflict exception:

Database values have been updated since the client last read them.

To solve conflicts, you can browse the ChangeConflicts property of the DataContext class instance. This property is a collection of objects of type ObjectChangeConflict, which you can enumerate to inspect all conflicting items. Every ObjectChangeConflict instance exposes a property named Object, of type System.Object, that references the current conflicting item. You can cast that property to the real target entity instance. In addition, you can inspect the conflicting members of the current conflicting item by enumerating the MemberConflicts property of every ObjectChangeConflict instance. Finally, each element of the MemberConflicts collection is of type MemberChangeConflict and provides you with some detailed information about the member conflict. For example, you can see the member name and type, the original value of the member when you retrieved the entity from the SharePoint content database, the current value in memory, and the actual value in the content database.

With that information, to solve concurrency issues, you need to invoke the Resolve method, which has several overloads for both ObjectChangeConflict and MemberChangeConflict values. In essence, the Resolve method lets you determine which values win—those of the current user or those in the content database (the other concurrent user).

Here are the overloads for the Resolve method of the ObjectChangeConflict class:

public void Resolve();
public void Resolve(RefreshMode refreshMode);
public void Resolve(RefreshMode refreshMode, bool autoResolveDeletes);

The RefreshMode argument is the most interesting part of these method overloads because it determines how to resolve conflicts. RefreshMode is an enum type, defined as follows:

public enum RefreshMode {
    KeepChanges,
    KeepCurrentValues,
    OverwriteCurrentValues
}

The ObjectChangeConflict.Resolve method changes its behavior depending on the RefreshMode value you provide:

  • KeepChanges. Accepts the current user’s changes, if any; otherwise, it reloads values from the content database. This acts like a synchronizer with the content database, without losing the user’s changes.

  • KeepCurrentValues. Causes the current user’s values to win over the current database values.

  • OverwriteCurrentValues. Makes all values match the latest values in the content database (the other concurrent user’s values win).

The first overload of ObjectChangeConflict.Resolve internally assumes a value of KeepChanges for its RefreshMode argument. The third overload accepts a Boolean argument named autoResolveDeletes, which when false, instructs the entity tracking engine to throw an InvalidOperationException if a target item has been deleted.

Table 6-3 contains a matrix of possible values, which helps to explain the behavior of the ObjectChangeConflict.Resolve method.

Table 6-3. Schema of the behavior of the ObjectChangeConflict.Resolve method

Refresh mode

Original values

Current values

Database values

Final values

KeepChanges

Country = Italy<br/>Company = A

Country = USA<br/>Company = A

Country = Germany<br/>Company = B

Country = USA<br/>Company = B

KeepCurrentValues

Country = Italy<br/>Company = A

Country = USA<br/>Company = A

Country = Germany<br/>Company = B

Country = USA<br/>Company = A

OverwriteCurrentValues

Country = Italy<br/>Company = A

Country = USA<br/>Company = A

Country = Germany<br/>Company = B

Country = Germany<br/>Company = B

The MemberChangeConflict.Resolve method works almost the same as the one provided by the ObjectChangeConflict class. However, it affects only one member at time, instead of the whole entity. It also has a couple of overloads:

public void Resolve(RefreshMode refreshMode);
public void Resolve(object value);

The first overload works exactly the same as the ObjectChangeConflict method, but affects only the current member. The second overload lets you provide a custom value to force onto the content database. Thus, in this last case, you can completely change the final value of the member, providing a new value that’s different from the current, original, or database values.

Lastly, there is also a ResolveAll method provided by the ChangeConflictCollection class. It is useful when you want to solve all conflicts in one shot by applying the same conflict resolution logic to all the conflicts.

Concurrency conflict management using LINQ to SharePoint shows a complete code example of managing concurrency conflicts in LINQ to SharePoint.

Concurrency conflict management using LINQ to SharePoint uses a couple of DataContext instances to simulate a concurrency conflict. It asks the end user, via a console-based UI, how to solve the generated conflict. It also demonstrates that LINQ to SharePoint provides a rich set of capabilities for resolving concurrency conflicts, making it a mature technology suitable for real-world business solutions.

Identity management and refresh

At the base of every O/RM framework, there is an engine—generally called an identity management service—that avoids having duplicate in-memory instances of the same entity. LINQ to SharePoint also provides such a service. Consider the sample code in Code excerpt that illustrates identity management service behavior.

The code retrieves the contacts whose CompanyName field contains “DevLeap” from the DevLeap Contacts list, and changes the Country property of the first contact to USA. A second LINQ query retrieves the same list of contacts to check whether the result comes from the content database or from existing in-memory instances. To determine which, the code writes the Country value of every retrieved contact and compares the HashCode values of the first two instances of the retrieved contacts.

The following code is the output generated by Code excerpt that illustrates identity management service behavior at the console window:

Customer with ID PP001 has a Country value of USA
Customer with ID AP001 has a Country value of Italy
------------------
Customer with ID PP001 has a Country value of USA
Customer with ID AP001 has a Country value of Italy
Do the contacts have the same HashCode? True

Not surprisingly, the entities are the same; in other words, the modified contact instance takes precedence over the instance retrieved from the content database. In fact, under the covers, LINQ to SharePoint queries the content database twice, the first time executing the former query, and the second time the latter. However, because the entities requested by the second query are already in memory, the identity management service skips the data from the content database and uses the data of the existing in-memory instances instead. You might be wondering why it still executes the database query rather than using the in-memory data directly without stressing the database. The reason is that the engine merges the results retrieved from the database with any existing in-memory entities. If there are more items in the database than in memory, the engine will merge the new ones from the database and the rest that are already in memory. This is good behavior because it avoids duplication of data and instances.

Given this behavior, you’re probably wondering how you can refresh an entity from the content database, skipping any existing in-memory instance. To do that, you can use a different DataContext instance, as long as you do not have to use the same DataContext instance. Otherwise, you can call the DataContext class’s Refresh method, which has these overloads:

public void Refresh(RefreshMode mode, IEnumerable entities);
public void Refresh(RefreshMode mode, params object[] entities);
public void Refresh(RefreshMode mode, object entity);

All of these overloads accept an argument of type RefreshMode, which you may remember from the “Handling concurrency conflicts” section. Depending on the value you choose for the RefreshMode argument, the Refresh method will either forcibly reload data from the content database (OverwriteCurrentValues) or merge your changed values with those in the content database (KeepChanges). Generally, the value of KeepCurrentValues is not very useful when provided to the Refresh method, because it simply forces the entities to use the values already in memory.

Disconnected entities

In software solutions with a distributed architecture, you sometimes need to serialize an entity, transfer it across the wire to a remote site or consumer, and eventually get it back to update the persistent storage. When your data is stored in SharePoint, LINQ to SharePoint becomes an interesting solution for working in a disconnected manner. In fact, when you generate the entity model with SPMetal.exe and provide it with the /serialization:unidirectional command-line argument, the tool will mark all the generated entities with the DataContract attribute of the .NET runtime serialization engine. Consequently, you can serialize your entities and use them, for example, as the content of a Windows Communication Foundation (WCF) message.

More Info

If you would like to learn more about WCF, consider reading Windows Communication Foundation 4 Step by Step, by John Sharp (Microsoft Press, 2010).

Serializing a LINQ to SharePoint DevLeapContact entity shows a code excerpt that serializes a LINQ to SharePoint entity.

Note the line that disables DeferredLoadingEnabled. This is done to avoid circular references during entity serialization. The XML produced to serialize a DevLeapContact entity with DataContractSerializer shows the XML produced by the DataContractSerializer engine.

The XML stream contains the basic private fields of the entity, its original values, and the entity state. Thus, the XML produced is not an ideal solution for an interoperable cross-platform solution, but can be used to connect WCF consumers with WCF services (from .NET to .NET).

When the consumer makes changes to the received serialized entities and sends them back to the server, you can use the Attach method of the EntityList<T> class on the service side to reattach the entity to the DataContext and update the content database. Here’s the signature of this method:

public void Attach(TEntity entity);

This method simply accepts the entity to attach back to the DataContext tracking engine.

Note

Even if this serialization behavior seems to be a great opportunity for defining enterprise solutions that use SharePoint as their back-end storage, it is important to understand that when you have many thousands of items corresponding to data records, it is bad practice to use SharePoint as the persistence storage. It would absolutely be better to have an external DBMS with a specific and well-designed schema, with indexes and stored procedures. Instead, when you need to render your external content as a standard SharePoint list, you can use Business Connectivity Services. In software with a distributed architecture, you should create a persistence-ignorant data access layer that ignores how, where, and what the persistence is.

Model extensions and versioning

A final topic to cover here relates to managing model extensions and entity versioning. Let’s start with a couple of examples. Imagine that you have a well-defined LINQ to SharePoint model, such as the one created at the beginning of this chapter. At some point, a power user changes the data schema you provisioned, adding a custom column—such as a new Address column—to the DevLeapCustomer content type. To be able to see this new property, you should refresh the model via SPMetal.exe, which will then update the entity definition. However, it is not always possible to update the entity model and refresh all deployed assemblies.

Now consider a situation in which you have a content type that uses a custom field type, developed with .NET code and Visual Studio 2012, and you want to use that content type with LINQ to SharePoint. Unfortunately, SPMetal.exe does not support custom field types. Thus, you need to autonomously manage the code for reading and writing the custom field type.

To manage these situations but still use LINQ to SharePoint, you can implement the ICustomMapping interface for entities that you want to extend or update. This interface was specifically designed to support you when extending LINQ to SharePoint entities. Here’s its definition:

public interface ICustomMapping {
    void MapFrom(object listItem);
    void MapTo(object listItem);
    void Resolve(RefreshMode mode, object originalListItem, object databaseListItem);
}

The MapFrom and MapTo methods both receive an argument of type Object, which internally is an SPListItem instance that corresponds to the native SharePoint item behind the current entity. Using the MapFrom method, you can read untyped values from the low-level SPListItem instance and use them to configure a property—or whatever you want—in the entity. The MapTo method writes these properties back to the underlying SPListItem object. The Resolve method is a conflict resolution method similar to the ObjectChangeConflict and MemberChangeConflict methods you’ve already seen; however, in this case, it is up to the project developer to define the concurrency conflict behavior. Implementing the ICustomMapping interface shows a custom entity type created using SPMetal.exe and extended using the ICustomMapping interface.

Note the CustomMapping attribute applied on top of the MapFrom method. This is an attribute that identifies new columns mapped with the MapFrom method. It requires an array of InternalName values of supported columns. In this example, the CustomMapping attribute accepts any kind of new column (through the use of *) in order to be useful in case of versioning.

Summary

In this chapter, you learned how to implement LINQ to SharePoint to model SharePoint data as a set of typed entities, how to query that entity model, and how to manage data retrieved from LINQ queries. You also read about some advanced topics, such as managing concurrency conflicts, identity management, serialization, and versioning of entities.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset