Who?

Creating requirements for a records management solution is particularly difficult, because they originate from the overall business needs and there are many stakeholders, so multiple aspects have to be looked at thoroughly. Therefore, records managements should not be undertaken as merely an IT project. Let’s make it clear: implementation with SharePoint is just the final step of the journey.

By defining the roles and understanding the needs of the different stakeholders, we also indirectly answer the “Why” question—Why should they care? Why are we doing it at all? As mentioned before, this kind of projects is always driven by the needs of the organization (Figure 1-3).

Figure 1-3. Organizational data compliance needs

Preserving vital data is covered by the broader Office 365 Data Governance capabilities, under which falls the SharePoint Online records-management functionality. You would require the following organizational roles around the table, to have a discussion (or, most likely, a series of discussions) about the RMS design:

• Records managers and compliance officers to categorize the records in the organization and to oversee the overall records management process

• IT personnel to implement the systems that efficiently support records management

• Content managers to find where organizational information is kept and to make sure that their teams follow records-management practices

• General users who work with content daily, to get honest feedback on the system’s usability

Now, when all the right people are engaged, we can have the remaining three questions answered as accurately as possible.

What?

We will first have to find out what information should become records and what the retention and disposition policies are. That is, how long each record type should be retained and how records should be disposed.

Records and content managers will survey document usage in the organization to determine which documents and other items should become records. This content-analysis exercise, along with the compliance requirements documentation, usually results in the Holy Grail of the records management—the File Plan, also known as a Retention Schedule . What information does it usually capture?

• Types of items for records

• Retention periods for records

• Who is responsible for managing the various kinds of records

For each record type, the File Plan determines

• When this type of content is no longer active

• How long it should be retained after that

• How it should ultimately be disposed of

File Plan is typically owned by records managers and in our sample scenario looks like this (Figure 1-4).

Figure 1-4. Sample file plan

We are also told that from the records-management perspective, Manuals fall under the Reference type, and Company Guidelines are considered the Compliance kind of record. Records managers then additionally asked for a “soft delete” functionality in case of Manuals. They would like to have a “back door” and be able to un-delete any purged records of this kind within a short period of time following the disposition. On the contrary, for all Legal records (including the guidelines), they want permanent destruction without any recovery option. Let’s update the requirements based on the “What?” information from the File Plan.

How?

We now have a good idea about what type of records we are dealing with as well as their retention and disposition needs. But how should we convert active documents into records? In SharePoint, we can use the following techniques for records declaration :

• Records can be declared manually by end users.

• Records can be declared automatically

  • via Information Management (IM) Policies

  • or by creating a workflow that sends a document to a Records Center

Let’s have a closer look at those methods.

Where?

Phew! We have crossed three questions off our design list. There is now the last “Where?” question remaining unanswered before we can get to the most exciting part of the project: implementation.

Location-wise, SharePoint provides two strategies for managing records:

• In-place records management allows you to archive content directly in your site. That is, the site can contain both active documents and records. SharePoint blocks in-place records, as a result of which they can no longer be manually altered, removed, or relocated. You can even specify different retention policies for active documents and records.

  Note One limitation is that you cannot use in-place records management with document sets.

• Records Center site collection(s) can serve as a content vault. The last version of the source “active” library (whether this is a major or minor version) becomes the first version in the target “record” library.

The fundamental attribute of in-place records is that the records stay in the same location and exist alongside active documents. All previous versions remain visible and accessible. From the end user perspective, in-place records largely act as active documents, still have the same permissions, and don’t disappear anywhere, therefore fully preserving the context. On the other hand, downsides for this management approach are an increased difficulty of records discovery and a weaker security model. Also, by combining active content and records in the same library, we cause its size to grow more rapidly.

For a Records Center, structure and security are on its strong side. All records are distinct and easily accessible by records managers in a centralized and secured manner. Stricter rules are applied, and the records cannot be altered or deleted for the duration of the retention period. The drop-off library and its routing rules enforce structure. On the flip side of the advantages coin, there is a reduced visibility for the end users: documents are removed from the original libraries and context.

Other Things to Keep in Mind

At each point, you should thoroughly document your records management guidelines, plans, and any defined metrics. If your enterprise becomes engaged in any records-related litigation, you might be obliged to present this type of documentation. Make sure you also have the auditing, monitoring, and reporting covered.

Solution Elements

We will be working with a few SharePoint components, among which the following major ones:

• Content Type Hub (sometimes also referred to as Content Type Publishing Hub), to centrally manage Enterprise Content Types

• Document Collaboration site, which is just a classic team site with the following document libraries:

  • “Manuals” library (with in-place records management enabled)

  • “Legal” library

• Records Center site

Of course, there are other components involved, such as the Taxonomy Term Store for Managed Metadata, but they are not directly specific to the RMS solution we are implementing and, therefore, will be considered generic SharePoint elements in the context of this chapter.

Be aware that the implementation process is not linear and will require us to frequently switch between different sites, so bear with the progression and stay focused. You will see that, logically, all the steps are nicely aligned and, at the end, should make total sense to you and your customers.

Create Records Center

We will start our implementation activities with the Document Center site collection creation (Figure 1-5).

Figure 1-5. Create New Site Collection

In SharePoint admin center , create a new site collection.

Select a Records Center template under the Enterprise tab (Figure 1-6).

Figure 1-6. New site collection template selection dialog

After some time, your brand-new site collection will be created.

Right out of the box, this new site is optimized for high-volume document submission and equipped with many useful enterprise records management features (Figure 1-7).

Figure 1-7. Records Center home page

In the spirit of focusing only on the practical aspects of the implementation, I will not be providing a detailed theoretical description of all the Records Center site collection capabilities (or, for that matter, one of any other standard SharePoint components we are about to create). There is more than enough general information about all of those, which is readily available on TechNet.com and via your favorite search engine.

Configure “Send To” Location

We must now make SharePoint aware of our newly created site. Do not leave the Records Center yet; we must get some information about it first.

Go to Site Settings and click Content Organizer Settings, under the Site Administration section (Figure 1-8).

Figure 1-8. Content Organizer Settings link

On the settings page displayed, scroll all the way down and copy the Submission Points URL (Figure 1-9).

Figure 1-9. “Send to” URL

The URL will look similar to this: https://contoso.sharepoint.com/sites/rc/_vti_bin/OfficialFile.asmx .

Now, let’s get back to the SharePoint admin center (remember—a lot of switching!). This time let’s click the records management link in the left-hand navigation panel (Figure 1-10).

Figure 1-10. Records management settings

Here, you will configure a new “Send To” connection. Just give it a name (“Records Center,” on the following screenshot) and paste the “Send To” URL you have obtained in Step 2. The end result will look similar to Figure 1-11.

Figure 1-11. “Send To” configuration

Just to verify that the connection is successfully configured, you may want to check out the Send To drop-down in the document ribbon menu of a site collection of your choice, within the document library ribbon’s Files tab. You should see your connection available there (Figure 1-12).

Figure 1-12. “Send To” drop-down

It can take a while before the connection is propagated to all the sites. The timer job will eventually pick it up, so be patient.

Create Content Types

Now we proceed to the creation of what I call “pillars of document management”—the mighty content types. In SharePoint, you can control content types at different levels, but we will be doing it in a centralized “enterprise” manner. Our approach will allow maximum control over the metadata and policies to the records managers, while hiding most of the complexity from the content owners and end users.

I would like to point out a very important piece of information: Enterprise Content Type Hub is, in fact, pre-created for you in SharePoint Online. However, if you try to find it under the list of existing site collections in SharePoint Online admin center, you will be disappointed; it is not there. To get to this hidden site collection, you will have to specify its URL directly. Just add “contenttyphub” after the “sites” managed path of your SharePoint Online URL. For the Contoso tenant, the URL will look like this: https://contoso.sharepoint.com/sites/contenttypehub .

You will then see a standard site collection based on the Team Site template. The only difference from a regular out-of-the-box team site will be an activated “Content Type Syndication Hub” site-collection feature.

In my case, I just removed all the clutter from the home page, changed the color theme, and updated the site icon, so it can be easier to differentiate visually (Figure 1-13).

Figure 1-13. Enterprise Content Type Hub home page

Go to Sites Settings and click the Site Content Types link under the Web Designer Galleries section (Figure 1-14).

Figure 1-14. Link to Site content types settings

Using the standard Content Type controls, create the hierarchy of content types for both General and Legal kinds of records (Figure 1-15).

Figure 1-15. Content types hierarchy

This is mostly a document management exercise, so you can plan your hierarchy differently and add as much metadata as needed at any level. This approach is a great way to control the enterprise metadata and be in compliance across your entire SharePoint environment. Just make sure you create your Managed Metadata using the centralized Taxonomy Term Store.

In our sample implementation, we will keep the metadata to a minimum, to avoid unnecessary cluttering.

Starting with “100—General ” and build your way down to the “105—Reference” (Figure 1-16).

Figure 1-16. General content yype

You can see the inheritance relationship in the following screenshot (Figure 1-17 ).

Figure 1-17. Reference content type

Last, you create the Manual content type. In the example following, I chose to place it under a new group, called “Enterprise Content Types,” just to make it easier to differentiate between the pure records management content types (“Records Management” group) and the end-user facing content types we are going to publish (Figure 1-18).

Figure 1-18. Manual content type

We will then go through the same motions for our Legal content type hierarchy, resulting in the Company Guidelines content type. The only difference for this content type is that we have added one custom column to it: “Final Approval Date” (Figure 1-19).

Figure 1-19. Company guidelines content type metadata

After you are done, the final content type structure should look like what you see in Figure 1-20.

Figure 1-20. All our custom content types

Create Information Management Policies

The next step will be to create and apply the appropriate information management policies. We will define them according to our design requirements. Also, we have decided to create content type-based policies to ensure centralized control. But then we are faced with a choice of to which level of the content type hierarchy the policies should be applied. Keep in mind that we can only assign a policy to a certain single point in this structure. The top level would be too generic, because we will have more kinds of records beyond Reference under the General category. The File Plan, in fact, defines retention and disposition policies at the record-kind level, which, in our case, is represented by “Reference” and “Compliance” in their respective categories. Therefore, we will assign the policies at that level (Figure 1-21).

Figure 1-21. Information management policy assignment

We are still within the Content Type Hub site, and we should stay in its context, because all the policy configurations will be performed here. Remember: The first- and second-level content types are staying within the Content Type Hub and will not be published. Only records managers will be able to access the Enterprise Hub, to update the policies and other content type properties. For content owners and end users, the inherited policies will just “magically” work at the level of the published content types—Manual and Company Guidelsines.

To create a policy for a particular content type, just click the content type name (under Site Settings ➤ Site Content Types) and then, on the Site Content Type page, in the Settings section, click Information management policy settings. There, you can define a policy by updating the Edit Policy page.

To specify a retention period for documents that are subject to this policy, click Enable Retention, then specify the retention period and the actions that you want to occur (Figure 1-22).

Figure 1-22. Policy update form

For the Reference content type, specify two retention stages, as shown in Figure 1-23.

Figure 1-23. Retentiovn stages for Reference records

For the Compliance content types, there will also be two retention stages to satisfy the requirements (Figure 1-24).

Figure 1-24. Retention stages for Compliance records

The “Final Approval Date + 0 days” stage is somewhat artificial and is used here only for demonstration purposes. As mentioned previously, you will likely rely on a workflow to send documents to the Records Center in a real-world scenario. However, it is perfectly fine to use this stage when you are explaining to your customers how the policies work. In particular, there is one somewhat obscure point we are trying to convey by doing that.

Let’s take a look at the second stage of the policy assigned to the Reference content type. To set a stage based on a date property, in the Event section, we click “This stage is based off a date property on the item” and then select the action and the increment of time after this action (number of days, months, or years) when we want it to be triggered (Figure 1-25).

Figure 1-25. Stage editing for Reference records

Note that the only date properties available in the drop-down list are Created, Modified, and Declared Record. In some articles and blog posts, those three options are described as the only ones possible.

That, in fact, is not quite true. Any date-based property can be used here. And we can easily demonstrate it when creating a policy for the Compliance content type. Do you remember the custom column “Final Approval Date” of the Date and Time type that we created for this content type? When editing the first stage for its policy, low and behold, we now have the fourth option available (Figure 1-26).

Figure 1-26. Stage editing for Compliance records

At this point, you should have two information management policies (each with two stages) created for the Reference and Compliance content types. Any content types created that are based on those two would automatically inherit, among other properties, their information management policies. And that is exactly what we want to achieve for the Manual and Company Guidelines content types.

Publish Content Types

All the information management policies are now created, and it is time to publish our content types. As designed, we will only be publishing the business user-facing content types Manual and Company Guidelines (Figure 1-27).

Figure 1-27. Published content type

Go to Site Content Types, choose “105—Reference ,” and click “Manage publishing for this content type.” As you can see following, the “105—Reference” content type is not being published, even though we applied the Information Policy to it (Figure 1-28).

Figure 1-28. Higher-level content type is not published

Only the “Manual ” content type (that is, that below “105—Reference” in the hierarchy) is getting published. After you publish it, the publishing settings for this content type will look as shown in Figure 1-29.

Figure 1-29. Manual content type is published

The same goes for the “Company Guidelines” content type. It would be the only one published in the Legal content types hierarchy.

We now have all the content types ready to set up document libraries.

Create and Configure Collaboration Libraries

Let’s now switch to the site collection whose documents the business user will be working with on a daily basis. Create two new document libraries here: “Legal” and “Manuals” (Figure 1-30).

Figure 1-30. Document Space libraries

We should also make sure that the content types we published earlier are available in this site collection (Figure 1-31).

Figure 1-31. Available content types

Again, it can take a while before the published content types are propagated to all the sites, so be patient.

Now, let’s configure the Legal library to accept only a single content type, “Company Guidelines.” Add it as the default custom content type and delete the out-of-the-box “Document” one (Figure 1-32).

Figure 1-32. Company Guidelines content type in Legal library

For the Manuals library, we will only allow Manual content type (Figure 1-33).

Figure 1-33. Manual content type in Manuals library

Now the business users can start working with the documents.

Configure In-Place Records Management

As you remember, we decided to configure in-place records management for the Manuals library. Let’s first activate the respective site collection feature (Figure 1-34).

Figure 1-34. Activate In Place Records Management feature

We will then disable location-based policies for the site collection, by deactivating the Library and Folder Based Retention feature. This enables content owners to ensure that the content type policies are not overridden by the list administrator’s location-based policies (Figure 1-35).

Figure 1-35. Activate Library and Folder Based Retention feature

We don’t want our in-place records to be modified or deleted, so let’s block Edit and Delete for records, via Site Collection’s “Records Declaration Settings” (Figure 1-36).

Figure 1-36. Record Restrictions configuration

Finally, let’s enable in-place records management for the Manuals library, via the library’s Library Record Declaration Settings (Figure 1-37).

Figure 1-37. Library Record Declaration Settings

To confirm application of the policies, you can check the Manuals library’s “Information management policy” settings (Figure 1-38).

Figure 1-38. Information Management Policy Settings of library

Because we are controlling the policy at the higher level in our content type hierarchy, site collection administrators won’t be able to modify it (Figure 1-39).

Figure 1-39. Information Management Policy Settings cannot change

You can now upload a test document to Manuals library and check its compliance details. To access a document’s compliance details, click “…” beside the document in the library, then click “…” again in the info panel, select “Advanced,” and. Finally, click “Compliance Details” (Figure 1-40).

Figure 1-40. Compliance details for non-record

It’s time to declare a document as a record in-place, via the ribbon menu (Figure 1-41).

Figure 1-41. Declare in-place record

The lock icon will identify the in-place records (Figure 1-42).

Figure 1-42. In-place records

You can check the document’s compliance details again , to confirm that it’s now a record (Figure 1-43).

Figure 1-43. Compliance details for record

Go back to the ribbon menu, to ensure that no deletion or modification has been allowed (Figure 1-44).

Figure 1-44. Record modification is disabled

Looks like we have just fulfilled all the requirements for Manuals. Hooray!

Configure Content Organizer Rules in Records Center

Now it’s time to tackle all the remaining legal requirements for Company Guidelines documents. We must switch to the Records Center.

There, create a “Legal” records library.

We then add the Company Guidelines content type to the library’s content types. Good thing we created all our content types centrally, so they are now readily available in all site collections throughout the entire SharePoint Online tenant!

The next step will be to create a new Content Organizer Rule, based on the content type we have just added (Figure 1-45).

Figure 1-45. Content Organizer Rules settings link

You should end up with something similar to what is shown in Figure 1-46 and Figure 1-47.

Figure 1-46. Content Organizer rule for Company Guidelines—overview

Figure 1-47. Content Organizer rule for Company Guidelines content type

Now, get back to the Document Space site collection, navigate to the Legal library, select a document, go to the ribbon’s Files tab, and test the new Routing Rule, by sending a document manually to the Records Center (Figure 1-48).

Figure 1-48. Sending to Records Center

When looking at the Records Center, you should see the document automatically routed and placed into the appropirate Legal records library.

Challenges

If it is all that simple, why do we hear again and again about failed RMS implementations? The challenges companies face when working on records management are not exclusive to SharePoint.

Here are some of them:

• Issues

  • Missing structure(s)

  • Missing business support

  • Poor process support

  • Treated as infrastructure

  • Low user focus/poor UX

  • Missing taxonomy

  • Missing metadata

  • Missing standards alignment

• Result in

  • Low capture ratio of business-critical information

  • Low perceived value for users and business

  • Duplications

  • Inability to find existing information

  • Information overflow

  • Information silos

  • Compliance risk

Those items should be addressed in order to achieve any success with records management projects.