Perhaps the best illustration is the actual flow of data within the Search, from the time a document is uploaded to the portal to the moment that the document is found via search by another end user:

Although the data in the portal can be backed up with an off-the-shelf SQL backup tool, the backup of the index is slightly more complicated because the index includes database tables as well as the resulting index files. As such, you must use the built-in backup utilities to achieve a full fidelity index backup. With a basic SQL backup, you will back up the database part of the index, which means that upon a restore, index propagation will commence, and the end users will not be able to search immediately. Thus, as part of your disaster recovery plan, you should include the time to propagate the indexes.

Depending on the needs and characteristics of the deployment, you may choose a variety of configurations. The typical choices will be influenced by server geography, volume of data, and volume of searches. Table 27-8 describes these factors and their solutions. There are several important rules to observe:

One of the Search administration pages actually lies within the Application Management page, within the Manage Search Service section (Figure 27-12). It is a mishmash of farm-wide and application-specific settings, which generally ties the integration between the application, the farm, and the SSP on a single page. The Shared Services section simply indicates and links to the Shared Services Provider associated with a given WebApplication, as well as the management pages we’ve worked with throughout this chapter.

Figure 27-12. Manage Search Service

The Manage Search Service page is the place where you can change the options that were originally set during the configuration of the farm, either on the Farm-Level Search Settings page or via the Office SharePoint Server Search Indexing and Query link.

The Managing Farm-Level Search Settings deals with configuration for external searches, and allows you to configure information such as:

Next, in Figure 27-13, you can see the Configure Office SharePoint Server Search Service Settings On Server servername administration page. This page mixes server and farm settings for configuration of service-related properties. Table 27-10 lists the search configuration elements.

The last element of the farm-level management is the Impact Rules management page, which allows us to edit Crawler Impact Rules, as in Figure 27-14.

Crawler Impact Rules are used to throttle the retrieval of requests that are made against the WebFront End (WFE) server. In many cases, the WFE simply is not capable of serving as many pages as the crawler can request. This results in poor performance at the WFE, along with crawl errors associated with the data source. Most likely this type of problem indicates an underpowered server, combined with heavy ASP.NET processing, which results in longer processing of the pages. This can be fixed in two ways: by disallowing the crawl of the ASPX pages within the site (this is configured within the Site administration settings) or by changing the Request Frequency within an Impact Rule.

Figure 27-13. Office SharePoint Server Search Service Settings

Figure 27-14. Add Crawler Impact Rule

Each Impact rule is associated with a URL that is used for crawling a SharePoint site, and then a request frequency, which can be represented either in terms of the number of simultaneous requests or in terms of a delay between requests. You must look at the performance of the WFE server to determine the optimal request frequency, while also balancing out the need for a short indexing time.

Content Source configuration allows for indexing and management of schedules of different sources of data that flow through the index. Most commonly, this will be used to set up a schedule for a default data source, but also it will be used to add external content to the Search engine, such as external web sites or file shares.

Content Source configuration is available from the SSP, from the Content Sources And Crawl Schedules page. There you can see the listing of available Content Sources and their schedules. Clicking the New Content Source link opens the Con-tent Source Configuration Page (Figure 27-15).

After specifying the name, you have to specify the desired indexing handler or effectively indicate the appropriate Content Source Type, which can be one of the following:

Next is the start address, which will typically be http://servername or \servername share for a folder share. Unlike previous versions of Index server, you can associate a single content source with multiple addresses. In order for the Business Data crawl to take place, the Business Data Catalog must first be configured within the Shared Services Provider. For the Exchange Server, you’ll typically configure the search of public folders via the HTTP protocol.

The Crawl Settings section adjusts dynamically and is context-sensitive to the Content Source Type (Table 27-11). Crawl Settings displays several options, ranging from single page crawls to unlimited crawl within the content source. It is unadvisable to use very broad settings, especially for web sites (e.g., unlimited page or server hops), as broad settings will potentially kill the content source and may never finish indexing. In fact, it may be easier and more economical in the long run to integrate results from external search engines into MOSS rather than index big sites.

Figure 27-15. Content Source Configuration page

The last setting for each Content Source is the schedule, which resembles a typical scheduling calendar. Besides the choice of Crawl Schedule frequency, which can be set at Daily, Weekly or Monthly intervals, there are additional settings that indicate how often the given schedule should be repeated and how long the crawl should last. Each Content Source can have two different types of crawls, Full Crawl or Incremental Crawl. Obviously, to build an index from scratch, you will need to start off with a Full Crawl, and the Incremental Crawl will intelligently pick up additions, deletions, and changes to the content source. There are, however, other times when the Full Crawl must be run:

In most circumstances, a daily Incremental Crawl schedule should be sufficient.

Generally, the Authoritative pages make sense only in the context of using multiple Content Sources, and the concept of Result Relevance may not be applicable to single portal sites. Essentially, the Authoritative pages increase the relevancy of a page by decreasing the click distance to the source of truth. Administration is very simple, as there are four entry boxes, allowing for entry of the following:

As such, when indexing a number of content sources and observing the quality of the search results (as described in the next section), there is a possibility that a page with better information could be presented further down the results listing than a page with the best match. Thus, in order to increase the rank position, the administrator should add the site that hosts the more relevant data as the Most Authoritative Page, and perhaps add the useless site as the Site to Demote.

This feature is useful when indexing many content sources that potentially have similar data—for instance, your intranet site versus your public Internet site. Most likely, your intranet would have the most relevant information.

One of SharePoint’s most desirable features for information officers is the ability to monitor query and results statistics via the Query Reporting Tool (Figure 27-16). The tool creates graphs as well as different types of reports that can be exported to Excel or PDF formats. These are available for each SSP within the Search Usage Reports section, and the Search Results page is available via the menu on the lefth and side of the page. Based on this information, you may want to fine-tune best bets, scopes, authoritative sites, and plan capacity of your Search services. This page displays the following reports:

Figure 27-16. SSP Search Queries Reports

Similarly, the Search Results Report has the following reports available:

The Search Results Reports page is very useful because it highlights the problem areas, or at least the areas where addition of Best Bets or Authoritative sites may increase the end users’ ability to find relevant information using Search.

Unfortunately, these pages are buried within the administrative features, and may not find an appropriate audience. The typical consumer of these reports should be the site collection administrator who is managing the Best Bets.

Looking away from the SSP, and at individual sites and portals, there are many advanced search settings that are configurable only from the Site Collection or Site level. In order to open these settings, you have to select Site Settings (or Modify All Site Settings) from the Site Actions drop-down menu. Depending on the site template and privileges, your Site Settings page may be hidden under a particular submenu. Figure 27-17 displays the Site Settings page at the Site Collection level, and highlights management pages that are relevant to Search.

Figure 27-17. Site Settings page at Site Collection level

Depending on the context of the Site Settings page, some of the management options that are listed in Table 27-12 and visible in Figure 27-18 may not be available; however, if they are available, their meaning and management screens will be identical.

Finally, the Add/Edit Keyword page, as seen in Figure 27-18, can be launched at a site collection level from the Manage Keywords page. It provides a quick way for an information officer to enhance the search experience. Based on the Query and Search Result data, you can easily steer the users in a proper direction by providing additional synonyms, a definition, or even best bets to increase the value of the Search Results page (this is where the contents will eventually show up).

Similar to the result advertisements in Google, the targeted results show up on the righthand side of the page (refer back to Figure 27-1). Typically this feature is used when a large number of queries target a particular search term—for instance, “vacation” on an HR site, with no proper exit pages. In such a case, the administrator can associate the term “vacation” with a couple of synonyms, such as “holidays” or “days off,” and provide a link to the company policy on vacations.

A Keyword or a Keyword Phrase consists of the following elements: synonyms, best bets, definition, contact, and various dates associated with publishing and visibility. Synonyms are important because they allow association of various elements with a best bet or a definition. Best bets are important because they allow the site administrator to advertise a particular link or a location as a good result for a particular query. Similarly, a definition, as well as a contact, can provide additional clues to the end user about the applicability of the suggested result.

Figure 27-18. Add Keyword

Besides enhancing the search with a keyword or a best bet, the second typical scenario is the ability to add a site column that shows up in a custom list to the Advanced Search screen. For the sake of the exercise, we’re assuming that this column already exists and is called “Company Name”:

The Enterprise Search extends the SQL-92 and SQL-99 standards to provide additional functionality in the area of search. For those who already know standard SQL, the query format is very familiar:

	SELECT <columns>
	FROM SCOPE( )
	WHERE <conditions>
	ORDER BY <columns>

The following table describes these standard clauses.

Although the basic clauses are the same as in standard SQL, some other features and keywords may be unavailable, and there are two specialized predicates: CONTAINS and FREETEXT. Additionally, the main areas where Search queries extend SQL Search include the ability to use 127-character column names, accent insensitivity, use of a thesaurus, and a looser interpretation of the NULL predicate. A typical query to display a title, path, and author is:

	SELECT title, path, author FROM Scope() WHERE CONTAINS('author:Piotr Prussak') AND
	"scope"='Books' AND FREETEXT(DEFAULTPROPERTIES, 'MOSS') ORDER BY Path ASC

Unlike SQL Server queries, where columns are defined in tables or views, the columns that are available for Searching are only defined in SSP’s Search Settings on the Metadata Property Mappings page. What makes the metadata search work well is the ability to map multiple site columns into a single search column, where, for instance, a Contact Name column and an Employee Name column could both be mapped to the same property LastName. Two additional common results columns are Rank and LastModifedTime, which are often used to sort the results.

Another possible way of utilizing MOSS Search is via the URL syntax. The decision to have either a piece of code that submits HTTP GET queries or screen-scrapes pages is up to you. The syntax is very simple, but it is somewhat more limited as compared to the other forms of Search. Typically, the URL queries are submitted to a specialized search page, such as results.aspx within the Search Center. In most circumstances, all parameters are combined, but be careful when submitting only a partial URL syntax, as the results page may fail to render results. The parameters are outlined in the following table.

A full URL query may look like this:

That URL query would look for the following: