CHAPTER 4

DAM Technology Services: Asset Creation and Management

Ingestion

Asset Lifecycle Services

Media Processing

Taxonomy Management

Vendors usually call asset creation ingestion. The meaning of “creation” in the DAM world differs from its meaning in other content technologies, such as with a web content management (WCM) system. In those technologies, content creation actually happens in the system: A user logs in, types in content, and then publishes it to a website. The DAM system “creates” an asset when it ingests a file and accounts for its corresponding metadata. Both the file itself, such as an image or video, and its metadata may have actually been created outside the system, but in the context of the DAM system, the asset is not created until it and its metadata are ingested. As such, unlike WCM or DM systems, DAM systems typically don’t offer native creation services. You’re expected to author files in some other tool. That’s why “connectors” to native applications such as Avid Interplay, Adobe Premier, Adobe InDesign, and so on, are so important.

In a DAM system, ingestion typically combines several smaller and more discrete services, capabilities, and user actions to turn files into assets in a single large step. In this context, we discuss ingestion in some detail, as it is one set of capabilities needed in the overall asset creation process.

Ingestion

Ingestion is one of the core capabilities in DAM software. It’s how files, media, and metadata are transferred to the system, prepared for system admission, and become managed assets. Prior to ingestion, you often set up what are called “hot folders”: an easily accessible, topic-based grouping for assets of particular types or categories.

Ingestion includes the set of capabilities or services that determine the following:

• What types of files are supported (see Table 4.1)?

• How are files uploaded, copied, or moved from their current location into the DAM, or referenced at their current location from the DAM system? (By “reference,” we mean a digital “pointer” to the content’s actual location, which can be on essentially any kind of storage—see “Copying Versus Moving Files.”)

• What asset processing can be performed at this initial stage?

• How are the files and metadata “turned into” assets?

TABLE 4.1 TYPICALLY SUPPORTED FILE FORMATS AND CODECS

Supported File Formats	.3gp, .aac, .flv, .m4v, .mov, .ogg, .vc1, .3g2, .amr, .gfx, .m4a, .mpa, .mfx, .wmv, .avi, .asf, .h263, .mp3, .mpg, .rm, .wma, .ac3, .dv, .h264, .mp4, .mpeg, .ts, .vob
Supported Audio Codecs	AAC, AMR-NB, DV Audio, MPEG-1 (mp1, 2, 3), Windows Media, Audio, Pro, AC-3, AMR-WB, orbis, PCM (16-, 24-, 32-bit), Real Audio
Supported Video Codecs	Cinepack, DV Video, DVC Pro 25, DVCPro 50, DVCPro 100/HD, Flash Video, H.263, H.264, HuffYUV, M-JPEG, MPEG-1, MPEG-2 (PS and ES), MPEG-4/XVID, Microsoft MPEG-4, ON2 (VP5, 6), Sorenson, Theora, VC-1, VC3/DNxHD, Windows Media Video (7, 8, 9), XVID

Supported File Types

Your first task is to consider what types of files the DAM system supports. Because digital asset management and marketing asset management have emerged out of multiple technology and industry origins, such as publishing, broadcasting, and content management, a digital asset management system may have a bias regarding the types of files it supports. Many can handle almost any file type. Others handle only video, images, or a smaller range of rich media file types. For example, some may or may not handle Flash files.

By “handle,” we mean the product is able to recognize or process the file type, even if not completely. In its basic sense, processing means that the system recognizes the file format, can create a thumbnail and other “renditions” of the file for preview or proxy use, and, in some cases, can transform or transcode the file into other file formats. Some systems can create the standard and basic thumbnail for the file but cannot perform any transformation functions.

Processing can be the ability to extract or pull out of the file embedded information, in various formats, such as IPTC, EXIF, XMP, file attributes, Microsoft Office file metadata, key frames, and closed-caption text from a video. Regardless, the DAM system must support the specific set or breadth of file types with which your users will be working. For instance, if you’re using a specific Camera RAW file format or video format, make sure that the system can support it and its manipulation into other required or low-resolution formats.

DAM vendors—especially those with systems that can generally handle all file types—try to ensure that adding new file types is well planned for, and their approach to this may be an important discussion to “future-proof” your system.

Copying Versus Moving Files

DAM systems either transport files from their initial location, such as on the users’ desktops, to the common, often centralized repository, or leave them where they are and create a reference or link to them in the repository. This architectural distinction is important.

The first case implies several things. First, it implies that you have a mechanism for transferring the files. Enterprises typically transfer or upload files using either FTP or HTTP. Which protocol you use should depend not on what the system provides, but on what your requirements are. For example, FTP works well for small- to medium-sized files and for bulk file transfer, but it may not be reliable enough for very large files. You don’t want a file transfer to fail halfway through a large file, and then you are unable to restart it from the point of failure. Alternatively, you may need to transfer files through a corporate firewall, and the IT department may not leave open the FTP port for security reasons, so you have to transfer the files using HTTP over port 80 or, in some systems, FTP over HTTP on port 80. The key point is that the IT department may dictate the File Transfer Protocol, which will affect the speed and reliability of delivery. Therefore, be aware of the File Transfer Protocols supported by the DAM as part of ingestion.

Most DAM systems copy the file into the repository, duplicating its storage. They establish the file in the repository as the “master,” creating a centralized repository, and leave the original file where it was. This feature ensures that if the transfer fails, you still have the original file and can attempt to upload it again.

Alternatively, the DAM system moves the file, rather than just copying it, and then it removes the file from the original location when it completes the transfer. As a service, this variation is atypical, but some systems support it. If the system supports moving the file and the ingestion fails, it should leave the file where it was. Based on your particular needs, you need to examine where you want to store your files. You may want to leave your huge video files on a production server in Los Angeles rather than in the main repository in New York.

This example leads into the second case: references or “pointers” to remote files. Many asset management systems, particularly smaller workgroup and original desktop systems, ingest a file by creating a thumbnail of it and leaving it where it originally sat on the file system. Without going too far into storage management territory here, in this case, the management system treats the file system as a virtual repository and simply catalogs what is on the file system (which could be distributed across numerous machines). Simple consumer DAMs and media management systems, such as iTunes, function in this way.

For an enterprise, this approach works best for very large files, such as HD video or print-ready PDFs. In this case, you don’t want to incur the transfer cost but need to be able to catalog and rendition the assets in place and reference them and their storage location from the DAM system. This approach also comes in handy if you later move to a secondary or tertiary storage and archive format.

The downside: If the storage location becomes inaccessible or the file is moved, the asset becomes inaccessible. Systems often try to recover or catch this circumstance and provide a mechanism for recovery, but the resolution may require significant human intervention.

Bulk Upload

DAM systems typically can bulk upload, do batch transfers, select, and transfer multiple files to the DAM in one fell swoop. The systems vary in how they support bulk upload, specifically, in how they handle the failure and recovery of the upload. For example, if you’re ingesting 1,000 files, what happens if a file fails to transfer due to a disk, network, or file system error? Are you able to restart the transfer from the point of failure? Does the system ignore the one file and continue, telling you at some point that it failed? In the worst case, if one file fails, the whole batch fails. Similarly, what happens when the system fails in processing a file? Perhaps the file format is corrupted or the system doesn’t recognize it, so it can’t extract the metadata or generate renditions. Does the system make a list of all the failed files? Is this list visible to the person uploading the file, to the admin, or both?

Compound File Handling

Compound files contain, embed, or reference other assets. Examples can include Adobe InDesign or Illustrator documents, books, catalogs, brochures, ZIP archives, PowerPoint presentations, or a DVD. Generally, compound assets are master container files that embed or reference other files, which may or may not also be compound.

Uploading compound assets introduces a number of challenges for a DAM system. How they’re handled varies from product to product. The first challenge is how the system marshals the externally referenced and internally embedded files. In other words, how much work do you have to do to upload a compound file? Quark and Adobe layout documents commonly reference files that live elsewhere on the file system. Are you required to gather all of these files into a location and upload them as a batch? Do you need to create a special ZIP file that contains the full document so that the system has and can reference all the linked files? Or does the system provide mechanisms or tools to perform this process for you?

PowerPoint files are a common example. A slide deck may reference images and files that live elsewhere on the file system. They aren’t assets (yet), but they need to be included in the upload. If the system doesn’t handle this process properly, when you try to download the file from the system, it will have unresolved links to files and may corrupt your presentation. Compound files become what we generally call “compound assets.” We discuss them in more detail in the section “Asset Creation” later in this chapter.

Video Ingestion

Unique to video and media-specific workflows, the video ingestion process can vary due to the complexity of video. Ingestion can include transferring digital video files, converting from analog videotape or formats to a digital format, or receiving a streamed or live video feed, particularly in archiving and producing video. Even in deep video use cases, however, we’re increasingly seeing fully digital (or “file-based”) workflows.

If you work with digital video, you need to consider the system’s transcoding capacity. How many video files can the system convert at one time to other typically lightweight formats for preview and distribution? In some use cases, the system may need to bulk ingest tens or hundreds of longer video clips, which will later be segmented into smaller clips. With the increased use of high-definition video as a source format, more users need to transcode HD to low-resolution formats, particularly for distribution to the Internet or mobile devices. An increasing number of systems can transcode to Windows Media and QuickTime formats, with some variation in the file resolution, bit rate, and encoding. HD files require significant storage and processing to convert, particularly for longer videos of more than half an hour. Often, the system uses dedicated hardware and software to create “transcoding farms,” resources that increase the capacity and throughput of the process.

Metadata is yet another concern. What, if anything, does the system extract from the video in order to later retrieve it? Because of the video asset’s rich visual and audio content, video ingestion and automated video metadata extraction are more complicated and involved. Video ingestion typically includes the following automated processing:

• Extraction of file information, as would occur with any file

• Extraction of key frames and their associated time codes

• Extraction of closed-caption text, if it exists, and indexing that text back to the time code and frame in which it occurs

• Extraction of final production script information with time codes

• Optional conversion of speech into text, specifically into a text transcript that indexes the text back to the particular frame in which the spoken words occur, thereby enabling full-text search to locate sections of relevant video

• Optional identification of the speaker or, in some cases, speakers

• Transcoding the full video into desired stored renditions

When you are considering production video ingestion, the volume of simultaneous streams or feeds can be another initial capacity concern. Does the system have the ability to receive and handle the volume and speed of the feeds? Can it then transcode the received video into other specific formats in a timely manner? Can it recognize camera location or geospatial data? Beyond that, video ingestion includes the manual (human) or automated tagging of the video from the feed with metadata, either in real time or with some delay.

Metadata Extraction

As mentioned earlier, ingestion includes processing the incoming file. Typically, it includes extracting or pulling out the embedded information or data that becomes metadata in the DAM system. The information could be embedded in the file as file attributes in various standard formats, including IPTC, EXIF, and XMP; as Microsoft Office file implicit metadata, such as author, date, and modified by; or as (in the case of video) scene changes, key frames, time codes, closed-caption text, or possibly through speech-to-text conversion that indexes text back to the video.

Systems vary widely in the kinds of file attribute and metadata extraction services they provide. You need to be sure that if your workflows or metadata include information stored in standard formats, you can extract and store—or map and store—in the system. More generally, you need to understand which information the file system can extract for a given file type and which it cannot.

External Metadata

Our discussion thus far implies that the system stores the metadata in the actual media file itself. In many use cases, the metadata persists in a separate database or in a separate file, such as an Excel spreadsheet, that is married with the file at ingestion. In some cases, this metadata file may itself become an asset later in the process (in which case it’s sometimes called a metadata sidecar). Not all systems support marrying separate metadata to individual assets. If this feature is important for your use case, ascertain whether the system provides the capability.

Additionally, metadata can live in an external system and be pulled by the system at ingestion. We expect this ability to access or query external systems for metadata to become increasingly standard. It can assist many workflows where the information already exists in other systems and simply needs to be aggregated in one place. The file is part of the asset, providing increased value and searchability to both the file and to the asset.

Automated Ingest

Note that new videos are not created in a DAM system, but existing content is brought into (or ingested into) the system from a variety of sources. The video feed can be from a live source or from a previously existing one. Live sources can be via a video camera, a satellite, or even a smartphone. Existing content may be ingested from tapes on videotape recorders (VTRs) or may already be in digital form on another server.

The ingest module in a DAM system allows simultaneous recording from multiple sources of feeds. There is typically functionality to schedule recordings at predetermined times and dates, and they can be set for one time or repeating. There is also a tool to monitor ongoing recordings and check the details of past recordings. Monitoring tools let users preview the feed while it is being recorded instead of waiting for the recording to finish; this feature permits them to make notes of interesting parts of the video and subsequently use them.

FIFO Ingest

Some DAM systems record feeds from live sources (for example, satellite) in a temporary storage area. The amount of this storage is configurable based on requirements. Frequently, the First In, First Out (FIFO) rule is followed here; that is, recordings continue until the temporary storage is full, and then the system continues to record by erasing the first-in (oldest) video content.

This temporary storage feature is useful in recording feeds even while operators are not monitoring the feeds. Unless it’s deleted or overwritten, the video is available in the temporary storage. Users can browse the contents of the temporary storage and extract clips by marking in and marking out relevant portions of the video segment.

There is a plethora of details about ingestion technology. A DAM package that does this well is able to do the following:

• Ingest a wide array of formats.

• Ingest in multiple ways, such as uploading via web browser, FTP, mobile, and tablet devices. Most vendors are adopting simple drag-and-drop interfaces for ingestion.

• Read any previously embedded metadata at ingest. Some vendors require this to be configured at setup, whereas others do this out of the box with minimal to no configuration.

Asset Lifecycle Services

Asset lifecycle is the set of services for creating, reading, viewing, updating, versioning, and deleting assets and metadata. It also includes locking of assets, such as check-in/check-out, and governing of “asset uniqueness.”

The system must provide asset lifecycle services. Without them, not much would happen. You need to determine what functions the system provides as part of these services:

• How does the system detect unique files and handle duplicate files?

• How does the system handle the versioning of assets and metadata?

• How does the system handle asset deletion, as opposed to asset removal?

• Does the system support a simple locking or version control mechanism, such as check-in/check-out?

• Does the system support asset or metadata placeholders, so you can store a job or create a collection before it contains assets?

• How does that placeholder support work?

• When and how does the system back up assets?

• How does the system restore these assets in the event of system failure?

• How does the system archive assets?

Asset lifecycle services tend to operate in the background, particularly if you work with the system through the DAM or from within a third-party application. If you plan or need to develop custom extensions to handle particular workflows or scenarios, access to these services via APIs or Web Services may be particularly useful.

Detecting and Determining File Uniqueness, Duplicate Handling

As discussed previously, ingestion focuses on the creation of new assets in the system. As such, the ingestion services need to work with asset lifecycle services. In particular, upon receipt of a submitted file, the system must be able to detect and determine whether the file is a duplicate of a file in the repository. If so, the system must determine what to do with the file and any associated metadata. For unique files, the system creates a unique identifier for the asset, which it keeps throughout the lifecycle.

The detection and determination are a bit of magic. Most DAM systems determine whether the file is unique by performing an algorithmic calculation that examines the file at a binary level and determines whether the mathematical result on the pattern of bits duplicates one it already has. The system typically performs a checksum calculation or calculates a hash (which you can think of as a kind of digital fingerprint), providing a value to compare with those already stored for files in the system. Many systems use the well-known MD5 hash algorithm. In theory, it is possible for two different files to give rise to the same checksum, but in practice, this is not something you need to worry about with MD5 (it’s only a concern with lesser algorithms). Still, you do need to know what happens when and if the system discovers a duplicate file, and you need to know in advance what you want it to do for your situation. Does the DAM system

• Ignore the asset?

• Ignore the metadata?

• Give notification that it is nonunique or is a duplicate?

• Version the asset?

• Create a new, duplicate asset with a different identification?

• Append the metadata?

• Overwrite the metadata?

• Version the metadata?

All of these are potential outcomes for handling duplicate files. Systems vary in their duplicate handling capabilities. You need to understand your workflows and how you use assets in order to align them with the various systems’ duplicate detection and determination process.

Asset Creation

Asset creation is another fundamental asset lifecycle service. It usually occurs after the system determines a file to be unique. Asset creation gives an asset a unique identifier. It may also invoke other services, such as allocating space for the file and the metadata; extracting, storing, and indexing the metadata; transforming to create predefined stored renditions; and performing any other initialization processing.

Some systems may provide flexibility on aspects of asset creation. For example, some systems allow asset or metadata “placeholders.” In instances in which the system only retrieves the metadata, this function creates a placeholder asset, which only has metadata, but there is no file associated with it; the file is added later. The reverse is also possible; the system creates the asset, but the metadata, or the bulk of it, can be married later.

With this flexibility comes increased power—and increased complexity. The system can handle a wider variety of workflows, such as allowing delayed or separate addition of assets and metadata. On the other hand, users must learn how to identify which asset(s) to marry with which metadata properly. They must also be aware that metadata may exist without files and files may exist without meaningful or searchable metadata.

Versioning

Upon duplicate detection, the system may create a new version of the asset. This implies that the system can maintain a lineage of historical snapshots of the asset and the metadata as they change. The asset’s identity doesn’t change; rather the system uses some mechanism to identify it and associate it with its various versions. Asset versioning facilitates creative workflows, tracking, and managing work in progress (WIP) among users. It also facilitates distribution workflows, in which different versions are distributed to people over time, or where one version is released while creatives work on the next version. Finally, it also facilitates historical views of the asset and its metadata over time, which has its own value (see Figure 4.1).

FIGURE 4.1
Creating a photo gallery is a common DAM scenario that requires versioning and multiple contributors. In a Serial Workflow scenario, versioning allows for rollback to a previous version or saving multiple resolutions of an image for publication to non-Web channels, such as print.

As with other content management technologies, versioning services vary from vendor to vendor. Some systems version the asset and its metadata as a unit. Some systems version just the asset; the existing metadata carries forward but can be changed. Other systems allow independent versioning of the asset and the metadata. A major version number indicates a change to the asset itself, such as a color correction to a photo; a minor version indicates a change to some metadata, such as changing the caption on the photo.

Systems also vary on version access. Some systems may present and provide access only to the latest version; others present and allow access to the entire history and any individual version. In both cases, security services typically govern access to a version. Lastly, systems vary on whether they allow purge or removal of obsolete versions. Many systems don’t allow deletion of versions because it can introduce additional integrity problems.

As with all features, you need to be clear on your needs for versioning. Specifically, what do your users need in their creative and distribution workflows and business processes?

Asset Removal and Deletion

The asset lifecycle also handles removal or deletion of assets. Most systems distinguish between removal and deletion/purging. Removal usually implies that the system removes the asset from most users’ view but retains it for administrator access. Deletion or purging means that the system has completely eliminated the asset and its metadata. Some systems have two-phase approaches. First, you mark an asset for deletion, which quarantines it. Then a privileged user—usually the administrator—deletes it.

Security services govern all users’ ability to remove and delete assets. Storage management services may reclaim and compress the now-unoccupied space. The distinction between removal and deletion and the two-phase approach provide less risk of accidental deletion and greater control over who can do what.

Version Control Via Simple Locking

Most asset management systems support a simple exclusive-lock mechanism called check-in/check-out or CI/CO. It allows you to check out an asset, which “locks” and gives you exclusive access to it and its metadata. Neither the system nor other users can change the asset while you have it checked out. Other users, however, can view and download the asset while it is checked out.

Version control is a useful feature for some creative workflows to guard against unwanted changes.

Of note, if you download a compound asset for editing, this function also locks the assets contained in the compound asset. If someone downloads an asset that’s part of a larger compound asset, you need to decide if that locks the compound asset as well.

DAM packages that manage asset lifecycles well have the following capabilities:

• They handle the creation, reading, viewing, updating, versioning, and deleting of assets and metadata.

• Multiple rights, roles, and permissions can be assigned at the asset level.

• The analysis of use, reuse, and shareability can be captured.

• An archive of the assets is present and available at the end of an asset’s lifecycle.

Media Processing

Media processing differentiates asset management systems from other content management technologies. Media processing takes multiple forms in a DAM system. It handles the changing, transforming, transcoding, extracting from, or inserting into the rich media file of the asset. It also deals with the automation of these processes, the integration of third-party tools to perform these functions, and the types of files these functions can support or process.

Proxy Creation

One of the critical functions of an asset management system is providing lightweight viewable renditions of the asset in an easily accessed format. This typically means creating and storing multiple individual renditions of the same asset at ingestion—for each version of that asset. This includes a representational thumbnail of the file content to identify and associate with the actual asset, as well as one or more other appropriate “proxies,” which are renditions that allow users to experience the asset easily and quickly, without accessing the often significantly larger master asset.

Transformation, Transcoding, and Rendering

Transformation, transcoding, and rendering are primary media processing capabilities that change assets from one form to another. Transformation refers to the processing of images, while transcoding refers to the processing of video, audio, or other time-based assets. These functions exist primarily to create a thumbnail, preview, proxy, or other renditions of the file, but they also produce on-demand output forms of a file for distribution or use. These three processes include several capabilities; they can do the following:

• Take an asset’s file and turn it into different formats, sizes, scales, file types, and resolutions.

• Create a new audio or video file with different characteristics, including format, resolution, encoding, bit rate, or frames per second. For example, they can create the low-resolution videos that can play in a variety of devices or in Microsoft’s Windows Media and Apple’s QuickTime formats over the Web.

• Create animations, such as animated GIFs, consisting of a series of renditions of pages from a document, brochure, or catalog; slides from a PowerPoint deck; or frames from a video, to a give quick visual flavor of its content.

• Separate layers in a multilayered photo or layout file for deeper representation of the asset. Conversely, they can combine layers into an output format.

• Segment a PowerPoint file into its individual slides, each as an asset, or further separate individual slides into their constituent parts, each potentially as an asset. Conversely, they can generate on-the-fly slides from assets or presentations in PDF, Flash, or PowerPoint formats from collections of individually selected slides.

• Generate appropriate output formats for mobile or handheld devices.

• Extract audio/text.

• Extract storyboards from video (see Figure 4.2).

• Extract content and instructions from digital signage control systems.

FIGURE 4.2
An extracted video storyboard from Oracle, which offers several lightweight DAM capabilities.

Metadata Extraction and Insertion

Media processing also includes all of the system’s metadata extraction and insertion capabilities. They include the ability to pull different and variously formatted metadata out of files and, in certain distribution scenarios, insert metadata into the file so that it stays with the file outside the system.

Third-Party Media Processing Tools

Asset management systems vary in their implementation of media processing. Do they make it available or configurable to users? What kind of media processing do they support? How adaptable is the system to adding support for processing continually emerging new media formats?

Most systems integrate third-party tools or engines to perform the work. They include but are not limited to

• Graphics conversion and metadata extraction tools.

• Video encoding.

• Transcoding and indexing tools.

• PowerPoint slide manipulation tools.

• XMP engines.

Some systems are more limited or flexible. You need to drill down into this area to ascertain that the system can do what you need it to do and that your users have easy access to those functions. As mentioned previously, it’s important to be sure that your DAM vendor has architecturally “anticipated” the ever-growing assortment of file types and provides a clear upgrade path for new utilities and file tools.

DAM packages that handle media processing well have the following capabilities:

• Modern-day file formats can be read and processed.

• A multitude of transcoding can take place.

• Metadata will travel with the digital assets and won’t be lost during the process.

Taxonomy Management

Finding information in your company is likely chaotic. A taxonomy is a great start to planning how assets should be classified and who should have access to each asset. A taxonomy should encompass the major touchpoints within your organization that represent how people need to find things. It likely encompasses the company organizational chart, with the roles of the individuals within the organization, including partners, clients, and stakeholders. It should factor in logical navigation by users and enable people to access information and content in common sense, logical ways. Like most things, change is constant, and taxonomies must adapt to change.

There are a number of differing approaches to managing taxonomy within DAM systems. Some products offer only a rigid way to tweak or change a taxonomy; others give administrators more flexible tools to change, import, and tweak. Some vendors seek to move the taxonomy away from the hierarchical navigation by using controlled vocabularies, collections, and relationships through the clever use of metadata. Other products allow for several different taxonomy structures depending on the business silo for which they are applied.

Metadata, Controlled Vocabulary, and Schema Support

Media companies strive to maximize the commercial potential of their assets and improve the quality of their programming. They want to repurpose, reuse, and enhance their content licensing revenues. Key to all of this is capturing the metadata associated with the assets. Enriching video assets by adding relevant metadata through the asset lifecycle increases the chance that relevant videos will be retrieved as required—without spending unnecessary time trying to locate the video assets from the library. We’ll examine different aspects of metadata management in greater detail in the following sections.

Metadata is data about data. Without it, your assets are essentially unmanaged. Metadata drives the security of these assets by way of rights, roles, and permissions. It drives the taxonomy, navigation, and the hierarchical structures of your data. Most of all, it drives findability, personalized experiences, and business intelligence.

Findability

If you or your users cannot find digital assets or aren’t aware of them, your DAM initiative will fail on multiple levels. The ability to find and reuse assets is a critical component of any DAM strategy. Vendors approach the area of search and findability in a variety of ways. Carefully review vendors against your search and find use cases and criteria. We explore this extensively in Chapter 5, “DAM Technology Services: Search, Retrieval, and Navigation.”

Establishing the Initial Information Model

Vendors typically provide a minimum set of services to assist your team in establishing the initial structure and organization of the asset management system’s information model. Most provide, or partner to provide, more—at a cost. Depending on the scope, breadth, and complexity of your deployment, this could entail significant consultation expense. However, it is well worth the effort to get the core of the information model right the first time. Regardless of how it’s created, the system needs to support the model you need. Aligning your needs with the information model is a critical step. Be aware that, whatever information model you establish as your initial model, it will change—and sometimes sooner than you expect it to.

More important than support of the initial model is understanding the underlying features of the system that support the development and maintenance of the information model, and their ability to support changes to the model.

The organization of assets and metadata is fundamental to both an asset management system and your application of the system within your enterprise. Successful asset management implementation requires organizing the metadata and the assets to fit the business processes and workflows of your enterprise, matching both the way your users currently work and the way your DAM team determines how the processes need to work.

Metadata management includes

• The definition and management of assets and metadata.

• The structure of predefined or evolving metadata—such as keywords, controlled vocabularies, or taxonomies—that can be defined, managed, and applied to assets.

• How assets are cataloged, tagged, ranked, rated, or annotated.

• How assets can be organized into groups, folders, or collections, or related to other assets, as in compound assets and configurations, and what can be done with and within these groups.

• The degree of flexibility administrators have in adapting or redefining the structure of the metadata model or schema.

Metadata Definition and Organization

A DAM system fundamentally must provide metadata definition and organization capabilities. DAM products vary in how flexible or limited these features are. A system can allow users to define a great deal of information:

• What information can be captured as metadata for each asset.

• The definition of metadata types or sets.

• The definition of metadata items.

• The grouping or clustering of metadata items into metadata types.

• The data type of the metadata item value.

• Whether structured data types are supported, such as an address consisting of street, city, state, and postal code.

• Whether the metadata item values are predefined, restricted to a controlled vocabulary, or free-form, user-provided text. If they are controlled, whether only one or multiple values can be selected.

• Whether a metadata item is required or optional.

• The default metadata item value(s).

• Whether the metadata item value is user provided or automatically extracted by the system. If so, how it is mapped between the extracted field or value and a specific metadata item.

• What implicit or explicit relationships exist between assets, including contains/is contained in, uses/is used by, derives from, parent/child of, sibling of, unidirectional/bidirectional, and how these relationships are implemented.

• What implicit or explicit relationships exist between metadata types and between metadata items, both on the same asset and on different assets, such as inherited metadata, multivalued fields, nested metadata, cascading metadata, conditional values, and rules.

• Whether multiple metadata types or sets can be applied to a single asset.

• Control of the visibility of metadata types, items, or values.

• How users can group or categorize assets and whether assets inherit metadata values from the group.

• The point in time at which the metadata is captured—synchronous with asset creation, ingestion, or delayed/later.

• If an asset is required or not (for example, you can have a metadata “record” without an asset).

• How and where the metadata is stored. Specifically, is it a literal database schema (that is, database tables are metadata records) or an abstract model implemented on top of a database, proprietary file system, or search engine?

• Whether they understand multiple taxonomies and polyhierarchy.

• Whether and how they can connect to an external enterprise reference repository.

In short, many possible subfeatures enable metadata definition. Because there’s so much variety in the structural support of metadata among products, you must know what metadata you need to capture and then deeply probe into what and how a product supports it. Systems may be limited in what they provide for metadata or asset relationship definition. This may limit your information model and restrict what you can capture and represent. For example, most systems may allow only one set of metadata per asset. This limitation can be problematic because, over time, the metadata that’s useful to some audiences may change or different use may require different metadata.

If you know how the system implements control over the visibility of an asset’s metadata (or whether it allows multiple sets of metadata to be applied to the asset), you can determine whether the system works for you. Otherwise, you may have to work around the system to capture the metadata you need over the lifetime of the asset.

In addition, how the system implements or stores metadata may lead to trade-offs. For example, a system that uses database tables as the metadata structure may provide great flexibility in the initial definition, but it may cost significantly more in terms of future modification and maintenance. It could require system downtime to change or evolve the metadata schema and migrate existing values to the new structure. A system that uses a more abstract model may provide easier maintenance and dynamic changes to the metadata schema—without downtime.

Dynamic Schema

Some systems can dynamically create the metadata schema and define new metadata types. You don’t need to stop or quit the system to add to or modify the schema. A system that supports dynamic schema creation can build the metadata model interactively—in front of your eyes—or read it from metadata definitions embedded in a file. This function can provide flexibility, adaptability, and power for maintenance and for working with files from other systems, partners, or organizations.

If the system supports both dynamic schema and XMP metadata extraction, you can leverage XMP to provide metadata about an asset, which the system could then use to create new assets dynamically. You can read more about XMP in the section “A Special Note About Standards” in Chapter 7, “DAM Technology Services: Architecture and Administration.”

Community Tagging

There are different kinds of metadata. Increasingly, asset management systems incorporate contemporary Web concepts of community and the collective view of the crowd to enhance collaboration and ease of use. One example of these concepts would be informal, bottom-up, evolutionary approaches that dynamically weigh keywords based on popularity, rather than the more traditional, formal top-down, predeveloped taxonomies. Other forms include community tagging (folksonomies), asset ranking, user-defined tags and categories, and comments or annotations.

Grouping Assets

All asset management systems provide mechanisms for grouping assets, and those groups are based on various types of metadata. Typically, the system supports named, hierarchical groups, folders, or collections, much like folders on the file system. (Note: We use the terms folder and collection interchangeably here.)

Some systems place an asset in one or more named folders or collections. In this instance, the system places a reference to the asset, rather than a copy of the asset. The system doesn’t duplicate the asset. Typically, the security system governs the privilege of creating new folders; for some users the hierarchy appears to be fixed, while it can be tailored for others.

Some systems allow collections to be dynamic, basing the content of a collection on an attached query or “saved search” performed at the moment the collection is opened. As a result, the collection content may vary from instance to instance, reflecting changes to the result set, and from user to user, illustrating the effect of access controls and potential visibility permissions on assets. These folder and collection facilities are useful for general organization and for restricting visibility or enhancing collaboration.

Permissions on folders can restrict visibility to assets. Some systems allow users to share folders privately with other users. This ad hoc collaboration and sharing of assets works well for project-centric workflows. Combining permissions and sharing may allow one user to put assets into a folder while others may only view them. This function suits DAM distribution applications in which approved marketing assets can be distributed for use in the field by putting them into a known location. If an asset can be placed in one or more folders, some systems can determine all of the locations in which that asset can be found. This capability is very useful for assessing an asset’s visibility and accessibility. You will need this feature in some form, so be sure to examine and inquire how the system provides, implements, and presents it through the user interface (UI).

For broadcast media scenarios, a media-centric asset management system needs to support collections or groups of different types of assets. For example, the still images, promotions, trailers, and the program video could be one collection. Another collection could be a TV series and different episodes within the series. A slightly different requirement is the ability to attach a document to a video asset, such as the script of the program, the rights information, or some other related document. The use cases may differ, but the DAM system’s ability to support asset collections makes it more intuitive for all users.

Implicit and Explicit Metadata

Metadata is added to the asset while it is being ingested and any time after that. We can think of two types of metadata in the DAM world: implicit (sometimes also called technical metadata) and explicit or descriptive metadata.

Implicit or technical metadata (as the name suggests) refers to the technical characteristics of the asset itself (such as format, size, and storage location). Implicit metadata does not require a human to define it because the system captures or creates this technical metadata automatically and adds it to the asset during the ingest process itself.

Descriptive or explicit metadata refers to the details or “about-ness” of the video clip and any other information that is deemed useful about the content of the video. This information is explicit because usually human beings are required to define it. Some of this information is manually entered by users at different times using a data entry form. The different fields and metadata vocabularies available in this form are usually decided at implementation time and configured accordingly. Consider what sorts of explicit, descriptive metadata will help your users find assets more easily and incorporate such information into your retrieval plan.

Controlled Vocabularies and Thesauri

To enforce discipline when categorizing content (for example, assigning tags to an asset), systems can enforce a controlled vocabulary by allowing users to choose from a preset list of categories or tags. The admin users have permissions to add or edit the controlled lists while others just make use of them to ensure consistent asset classification throughout the organization.

Certain vendors support the thesaurus functionality. Here, they make use of the “relatedness” concept to improve the relevance of search results. By creating a hierarchy of categories (such as country-state-city), you can broaden or narrow the searches. Figure 4.3 shows an example.

FIGURE 4.3
The Imagine Communications Thesaurus Editor allows you to enter broader, narrower, and related concept tags associated with a given term.

Automated Metadata Extraction

Certain rich media assets may already have other useful descriptors available like subtitles or captions. Some systems permit subtitle imports into the database and use them to create the index for search. Similarly, speech accompanying the videos can be converted into text using speech-to-text plug-ins and then can be used for search. Of course, the utility of that functionality is determined not so much by your asset management system, but by the efficacy of the plug-in provider.

Another important issue to note is the extent of the support for multilingual metadata. Users should be able to enter metadata in different languages and search accordingly.

As content increasingly is ingested from mobile journalists, the geospatial data associated with it is also captured as an important piece of metadata. This is particularly the case for video analysis in military applications and increasingly for law-enforcement or sports-oriented tracking scenarios.

Asset Relationships

While the preceding metadata aspects pertain mainly at the individual asset level, we should also consider the asset collections and associations between assets.

An asset is usually available in different formats, resolutions, and even versions; only one instance is generally considered to be the gold copy. The system maintains the relationships between the different asset avatars (renditions) and uses the relevant asset based on context and purpose. For instance, when a user is browsing search results, a low-resolution version of the asset is displayed. However, when the same asset is delivered to the editors for finishing touches, the high-resolution version of the same asset is available. Similarly, if an asset is being published to the Web, the appropriate format of the asset is distributed.

Another take on this is the genealogy of assets; here, we reference the parent-child relationships of a video and subclip, or derivative clips from the parent video. The system should keep track of such genealogy for many purposes, including rights management.

Enterprises often want to establish relationships among assets. These can reflect real, physical relationships, or establish a relationship that enhances the inner representation of the information from an information-management perspective.

Systems vary significantly, both in the kinds of relationships that they can support and how they implement and present these relationships to users. Some systems provide higher-level structures that represent a prepackaged form of a specific relationship. For example, a “compound asset” acts as a container or parent to other contained or child assets. Other systems provide lower-level mechanisms in which you establish specific relationships and build higher-level structures. For example, a system may provide parent-child, sibling, unidirectional, or bidirectional links that you use to relate assets and build more intricate structures of related assets. A few systems provide both. As with all the other features, you need to understand your use cases, the information model you need, what the products offer, and how they implement their model.

Compound Assets

Compound assets result from the ingestion of compound files. As previously described, compound assets consist of or contain other assets. Generally, compound assets consist of master assets and contain or reference other files or assets, some—or all—of which may be compound.

Asset management systems vary in how they support, represent, handle, and implement compound assets. You will encounter many subtleties to compound assets in the course of examining your workflows around the creation, alteration, and presentation of compound assets. For example, systems vary in how they manage changes to compound assets. Some systems perform deeper decomposition of the ingested compound asset, such as taking apart an InDesign document or separating layers. The difference lies in what the asset management does with the embedded or referenced files in the document or with each individual layer:

• Does it make each an asset or keep them as some lesser, nonasset entity?

• Does it make them visible only within the context of the master, or can they be reached on their own?

• What does it establish for metadata for the embedded or referenced file? Or for the separated layer?

• Does it have its own metadata independent of the master? Does it have none? Or does it inherit most or all of it from the master?

• Does it establish a reference or link to the file as a placeholder if the file is not included in the upload?

• How does it handle changes to these files?

• What does it do with respect to the versioning of both the file itself and the containing master file?

With respect to PowerPoint files, the questions are similar: Does the system treat the file, each individual slide, or individual elements within the slides as assets?

You need to understand these subtle—but critical—distinctions because they fundamentally affect your information model, workflows, and experiences with the system.

Related Assets

In some cases, users want to create a bunch of “related or associated assets.” For instance, when you select one asset, the system selects all of the other assets that are linked to it in some relationship, showing you that these assets are related and how to navigate through them. Examples include the following:

• A press kit that includes three data sheets, a company backgrounder, a FAQ, and the latest press releases.

• A package of assets for display on a specific Web property, such as three different encodings of a video clip, two different still images representing the video clip, and two different approved captions for the clip.

• Related images of an item for a given model number, such as an image of a faucet that (depending on the finish of the faucet) corresponds to different SKUs in a catalog. Each has different metadata in the DAM, yet they all correspond to the same model of the faucet.

Asset management systems can provide one or more prepackaged structures, end-user features, or lower-level definitional mechanisms that establish links or relationships among assets. Common mechanisms include parent-child links, sibling relationship, unidirectional, bidirectional, one-to-many, many-to-one, many-to-many relationships, or links. With these mechanisms, you can establish more complex relationships and structures of related assets. An administrator usually performs this task when defining the information model.

Configurations

In an asset management system, the term configurations describes the snapshot at a point in time of a bunch of related assets or related asset versions. Other content management systems might call these editions, or products, or simply snapshots.

Configurations come into play when you create an asset containing or relating to multiple existing assets and you want to capture the state of these relationships at significant points in time. For example, a brochure may include several photos, copy (text), graphics, and logos.

• What happens when you update an included photo and the photo asset is versioned? When you view the brochure, which photo version is presented?

• Can you see the former configuration as well as the latest, or only the latest?

• Can you go back to any point in history and see the configuration at that point in time? Can you, for example, see either the released configuration or the one you didn’t choose?

• How does the system handle changes to related assets or compound assets?

• Does it even provide configurations or configuration management features?

• Can these configurations of assets be managed by the system? If so, how?

• How does the system support configurations?

• Does it make them visible and easily understandable through the user interface?

Be sure to research this area diligently if it is important to your workflow. You may also want to share any “mission-critical” workflows with the vendor, such as an automated workflow with a postal office or printer.

Schema Change

The last area of organization concerns the degree of flexibility administrators have to adapt or redefine the structure of the metadata, such as the metadata schema or metadata taxonomy. Every asset management system allows you to define the initial scheme. That’s great, but you have no assets at that point and no metadata. An absolute empirical rule of an asset management system is that the metadata organization or taxonomy will change, and you may need to change it much sooner than you expected.

Choosing a system that flexibly supports change will be critical to your success. Determine what kinds of metadata changes the system supports easily (that is, out of the box) without additional coding. Understand whether changes to metadata organization require you to bring down the system or can be done dynamically. Determine how the system migrates metadata from the old structure to the new, and find out if search indexes are affected by the changes.

It’s a lot to digest. DAM packages that handle taxonomy management well have the following capabilities:

• Taxonomic structures are present and are used to group similar assets or projects.

• Users can navigate using taxonomic structures.

• Taxonomies are used to add parent categories on metadata entry (supports structured categories).

• Assets can be assigned to multiple taxonomies.

• Taxonomic structures form an integral part of the DAM architecture and are universal across search, data entry, and navigation.