Cisco provides tools that can make managing your Cisco CallManager system simpler and faster. The Bulk Administration Tool (BAT) helps you perform add, update, export, and delete operations on a large number of users or devices through a single transaction. The CDR Analysis and Reporting (CAR) tool helps you manage billing records, among many other tasks. In addition to these manageability tools, there are several monitoring tools. Monitoring your system with the tools described in this chapter can help you make sure your system is running efficiently and pinpoint the cause of problems when they arise. They can also be used to help you plan for expansion and ensure quality of service (QoS). The following products are discussed in this chapter:
Bulk Administration Tool (BAT)
CDR Analysis and Reporting (CAR)
Cisco CallManager Serviceability
Real-Time Monitoring Tool (RTMT)
Microsoft Performance
Trace Collection Tool (TCT)
Event Viewer
Terminal Services Client
Virtual Network Computing (VNC) Viewer
CiscoWorks IP Telephony Environment Monitor (ITEM)
Simple Network Management Protocol (SNMP) Management Information Bases (MIB)
Cisco Discovery Protocol (CDP)
Voice Log Translator (VLT)
The shaded blocks in Figure 6-1 illustrate the software layers and blocks within CallManager that contain manageability- and monitoring-related functionality. Unlike the other block diagrams in this book, Figure 6-1 is completely shaded. Because virtually every component in CallManager logs alarms, constructs CDR information, receives provisioning information through BAT, logs trace information, or sets CallManager performance counters, not a single CallManager layer or block escapes manageability and monitoring.
Manageability tools ease the burden of managing a large system. BAT and CAR are discussed in this section. Using these tools, you can perform bulk transactions, where a large number of users or devices are added, updated, or deleted from the CallManager database. In addition, you can run reports on the system to generate information about billing, traffic, gateways, and QoS statistics.
BAT lets you perform bulk add, update, and delete operations on the CallManager database. This means that for large systems, you can configure or update your CallManager database faster and with less manual entry. Different versions of BAT provide different features. This discussion focuses on BAT release 5.1, which is included with CallManager release 4.1.
With BAT release 5.1, you can perform the following bulk operations on the CallManager database:
Add, update, and delete Cisco IP Phones, VG248 ports, computer telephony interface (CTI) ports, and H.323 clients
Add, update, and delete users
Add, update, and delete user device profiles
Add, update, and delete Cisco IP Manager Assistant (IPMA) managers and assistants
Add, update, and delete ports on a Cisco Catalyst 6000 FXS Analog Interface Module (WS-X6624)
Add and delete Cisco VG200 analog gateways and ports
Add, update, and delete forced authorization codes (FAC)
Add, update, and delete client matter codes (CMC)
Add, update, and delete pickup groups
Export IP phones, users, and user device profiles
Create reports on IP phones, users, IPMA managers and assistants, user device profiles, and VG200 gateways
Using BAT templates in combination with comma-separated value (CSV) files that you create, you can effect basic database changes that previously required labor-intensive manual entries in CallManager Administration. With BAT, instead of adding 100 phones one at a time in CallManager Administration, you can create a BAT template and a CSV file (both of which are reusable for future bulk add transactions) and add the same 100 phones in one transaction. BAT includes an Excel template spreadsheet where you can enter data and create the CSV files automatically using macros in the spreadsheet. You should always use this Excel template, which is validated by BAT, for generating the CSV files to ensure they are properly formatted.
To help ensure against performance degradation on CallManager, BAT provides a formula to compute roughly how long it takes to complete a bulk transaction. This is useful information to know before you perform the transaction, because you can determine whether system performance will be impacted. If performance is an issue, perform the bulk transaction when the system is not heavily used.
BAT is available as a plugin application to CallManager Administration (Application > Install Plugins), and you must install it after installing CallManager. New versions of CallManager do not automatically upgrade BAT, so you should always run the BAT plugin installer after any CallManager upgrade.
Note
Depending on the number of records you are adding, updating, or deleting to the CallManager database during the bulk transaction, BAT can consume a great deal of system resources. Therefore, Cisco recommends that BAT be used only during off-peak hours to minimize the impact on CallManager performance.
BAT’s goal is to save you time by reducing repetitive labor-intensive manual data entry tasks. This goal remains the same whether you are installing a new CallManager system or already have an existing system. BAT also combines tasks by allowing you to add users and devices (phones, gateways, or CTI ports) and associate users with their devices, all in one bulk transaction. BAT can also be very useful for adding or deleting FACs, CMCs, or setting up IPMA users.
BAT is most useful when you are first setting up a system. The information you provide in the BAT template and CSV file (described in the section “CSV Files”) can be as specific or generic as you want. If the information is generic, you can always update the device or user information in CallManager Administration later.
A common problem with new systems is misconfiguration. This problem can result because of incomplete understanding of how a feature works; often it is simply a data-entry mistake. With BAT, if you add devices, for example, and then realize after doing so that there is an error in the device configuration, you can simply modify the same CSV file to correct the error. You can then use it and the same BAT template to update the same set of devices. In this example, you would have to delete the misconfigured devices first before you could add them again, but you can do that using BAT, as well.
You might find that you want to restrict long distance dialing on certain phones in your company, such as lobby phones, conference room phones, or other phones that are located in common areas. You can use BAT to add or update that set of phones with a calling search space that blocks long distance dialing.
BAT also proves useful when you have an established system for which you need to add a new block of users or devices or make the same update to many devices or users. For example, if you plan to delete a device pool, all phones currently using that device pool must be updated with a new device pool before the old one can be deleted from CallManager Administration. You can quickly update all phones using the existing device pool with the new one by running a bulk update on all phones using the old device pool.
Perhaps you want to change the voice mail access number, add a new voice mail access number, or add a speed dial to the Human Resources department for all phones. BAT lets you update all phones that have a common characteristic, such as device pool, location, or calling search space.
In another example, you could set up all the phones used by salespeople in your company with a special set of Cisco IP Phone services. Phone services are Extensible Markup Language (XML) services that you can configure using the Cisco IP Phone Productivity Services Software Development Kit (see Appendix A, “Feature List,” and Appendix B, “Cisco Integrated Solutions,” for more information). Perhaps you have services that tie in directly to a database showing current sales goals or a list of reference account contacts. Although this information is critical to salespeople, it might not be of much interest to anyone else in your company. You can add a specific set of phone services to phones belonging to the sales staff so that only their phones provide this service when the services button on a Cisco IP Phone is pressed. Likewise, if you have a common service that you want to appear on all phones in your company, such as your company’s stock quote or a calendar showing paid holidays, you can specify this when adding the phones in BAT. As of mid-2005, BAT cannot be used to bulk add XML services that require parameters such as username and password. This means that services such as Personal Address Book (PAB) and the Fast Dial service cannot be added in bulk using BAT.
Figure 6-2 shows the Phone Options screen (Configure > Phones / Users) where you can perform various functions such as insert, update, and delete phones; export phones; add and update lines; reset phones; insert phones with users; generate phone reports; and perform bulk CAPF operations.
BAT’s strength lies in its ability to allow you to make large device or user changes to the database without compromising customization of the devices or users. Using the CSV file, you can specify as much detail as you like about the devices you plan to add. For example, for phones you can supply directory numbers, Media Access Control (MAC) addresses, call forward no answer directory number, and more. After you have entered all the data in the CSV file and have created a BAT template, you can insert the data from CSV files and BAT templates into CallManager Administration. When the phones are plugged into the network, they register with CallManager and find their device settings based on the information inserted into the database from the CSV file.
You can use the sample Microsoft Excel spreadsheet template (available on the Publisher at C:CiscoWebsBATExcelTemplate) to create the CSV file for each type of bulk transaction. A different tab is provided for users and each type of device: phones, gateways, gateways ports, and so on. When you first open the Excel spreadsheet, select the kind of device you want to create (phone, CTI port, H.323 client, and so on). For some operations you then click the Create File Title or Create File Format button, which prompts you to select the device and line fields you want to insert. Previous versions of BAT were not as flexible and required you to have all the available fields as columns in your spreadsheet.
After you have entered the data in the spreadsheet, click the Export To BAT Format button in the spreadsheet. A dialog box displays the default filename and location to which the file will be saved. The default filename is Filetype#timestamp.txt (e.g., PhonesUsers#06272005093545.txt). The filename indicates that this file was created on June 27, 2005 at 9:35:45 a.m. You can overwrite the default name with something more memorable, such as June_new_hires.txt. You should save CSV files in the appropriate subdirectory of the C:BatFiles directory on the Publisher where BAT is installed. A subdirectory exists for each type of BAT import operation. For example, there is a subdirectory named C:BatFilesPhonesUsers where you store CSV files for phone and user import operations.
In the CSV file, you can specify individual MAC addresses for each phone or use a dummy MAC address. If the dummy MAC address option is used, the address can be updated later using the Tool for Auto-Registered Phone Support (TAPS).
TAPS is an optional component of BAT that requires Cisco IP Interactive Voice Response (IP IVR) on a Cisco Customer Response Applications Server. You can also install TAPS on a CallManager server with Cisco Extended Services installed. You must enable auto-registration in CallManager for the TAPS feature to work. With TAPS, you can leave the MAC address blank in the CSV file and import this data using BAT with the Create Dummy MAC Address option selected. Later, when a Cisco IP Phone is plugged in, the user can retrieve device information for the new phone simply by dialing a TAPS directory number and entering his or her phone number. This is particularly useful when you want to configure devices for a group of users where a physical phone has not yet been assigned to the user. TAPS also proves useful when an existing user replaces his or her phone because of damage or defect. When users receive the new phone (same model), they can simply dial the TAPS directory number and then their current directory number, and TAPS downloads device information configured for the previous phone to the new phone.
Another important function of BAT is the capability to bulk update Locally Significant Certificates (LSC) installed on your IP phones. These certificates are necessary if you are using the device authentication or media encryption features introduced in CallManager release 4.0.
Without BAT, you would have to go to each phone in CallManager Administration and set the Certificate Authority Proxy Function (CAPF) configuration to update the certificate on the phone. With BAT, you can tell the CAPF service to issue new certificates to all phones of a particular model (for example, all Cisco IP Phones 7960). This is especially useful for phones that do not come from the factory with a Manufacturing Issued Certificate (MIC) and therefore require an LSC before device authentication and encryption can be used.
Detailed information about BAT is available at the following location:
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/admin/bulk_adm/index.htm
CAR, a web-based application, provides reports about QoS, gateway usage, traffic details, user call details, and more. CallManager stores information about each call in call detail records (CDR) and call management records (CMR), collectively known as CDR data. This CDR data serves as the basic information source for CAR. This discussion focuses on CAR release 4.1, which is included with CallManager release 4.1.
CAR is available as a plug-in application to CallManager Administration (Application > Install Plugins). Once installed, you can access CAR from CallManager Serviceability under Tools > CDR Analysis and Reporting.
CAR helps you obtain system capacity and QoS statistics by generating reports that give information about call activity and voice quality. CAR generates reports on the performance of the gateways and is also useful to associate calls to users and reconcile phone billing with usage. CAR can also be used for troubleshooting. For example, if several users report busy signals when dialing in to retrieve voice mail messages, you can run a voice mail utilization report in CAR to see whether all ports on the voice mail server are busy during peak usage hours. The information in this report can help you determine whether to add more ports to your voice mail server.
CAR provides reporting features for three levels of users:
CAR administrators
Managers
Individual users
CAR administrators can use all of the features of CAR. Managers can monitor call details for the various groups and individuals in the company. Individual users can view details about their calls. Numerous reports are available in either detail or summary format.
CAR administrators can use CAR to monitor QoS. For example, reports generated in CAR help you detect QoS problems in the system and, to some extent, diagnose and isolate QoS problems. Note that QoS statistics are available only from devices that support sending CMRs to CallManager. At this time, only Cisco IP Phones and MGCP gateways have this functionality.
Other reports provide metrics on traffic, system overview, gateways, voice mail utilization, conference bridge utilization, Cisco IP Phone services, and more. Managers can use CAR to generate reports about usage for their department or select users to view top usage by cost, call duration, or number of calls. These reports can be useful to keep watch over expenses or to determine ongoing budgeting for departmental phone usage. Managers can also run detailed reports that help determine whether any unauthorized calls have been made by their department or select users. Individuals can use CAR to generate summary or detail reports for calls they made, which can prove useful for tracking phone numbers and call duration and for billing purposes.
Note
Users must be given the URL for CAR before they can access the system. Authentication is handled by the user ID established in the Global Directory in CallManager Administration. See the CAR documentation for more information.
Figure 6-3 shows the QOS Summary screen in CAR (System Reports > QoS > Summary). In this screen, you can specify the types of calls to include in the summary report and the timeframe that the report examines.
You can also view monthly summary reports by selecting the report in the Available Reports list box (shown in Figure 6-3). Available Reports are reports automatically generated periodically according to the CAR configuration. Figure 6-4 shows an example of a QoS monthly summary.
Note
CAR consumes a great deal of system resources when generating reports. Therefore, Cisco recommends that CAR be used to generate reports only during off-peak hours to minimize the impact on CallManager performance. By default, report generation is configured to run during the night, when system utilization is typically low.
CAR provides all the reports already discussed plus access for the three levels of users. CAR enables you to schedule automatic generation of reports, the time that CDR data is loaded into the system, and CAR database maintenance. You can also set up alerts to notify you when certain conditions occur. The following sections describe the CAR features:
Loading CDR Data
Automatic Report Generation
Reports
CAR Database Maintenance
Alerts
Descriptions of these features follow.
CAR enables you to specify when CDR data (CDRs and CMRs) is loaded into the CAR database. You can schedule the loading of data at the nonpeak hours of CallManager. CAR uses data in the CAR database to generate reports, so data in CAR will be current only up to the last time CDR data was loaded into the CAR database.
CAR only loads CMRs for records that have an associated CDR. To get complete CDR data, ensure that both the CDR Enabled Flag and Call Diagnostics Enabled service parameters (Service > Service Parameters > select a server > Cisco CallManager) are set to True. CAR only purges CMRs if they are associated with CDRs, which means that if you set the CDR Enabled Flag service parameter to False and the Call Diagnostics Enabled service parameter to True, CallManager will continue writing CMRs to the hard drive until all available disk space has been depleted.
If you are using third-party software that reads CDR data from the CallManager database, ensure that it either deletes both CDRs and CMRs or does not delete CDRs, allowing CallManager to automatically purge the CDR data. Failure to delete CMRs associated to CDRs will result in disk space depletion which could eventually crash CallManager. Future versions of CallManager will prevent this condition from occurring.
CAR allows reports to be generated automatically at a user-specified time, which results in reports being automatically generated and stored for future use. You can view these reports quicker than reports that are generated on demand. Automatically generated reports can also be e-mailed to administrators after being generated.
CAR reports can be scheduled to run automatically or to be used on demand to track incoming or outgoing call quality, overall system performance, individual or group call usage (such as cost or duration of all calls), and gateway usage details, among many other functions. CAR reports display in either PDF form (using Acrobat Reader) or CSV format. Reports generated by CAR include the following:
Individual/department bill reports
Top N calls (by cost, duration or number) reports
Cisco IPMA usage reports
CTI application user reports
Cisco IP Phone services reports
Traffic summary reports
CDR error reports
Gateway reports
Route plan reports
Conference bridge reports
Voice messaging reports
QoS reports
System overview reports
FAC/CMC reports
Malicious call detail reports
Precedence call summary reports
CDR search reports
Reports can be generated for viewing, printing, or e-mail distribution to interested parties by clicking the Send Report button in CAR.
Individual/department bill reports (User Reports > Bills) provide information to enable users and managers to monitor their own or their department’s calling records. Reports about user calls can be sent to relevant personnel (managers, high-usage users, and so on) to inform them of possible anomalies in their usage patterns.
Administrators can generate a report detailing the users who have a CTI port enabled (User Reports > CTI Application User). They can also generate reports for Cisco IP Manager Assistant (IPMA) usage (User Reports > Cisco IPMA).
The Cisco IP Phone Services report (User Reports > Cisco IP Phone Services) enables administrators to view the number of users subscribed and the percentage of all users who are subscribed to each IP Phone service configured on the system. The report does not show how many times each IP Phone service is actually used.
Top N (where N represents “number of”) reports (User Reports > Top N) are used to analyze the calls based on destinations, users, and calls. Call reports can be generated by charge, duration, or number of calls. The various call reports are as follows:
Top N users in the organization or group who have incurred maximum charge or have used the phone for the maximum duration
Top N destinations to which the organization or group has incurred maximum charge or spent maximum time
Top N calls from the organization or group that incurred maximum charge or were for the maximum duration
CAR administrators or managers can generate these call reports.
The traffic summary report (System Reports > Traffic > Summary/Summary by Extension) displays network usage patterns on an hourly and daily basis. This report helps you determine whether too much or too little equipment is deployed on the network. Only the system administrator can generate these reports.
Gateway reports (Device Reports > Gateway) provide gateway traffic details to help system administrators analyze the performance of gateways. The following information describes some of the available gateway reports:
Gateway detail—. Shows the performance of the various gateways in the enterprise and provides the date, origination time, termination time, duration, origination number, destination number, origination codec, destination codec, origination IP, and QoS of calls that used the gateway. It can be generated on demand by specifying the call classification, QoS grades, gateways, and date range. The report provides a list of calls that used the specified gateways. Gateways can be specified by type or by only those gateways that use a particular route pattern.
Gateway summary—. Shows a summary of the performance of the various gateways. It presents a matrix of the number of calls of various call classifications and QoS through a gateway. It also gives the total number of calls and the duration under each of the categories. The report is scheduled for generation every month but can also be generated on demand by specifying the call classification and date range.
Gateway utilization—. Provides an estimate of the utilization percentage of the gateways. You can examine the usage based on each hour of a day or by days of the week or month. Reports generate for each selected gateway. This report is most useful for capacity planning.
CAR offers a variety of route plan reports (Device Reports > Route Plan). The following route plan reports are available:
Route and line group utilization
Route and hunt list utilization
Route pattern and hunt pilot utilization
Each of these reports provides an estimate of the utilization percentage of the route plan component. As with the gateway reports, the utilization can be displayed on an hour-of-day, day-of-week, or day-of-month basis.
Two reports are available for conference bridge resources (Device Reports > Conference Bridge).
Conference bridge utilization—. Enable you to do capacity planning for your conference bridge resources by showing the utilization of each of your conference bridge resources by time-of-day, day-of-week, or day-of-month.
Conference call details—. Shows information about all conference calls created during the report period. This report is available as a summary report or detailed report. The detailed report includes a listing of each participant and call detail information of each conference call, but the summary report does not.
The voice message report (Device Reports > Voice Messaging) enables you to determine the estimated usage of your voice mail ports, which proves useful for capacity planning to determine whether you have enough voice mail ports to meet expected demand.
You can use CAR to gather information on voice quality and manage CallManager QoS statistics and system capacity (System Reports > QoS). Calls are categorized into a voice quality category based on the information in the CMRs and the QoS parameters you provide. QoS reports provide information about the quality of calls for all phones in the network, which helps you determine whether any possible issues exist in the network. Reports include codec type, packets lost, jitter, and latency (although latency is not calculated by any endpoints at this time and will therefore always show zero). The administrator can specify the criteria to classify the QoS as Good, Acceptable, Fair, and Poor based on the reported statistics from an IP phone or gateway.
The QoS reports available are as follows:
QoS summary—. Helps managers and administrators analyze CallManager performance. The report can be scheduled for automatic generation every month or run on demand for a specified date range. It provides a pie chart that shows the distribution of the QoS grades that are achieved for the specified call classifications and periods and also provides a table that summarizes the calls for each QoS rating.
QoS detail—. Provides analysis of the voice quality grades achieved for calls. Use this report to analyze CallManager performance for a particular user or a group of users. The report provides the origination time, termination time, duration, origination number, destination number, call classification, origination codec, destination codec, origination IP address, origination span, and QoS. You can generate a report on demand by specifying the call classification, QoS grades, date range, and user(s).
The system overview report (System Reports > System Overview) provides a composite report consisting of some of the reports. This report gives a broad picture of the overall system performance, and it can be scheduled for automatic generation every month or generated on demand for any selected date range.
Three reports (System Reports > FAC/CMC) are associated with the forced authorization codes (FAC) and client matter codes (CMC) features. These features help you manage call access and accounting. FAC regulates the types of calls that certain users can place, and CMC assists with call accounting and billing for calls that relate to billable client matters. The three reports are as follows:
Authorization code name—. Provides the originating and destination numbers, date and time that the call originated, call duration in seconds, call classification, and authorization level for calls that relate to each chosen authorization code name
Authorization level—. Provides the same information as the authorization code name report for each chosen authorization level
Client matter code—. Provides the originating and destination numbers, date and time that the call originated, call duration in seconds, call classification, and the client matter code for each chosen client matter code
The malicious call detail report (System Reports > Malicious Call Details) lists all calls marked as malicious calls by the Malicious Call Identification (MCID) feature. This report is blank if the MCID feature has not been enabled or if no malicious calls have been flagged by end users.
The precedence call summary report (System Reports > Precedence Call Summary) displays a call summary for all precedence calls made when using the Multilevel Precedence and Preemption (MLPP) feature. The call summary is presented as a stacked bar chart showing the number of calls for each precedence level. The report also provides an overall percentage distribution for each precedence level.
CDR search enables you to view CDR data from the CallManager database based on a specified date, gateway, extension number, user, cause for termination, precedence level, or malicious call designation. The results display the CDR data fields for records that match the selection criteria. You can also use CAR to export the raw CDR data to a CSV file for processing by an external application.
You can configure CAR to notify you when the CAR or CDR database is reaching capacity (System > Database > select the alert for CAR or CDR). You can then manually purge the selected database (System > Database > Database Purge). You can also schedule automatic purging of records older than a specified number of days in the CAR database. You should periodically purge old database records to ensure you do not run out of hard drive space. Also note that after performing a purge, you could perform a system backup using the Backup and Restore System (BARS) utility to ensure the database transaction logs are also truncated.
You can configure CAR to alert you when any of the following conditions occur:
CAR or the CDR database size exceeds a limit you specify.
The percentage of good calls drops below a specified range or the percentage of poor calls exceeds a specified limit.
A user exceeds the daily charge limit.
Alerts are sent to the e-mail ID you specify when configuring the alert.
Detailed information about CAR is available through the online help documentation contained within the CAR application or at the following location (Cisco.com login required):
This section describes the tools that you can use to monitor CallManager and your network:
CallManager Serviceability provides alarms, traces, component version information, service activation and control, Quality Reporting Tool reports, and performance reports of Cisco IP Telephony components in a CallManager cluster. CallManager Serviceability is a web-based application installed with CallManager and accessible from CallManager Administration by clicking Application > Cisco CallManager Serviceability. You can access CallManager Serviceability by browsing to the CallManager server by IP address or Domain Name System (DNS) host name, if applicable.
CallManager Serviceability, shown in Figure 6-5, provides the following features:
Alarm configuration and detailed alarm definitions
Trace configuration, analysis, and collection
Component version information
Service Activation for adding and removing services
Control center for starting and stopping services
QRT Viewer for viewing IP phone problem reports
Serviceability Reports Archive for viewing archived serviceability data
The CallManager system generates alarms to notify you of various events. The alarms are categorized into different levels, as described in Table 6-1. As of CallManager release 4.1, only error levels Debug and Error trigger alarms. You can configure the system to have alarms forwarded to selected monitors, and for each monitor, you can also choose to capture alarms of certain levels. These settings are configured in the Alarm Configuration screen in CallManager Serviceability for each service (such as Cisco CallManager, Cisco TFTP, and so on). Alarm Configuration also enables you to apply a given configuration to all nodes in the CallManager cluster. The monitors to which alarms can be forwarded include the following:
SDI trace log (also known as CCM trace)—. Display the alarms in the CCM/SDI trace files.
SDL trace log (Cisco CallManager alarms only)—. Display the alarms in the SDL trace files.
Event Viewer—. View the alarms in Event Viewer; see the section “Event Viewer” later in this chapter for more information about viewing CallManager alarms in Event Viewer.
Syslog—. Specify the IP address of a syslog server to receive the messages.
Table 6-1. Alarm Event Levels
Level | Description |
|---|---|
Emergency | The system is unusable. |
Alert | Indicates a condition that warrants immediate action. |
Critical | Indicates a critical condition. |
Error | Indicates an error such as an unregistered device or a CallManager failure. |
Warning | Indicates a warning condition. |
Notice | Indicates a normal but significant condition. |
Informational | Provides information messages only. |
Debug | Provides detailed messages for use in debugging by Cisco engineers. |
You can view detailed alarm descriptions in CallManager Serviceability. Click Alarm > Definitions to view these definitions. A simple search dialog box is provided that allows you to search for and view alarms, their descriptions, and recommended actions. After a search has returned a list of alarms, you can click any part of the alarm description to see detailed information about that alarm. The detailed information also includes an explanation of any reason codes that are given as part of the alarm. For example, an alarm generated by a transient device will have a reason code that explains why the device is considered a transient device, which can help in diagnosing and solving problems in the CallManager system.
Figure 6-6 shows the Alarm Detail screen after a specific alarm (in this case, SDLLinkOOS) has been clicked from the Alarm Message Definitions screen. You can view detailed alarm information for any alarm by clicking it from the Alarm Message Definitions screen.
Traces are diagnostic tools that can help you determine the cause of a problem. Running a trace can generate information you might need to identify or isolate the source or symptoms of a problem. The path to problem resolution becomes much simpler after you can point to the cause of the problem.
There are two kinds of traces:
SDI trace, which is more commonly known as CCM trace
SDL trace
Note
To generate good, usable trace information, all clocks should match on all CallManager-related devices. CallManager automatically installs the Network Time Protocol Daemon (xNTPD) for time synchronization. xNTPD provides a consistent time for all devices that poll it. This results in trace data that accurately reflects a single time across the system.
SDI traces (also known as CCM traces) are useful for diagnosing most problems you might encounter with the CallManager system. SDI traces log different types of runtime events related to CallManager—including device names, their IP addresses, alarms, and other general information—to help you determine the origin of the problem. You can specify a set of devices for tracing so that the trace log contains only events that originate from the selected devices. The Cluster ID and nodeID appear in the trace files to help you determine which trace files belong to which node of the cluster.
For example, a user reports that a dialed number results in reorder tone. You can turn on SDI trace and trace only the phone and gateway involved to learn why CallManager cannot connect the user’s call. This could show a problem with a conflicting route pattern that might be preventing the dialed number from being routed. In another example, a user reports that calls are being dropped. You can turn on SDI trace on the gateways to help identify the reason that the call is dropping and determine whether the problem lies within the CallManager system or in the Public Switched Telephone Network (PSTN).
You can choose to log SDI traces in XML format or in standard text-based format. When XML trace formatting is selected, the resulting trace logs can be used with Trace Analysis feature; however, Cisco recommends leaving trace levels set to text-based format for readability reasons.
Note
SDI tracing is enabled by default; however, the default logging level is Error, which provides limited debugging information beyond critical system failures.
SDI traces generate files (for example, ccm000000000.txt) that contain traces of CallManager activities. These traces provide information about the CallManager initialization process, registration process, call flow, digit analysis, and related devices, such as Cisco IP Phones, gateways, gatekeepers, and more. This information can help you isolate problems when troubleshooting CallManager.
The trace files are stored in the following default location:
C:Program FilesCiscoTrace
Note
Cisco recommends changing the default file location for trace files to the F: drive on dual-processor CallManager servers equipped with extra hard drives for trace collection.
If a trace is enabled, a new trace file is started each time CallManager restarts or when the designated number of lines or minutes has been reached. Although SDI traces do consume some system resources, leaving SDI traces enabled during routine system operation does not typically impact performance unless the system is heavily loaded.
The Cisco Technical Assistance Center (TAC) and Cisco engineers use SDL traces to diagnose difficult problems. The only time you should change the SDL trace configuration is when directed to do so by TAC. SDL trace logs state transitions only for CallManager and Cisco CTI Manager. Cisco engineers use SDL traces to find the cause of an error. You are not expected to understand the information contained in an SDL trace. However, while working with TAC, you might be asked to change the SDL trace configuration and provide the resulting trace files to TAC.
Trace configuration allows you to specify the criteria for SDI tracing. Click Trace > Configuration to select the server you want to trace, and then select the service on that server, such as Cisco CallManager, the Database Layer, CTI Manager, and so on. Figure 6-7 shows the Trace Configuration screen with the default values selected. You can custom configure the trace by selecting the level of trace, the trace fields, and device names (if applicable). Table 6-2 provides a list of the debug trace level settings.
Table 6-2. Debug Trace Levels
Description | |
|---|---|
Error | Use this level for all traces generated in abnormal paths, such as coding errors or other errors that normally should not occur. |
Special | Use this level for all informational, nonrepetitive messages, such as process startup messages, registration messages, and so on. All system and device initialization traces are at this level. |
State Transition | Use this level to trace call processing events or normal events traced for the subsystem (traces for signaling layers). |
Significant | Use this level to trace media layer events. |
Arbitrary | Use this level to generate low-level debug traces. This level is best suited for a testing setup or for debugging difficult problems in a subsystem. |
Detailed | This level is used for detailed debug information or highly repetitive messages that are primarily used for debugging, including KeepAlives and responses. |
CallManager Serviceability provides the option of device-based filtering, which proves useful when a certain problem is known to be occurring on a specific device and you want to only view trace information related to that device. You select device name-based tracing in the Trace Configuration screen. Selecting the devices and then running the trace returns results for any events involving the selected devices.
By default, trace results are saved in a file. You can choose to save traces in TXT format, where up to 10,000 lines can be saved and the file can be viewed in any text editor, or you can choose to save the results in XML format; however, XML-formatted trace files are limited to fewer than 2000 lines.
After a trace starts, it continues until you turn it off. When tracing reaches its limit in the trace files, it begins to overwrite trace data, starting with the earliest trace files/lines. To view an XML trace, you can use Trace Analysis (see the following section, “Trace Analysis”). To view a text-based trace, open the trace in a text editor. Figure 6-7 shows the SDI trace configuration page in CallManager Serviceability.
The Troubleshooting Trace Settings page enables you to easily configure trace settings for troubleshooting most services without having to configure each service one at a time. The Troubleshooting Trace Settings page uses trace levels based on Cisco TAC recommendations. Figure 6-8 shows the Troubleshooting Trace Settings page in CallManager Serviceability.
To apply the TAC-recommended trace settings for a service, select the check box corresponding to the service for which you want to enable traces. You can choose Select all Nodes for a Service to enable traces for a particular service on all nodes in the cluster or use Check all Services for a Node to select all the services on a particular node. When Troubleshooting Trace Settings are enabled, you cannot manually change trace settings until you disable Troubleshooting Trace Settings.
Trace Analysis allows you to perform post-filtering on an XML trace file so that only the relevant information displays. Because trace files can be large and encompass so much information, the ability to reduce and sort that information can be useful. You must know which trace file you want to analyze to use Trace Analysis. Click Trace > Analysis to choose a trace file and specify the selection criteria and the fields you want displayed in the Trace Analysis results.
With Trace Analysis, you can specify collection criteria such as the following:
Cisco CallManager host
Device name
IP address
Trace type
MGCP endpoint
You can filter the trace so that only the pertinent information displays. The following information can be displayed or filtered out of the trace:
Cluster
Date and time
CallManager node
Trace type
IP address
Correlation tag
Application name
Information
Device name
In practice, administrators who are proficient in reading CallManager trace files find that reading and filtering through a raw text-based trace file is much quicker and easier than using Trace Analysis. Also, Trace Analysis is very slow for large amounts of trace data. Future CallManager releases might see XML tracing and the Trace Analysis features removed.
Trace Analysis runs as a low-priority task so that it does not disrupt higher-priority CallManager functions. However, you should us it judiciously because it can be resource-intensive. Memory impact on the CallManager system is minimal, as long as only a few concurrent users run analysis at any given time. If possible, run Trace Analysis only when the CallManager system is not busy. Because traces are being collected and merged into an output file, the tool continuously accesses the disk. Also, be certain the CallManager system has enough disk space for the temporary output files. Currently, no user interface is available to clean up these temporary files, but the tool does automatically recycle them.
Before a service can be used on a CallManager node in a cluster, it must be activated, which you can do on the Service Activation page in CallManager Serviceability (Tools > Service Activation). Activating or deactivating a service notifies members of a cluster that the service state has changed by adding a record into the database, and also modifies the startup type of the Windows service from Disabled to Automatic or vice versa.
CallManager Serviceability enables you to start and stop services by clicking Tools > Control Center. The Control Center also indicates whether the services on the CallManager nodes are currently running or stopped.
The Quality Reporting Tool (QRT) service enables you to add a softkey that allows end users to report problems with their IP phone. To view reports submitted by QRT, you must use the QRT Viewer in CallManager Serviceability (Tools > QRT Viewer).
QRT Viewer enables you to select a server in the cluster and a date/time range. After clicking Get Logs, a page showing all QRT reports submitted during that timeframe displays, including details such as the date and time of the report, the problem being reported, the device from which the report was submitted, and voice quality statistics (packet loss, jitter, and so on) if the problem was reported during a call.
Figure 6-9 shows a sample QRT report indicating that a user was unable to place a call and received a fast busy (reorder) tone. The report shows the date and time the problem occurred, device name, phone number, and IP address.
CallManager Serviceability monitors various aspects of a CallManager cluster and generates nightly reports in PDF format. You can access these reports from the Serviceability Reports Archive in CallManager Serviceability (Tools > Serviceability Reports Archive).
Each day, five reports are generated:
Alert report—. Shows the number of alerts by severity, number of alerts per server in the cluster, and the top 10 alerts.
Call activities report—. Shows an hour-by-hour graph of calls attempted and calls completed for the day and a breakdown of calls per gateway type. This information is helpful for determining your peak traffic periods.
Device statistics report—. Provides hourly graphs showing the number of registered phones, gateways, and trunks.
Server statistics report—. Shows CPU, memory, and hard disk usage statistics in an hourly graph.
Service statistics report—. Shows the number of CTI devices registered and the number of TFTP requests and errors in an hourly graph.
The Serviceability Reports Archive data is presented in a graphical format that enables you to view trends or any sudden events that might have occurred on a given day. For example, you might look at a report and see that CPU utilization is always high between 3 and 4 a.m., but there is no significant call activity during this time. You can then use this information to investigate what might be causing the CPU to be utilized during that time period.
By default, the Serviceability Reports Archive stores the last seven days of data. You can increase this in CallManager Administration (Service > Service Parameters > select a server > Cisco Serviceability Reporter > RTMT Report Deletion Age). You can also configure the time that the reports are generated. By default they are generated at 12:30 a.m.; you can modify the time in CallManager Administration (Service > Service Parameters > select a server > Cisco Serviceability Reporter > RTMT Report Generation Time).
In addition to these PDF reports, the raw performance data used to generate the reports is available in C:Program FilesCommon FilesCiscoLongsRTMTLogger on the Publisher. You can open these files in Microsoft Performance or any other software that supports the CSV file format (for example, Microsoft Excel).
You can check the versions of installed components by clicking Help > Component Versions. Version information proves useful when you are trying to verify whether you have the latest version of an installed component.
Detailed information about CallManager Serviceability is available at the following location:
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/4_1/service/serv412/index.htm
The Real-Time Monitoring Tool (RTMT) is a client plug in that enables you to view real-time serviceability information for a CallManager cluster from a remote PC. RTMT provides real-time clusterwide system monitoring, performance monitoring, and device monitoring. You can monitor CallManager and Windows 2000 performance objects and counters in chart view or table view using the Performance tab in RTMT. You can also set thresholds and generate alerts that can be sent to you as e-mail or pop-up messages. Alerts can be configured for CallManager on a per-node basis, or on phones, gateways, ports, Cisco TFTP, and much more.
RTMT includes several charts and tables that provide an overview of the CallManager cluster. Figure 6-10 shows the summary page in RTMT, which displays the memory utilization, CPU utilization, number of registered IP phones, calls in progress, and active gateway ports on a single screen. Note that each server in the cluster is represented by its own line on the chart.
RTMT has two main tabs on the bottom left:
View—. Enables you to view real-time data
Alert—. Lists all configured alerts and their status
The View tab has seven categories:
Summary
Server
CallProcess
Service
Device
CTI
Performance (called “PerfMon” in some versions)
Each category includes one or more subcategories, each of which has graphs and tables similar to the Summary category shown in Figure 6-10. The sections that follow provide additional information about the seven View tab categories.
The Summary category only has one subcategory, also named Summary, which shows memory utilization, CPU utilization, number of registered IP phones, calls in progress, and active gateway ports and channels.
The Server category has three subcategories that provide information about server resources such as CPU and memory. The three subcategories are as follows:
CPU&Memory—. Shows memory and CPU utilization graphs similar to the Summary category. Also lists all processes on each node of the CallManager cluster and process-specific information such as CPU utilization, private memory bytes in use, and virtual memory bytes in use.
Disk Usage—. Shows total and available hard disk space on each drive of each server in the cluster.
Critical Services—. Lists all services critical to the operation of CallManager for each node in the cluster along with a status of up (working), down (not working), or not activated. This table also shows how long each of the services has been up, enabling you to determine whether a service has restarted unexpectedly.
The CallProcess category has four subcategories that provide call processing statistics such as number of calls active on the system. The four subcategories are as follows:
Call Activity—. Shows the number of calls completed and calls attempted in the last sampling interval. Also shows the number of calls in progress.
Gateway Activity—. Allows you to view statistics for various gateway types: MGCP PRI, MGCP FXS, MGCP FXO, MGCP T1, and H.323. This page provides statistics such as Channels Active and Calls Completed for each gateway type.
Trunk Activity—. Shows the number of calls in progress and calls completed for H.323 and SIP trunks.
SDL Queue—. Displays the number of signals in each of the four SDL queues: High, Normal, Low, and Lowest. Also displays the number of signals processed during the last sampling interval. If signals are being processed at a rate slower than they are put into the queue, you will see the queue size increase. This can cause performance degradation on the CallManager system and is usually due to some kind of system resource limitation such as CPU, memory, or disk I/O.
The Service category has three subcategories that provide information about TFTP and directory services as well as heartbeat information for various services. The three subcategories are as follows:
Cisco TFTP—. Monitors the operation of all TFTP services in the cluster by displaying the total number of TFTP requests, the number of requests where the file was not found, and the number of TFTP requests that were aborted.
Directory Server—. Shows the operational and replication status of the directory services on all nodes in the CallManager cluster.
Heartbeat—. The CallManager, TFTP, and Telephony Call Dispatcher (TCD) services all have a heartbeat counter that increments periodically to indicate the service is still processing calls. This page shows the heartbeat status for CallManager, TFTP, and TCD. If the heartbeat of a service stops, this category indicates that the service is in a state where it is unable to process calls.
The Device category has two subcategories that provide information on registered devices and enable you to search for specific devices on the cluster. The two subcategories are as follows:
Device Summary—. Shows a graph of the number of registered phones, gateways, and media resources and provides a breakdown by gateway type and media resource type in table format.
Device Search—. Provides monitoring information about devices in the CallManager cluster, their IP addresses, their real-time status (such as whether they are successfully registered to CallManager and to which CallManager node), and other useful device information. For devices that support an HTTP server, such as phones and some gateways, you can right-click the specific device information to open an HTTP connection to the selected device. You can then browse information stored locally on the device, such as network configuration and statistics information. You can also launch related screens in CallManager Administration by right-clicking a device type (phone, gateway, voice mail, and so on).
The CTI category has two subcategories that provide information on CTI devices such as number of open devices and a CTI search similar to the device search on the Device category. The two subcategories are as follows:
CtiManager—. Shows the number open CTI devices, the number of open CTI lines, and the number of CTI connections to each node in the CallManager cluster.
CTI Search—. Enables you to search for CTI devices and applications registered to the CallManager cluster. This can prove useful for monitoring whether a CTI application is registered properly with CallManager. The CTI Search subcategory also tells you what user ID an application is using to authenticate with CallManager. If the login is failing, the CTI Search subcategory shows the reason for the failure.
The Performance category has only a single subcategory, called Performance. This category provides access to the same counters available in Microsoft Performance and enables you to configure alerts based on those counters.
Figure 6-11 shows various counters in table view in the RTMT window. RTMT provides the current, minimum, maximum, and average value for each counter. You can see descriptions of every counter by right-clicking the counter and selecting Counter Description.
With RTMT, you can select and monitor real-time performance counters that you specify in your CallManager cluster. You can configure multiple tabs under the Performance category. You can configure the view for each tab to be either graph or table format. The graph format provides 6 panes per category, accommodating up to 18 counters. You can add counters to the panes by double-clicking them from the list of counters. You can also drag and drop counters to layer up to three counters in a single pane. You can add new categories to monitor additional counters. For example, you might have eight T1 lines and are capacity-planning to determine whether to acquire additional T1 lines. You can monitor the counters associated with T1 gateways to determine when usage is approaching the limit. Another example is for load balancing. You can monitor the same counters from two different CallManager nodes in the cluster and put them on the same chart to see whether one CallManager is more heavily loaded than the other. You might decide to change the configuration to balance the load better.
Note that CallManager-related statistics must be enabled for RTMT to collect data. These statistics are enabled by default in CallManager and can be turned on or off in the Service Parameters Configuration screen in CallManager Administration (System > Service Parameters > select a server > Cisco CallManager > StatisticsEnable parameter set to True to enable or False to disable statistics).
The Alert tab in RTMT has only one category (Alert) with one subcategory (Alert Central). Alert Central provides two tables:
A list showing the status of all configured alerts
An alert history showing any alerts that have been previously triggered
Figure 6-12 shows the Alert Central window in RTMT.
The alert status table shows the name of each alert, whether it is enabled, whether the alert is currently active, the action to take if the alert is triggered, and the date and time the alert was last triggered.
If the current value for the monitored counter is outside the configured threshold, the InSafeRange column will show No. Being outside the safe range might trigger an alert if the counter remains outside the configured threshold for a specified amount of time (the duration is user-configurable). You can get additional details about the alert by selecting Alert Event Details from the Alert menu. Figure 6-12 shows the alert details for the MgcpDChannelOutOfService alert. The alert details shows the list of gateways that have the D-channel out of service.
You can configure various alert actions in RTMT, which will send an e-mail of the alert to one or more configured e-mail addresses. You can set different actions for each alert if you want a different set of people to be notified based on the type of alert.
In addition to the preconfigured alerts, you can create an alert based on any performance counter. For example, if you want to be notified any time a call is rejected because a location has no more available bandwidth to allow that call, you can set an alert for the LocationOutOfResources counter in the Cisco CallManager performance object. To do this, add the counter to either a graph or table view in the Performance category of RTMT, and then right-click the counter and choose Alert/Threshold.... You can then configure the level of the alert, a note that you want added to the e-mail any time the alert is sent, and the criteria under which the alert should trigger. The trigger criteria can be any of the following:
The value is over or under a specified limit
The value has changed by a certain amount or percentage since the last sampling interval
You can also choose to trigger the alert immediately or only when the threshold is exceeded for a certain number of seconds. You can also define during what times of the day the alert can be triggered.
Detailed information about Real-Time Monitoring Tool is available at the following location or search Cisco.com for “Real-Time Monitoring Tool”:
Microsoft Performance is a Windows 2000 administrative tool that monitors and logs resource counters from CallManager nodes in the network. Performance shows CallManager-specific status information and Windows 2000 system information in real time. The CallManager system gathers statistical information about the Cisco IP Telephony deployment and feeds it into Performance by way of objects and counters. For example, you can monitor the number of calls in progress at any time, or the number of calls currently passing through a specific Cisco gateway. Data for non-CallManager-specific objects and counters are gathered by the respective services or the operating system itself. For example, the World Wide Web Publishing service feeds information to Performance about the various web pages served on a CallManager server.
Performance can collect data from multiple CallManager servers at once and compile it into a single log file. You can view the logged information in Performance and then export it into tab-separated value (TSV) or CSV file format that can you can view with most spreadsheet applications. Performance enables you to view statistical data in graphical, histogram, and report form. You can access Performance by clicking Start > Programs > Administrative Tools > Performance.
Like the Real-Time Monitoring Tool, you can use Performance to monitor various real-time conditions in a CallManager system. For example, you can discover the number of calls in progress on a particular CallManager node at any time, or the number of calls currently being attempted in a CallManager cluster. This information is useful for capacity planning, network planning and design, load balancing, and troubleshooting, among other uses.
Each object includes counters that keep track of statistics such as the number of registered MGCP gateways or the number of registered hardware phones. These counters define current conditions within groups of related information. Each group of related information is called an object; each object contains one or more counters, such as Cisco CallManager or Cisco Software Conference Bridge, and each of these objects can have more than one instance of the object. Objects and counters are automatically added when CallManager or the related component (such as Conference Bridge) is installed. Using objects and counters, you can retrieve detailed, relevant, and timely system information. Performance can also be customized to track Cisco applications such as the IP IVR or Windows 2000 system objects and counters. This additional information can be useful to correlate system events with CallManager events. For example, if you notice a surge in CPU utilization, you might also notice that there is a surge in call volume which would explain the higher-than-expected CPU utilization.
Just as with RTMT, ensure that statistics are enabled in CallManager Administration for Performance to collect data. Statistics are enabled by default. You can stop CallManager from sending data to Performance and the Real-Time Monitoring Tool by setting the Statistics Enabled parameter to False in the Service Parameters Configuration page in CallManager Administration (System > Service Parameters > select a server > Cisco CallManager).
The Trace Collection Tool (TCT) is a client plug-in (Application > Install Plugins) that you can install on a Windows PC to facilitate collecting trace data from a CallManager cluster. Before you can use TCT you must enable traces as described in the section “Cisco CallManager Serviceability.”
You can collect traces for one or more services and one or more nodes of the cluster.
TCT collects traces for the following services:
Cisco CallManager
Cisco CDR Insert
Cisco Certificate Authority Proxy Function
Cisco CTI Manager
Cisco CTL Provider
Cisco Database Layer Monitor
Cisco Extended Functions
Cisco Extension Mobility
Cisco IP Manager Assistant
Cisco IP Voice Media Streaming App
Cisco Messaging Interface
Cisco MOH Audio Translator
Cisco RIS Data Collector
Cisco Telephony Call Dispatcher
Cisco TFTP
Cisco WebDialer
TCT also enables you to collect traces for the following CallManager applications:
BAT
CAR
Cisco Serviceability Reporter
Cisco Tomcat
Installation Log Files
Multilevel Administration (MLA)
Quality Reporting Tool (QRT)
Tool for Auto-Registered Phone Support
Finally, TCT enables you to collect the following system traces:
Event Viewer logs (application, security, and system)
Dr. Watson logs (crash dumps created by the Windows operating system)
Internet Information Server (IIS) logs
Microsoft SQL Server 2000 logs
Directory logs
System performance logs
ProgLogs (includes system information from CallManager startup)
Figure 6-13 shows the trace selection screen. TCT enables you to collect all traces for the services selected or only those that fall within a specific timeframe. TCT can also optionally compress the collected files into a Zip archive to save hard drive space on the PC where you are collecting the traces.
Time-based collection can prove useful for diagnosing a problem where you know approximately the time of the problem. For example, if a user complains that a call was cut off, you can collect traces for the time period in which the dropped call occurred (say, yesterday morning between 7:45 a.m. and 8:15 a.m.).
Detailed information about the Trace Collection Tool is available at the following location or search Cisco.com for “Trace Collection Tool”:
Microsoft Event Viewer helps identify problems at the system level. For example, a group of users cannot make calls in a system with one gateway. You can use Event Viewer to look for events about the gateway (such as registration or unregistration events) to pinpoint the problem. Event Viewer starts automatically when Windows 2000 is started and records events in three kinds of logs:
Application log—. Contains events logged by applications or programs, such as CallManager.
System log—. Contains events logged by the Windows 2000 system components, such as the failure of a system component.
Security log—. Contains records of security events such as login failures.
Event Viewer displays the following types of events:
Error—. A significant problem, such as loss of data or loss of functionality. For example, if a problem occurs with a device that is registered with CallManager, an error event provides device information and error details to help you isolate the problem.
Warning—. An event that is not necessarily significant but might indicate a possible future problem. For example, warning events can include CallManager services that have stopped or started.
Information—. An event that describes system information, such as host name or IP address and the version of the database layer in use.
By default, CallManager alarms are sent to the Event Viewer at the Error level. You can change the alarm event level in the Alarm Configuration screen in CallManager Serviceability (Alarm > Configuration > select a server > select a service). Figure 6-14 shows the details of an information message that indicates that the gateway has registered with CallManager.
You can access Event Viewer by clicking Start > Programs > Administrative Tools > Event Viewer.
Terminal Services Client, part of Windows 2000 Server, enables you to export a remote desktop to your PC. It is particularly useful because it allows you access to the remote PC as though you were local to the machine. You must know an administrator-level password on the remote server to access it via Terminal Services. After you have accessed a server using the Terminal Services client or Microsoft Remote Desktop client, you can perform tasks such as system administration, viewing log files or event logs, and so on.
Installing the Terminal Services client is possible using a variety of methods. Cisco recommends using the latest Microsoft Remote Desktop Connection (RDC) client available from Microsoft.com.
Windows XP comes bundled with the RDC client and is available from the Start menu (Programs > Accessories > Communications). If you are not running Windows XP, you can obtain the RDC client from Microsoft.com for Windows 2000 or Mac OS X.
If you would like to install the older Terminal Services client instead of the RDC client or do not have Internet access, you must first create floppy disks containing the application. From the CallManager server, click Start > Programs > Administrative Tools > Terminal Services Client Creator. Use the Terminal Services Client Creator to create floppy disks; then install Terminal Services on any PC using the floppy disks. When installed on the PC, you can use Terminal Services by clicking Start > Programs > Terminal Services Client and designating the IP address or host name of the remote server you want to access.
Virtual Network Computing (VNC) Viewer is a remote display system that enables you to view a remote desktop environment, similar to Terminal Services. VNC allows you to use one computer to drive actions on a target computer, but differs from Terminal Services because with VNC, any actions performed by you that occur on the target computer can be seen equally by the local user. Unlike Terminal Services, you can use VNC to install, upgrade, or apply patches to CallManager.
You can access the VNC application and documentation files on the operating system (OS) version 2000.2.2 and later installation disc or download. If you’re running an older version of the OS, run the OS upgrade for version 2000.2.2 or later to gain access to the VNC files. OS upgrades are available at the following link (requires Cisco.com login):
Caution
Using VNC can expose you to a security risk. Review the “Security Best Practices” section in the Cisco-produced document for installing VNC, which is available on the OS 2000 version 2.2 and later installation disc or at the download link previously shown. Use a complex alphanumeric password for VNC. VNC does not have a username/password structure; it uses only a single password, and VNC limits the password to eight characters, so make sure the password you choose is difficult to crack. A good password includes numbers, upper- and lowercase letters, and special characters, and does not use any known word. For example: 123eye67 is not as good a password choice as 4hW9Lv#g.
CiscoWorks is the network management system (NMS) of choice for all Cisco devices, including the CallManager system. CiscoWorks IP Telephony Environment Monitor (ITEM) adds additional functionality to the base CiscoWorks package, which provides the capability to continuously evaluate and report the operational health of your Cisco IP Telephony implementation and provides some diagnostic tools to help troubleshoot problems that occur in the IP Telephony network. ITEM provides specialized operations and security tools beneficial to large and small IP telephony implementations. ITEM is not bundled with CallManager and must be purchased separately.
Using CiscoWorks, you can configure and produce reports on log messages collected from CallManager nodes and other IP telephony devices. CiscoWorks provides a common system log for applications in the multiple-host and multiple-platform Cisco IP Communications environment. ITEM uses SNMP and CallManager’s AXL SOAP interface to provide additional information on each device from which the log messages originate.
Each time a device is added to the ITEM device inventory, a new database entry is created. After the device is added to the list, ITEM gathers some device information over SNMP. You can read and use this information for system maintenance and problem solving.
ITEM includes five components.
CiscoWorks IP Telephony Monitor (ITM)—. Monitors Cisco voice elements in the network to alert operations personnel to potential problems and to help minimize downtime. This component also includes CiscoWorks Common Services, a common foundation for data storage, login, access privileges, and navigation and launch management for all CiscoWorks applications.
CiscoWorks IP Phone Information Utility (IPIU)—. Provides operational status and implementation details about an individual IP phone. This component also provides security reports that document IP phone “moves, adds, and changes” as well as information about the physical and logical connections of every Cisco IP Phone installed in a given network.
CiscoWorks IP Phone Help Desk Utility (IPHDU)—. Reports operational status and implementation details about individual IP phones. This component works in conjunction with the IPIU to make read-only access to Cisco IP Phone installation details available to help desk personnel.
CiscoWorks ITEM Gateway Statistics Utility (GSU)—. Collects performance and behavior statistics about CallManager-controlled and Cisco IOS Software-based IP telephony gateways, which can be processed by third-party software to produce utilization and capacity management reports.
CiscoWorks WAN Performance Utility (WPU)—. Measures the performance, latency, and availability of multiprotocol IP networks on an end-to-end and hop-by-hop (router-to-router) basis.
In addition to ITEM, the CiscoWorks family of web-based products supports maintenance of Cisco enterprise networks and devices. The products include Resource Management Essentials and Campus Manager, which provide syslog analysis, topology services, path analysis, user tracking, fault management, and other network management services.
The syslog analysis tools are Syslog Collector and Syslog Analyzer. They are offered with CiscoWorks as part of the Resource Management Essentials package. Syslog output from CallManager can alternatively be adapted for use with other NMSs that support the standard syslog format:
The Syslog Collector keeps common system logs that record messages reported to the CallManager system.
The Syslog Analyzer controls and displays all events so they can easily be read, interpreted, and used for system maintenance and problem solving.
Using the reporting and managing capabilities of these tools, you can monitor and manage a wide range of events and error messages concurrently on each CallManager node and other Cisco devices.
Syslog Collector gathers log messages from a CallManager cluster or node at any network installation. The service collects a wide range of significant event messages that reflect system status. After validating the events or error messages collected, Syslog Collector passes them to the Syslog Analyzer. When this process is complete, you can use Syslog Analyzer to analyze the log messages.
Syslog Analyzer, which resides on a CiscoWorks server, receives the messages collected from multiple applications by the Syslog Collector. When a collection of data is received, the Syslog Analyzer parses and stores the results in the CiscoWorks database. This interface enables you to access and manage whatever data is collected from the system’s managed devices.
With Syslog Analyzer, you can examine the event log reports from each Cisco CallManager system, including the description and recommended actions for each log message. In addition to a cluster of Cisco CallManagers, a network installation can also have some voice equipment, routers, gateways, and other devices generating log messages. After you have set up your system, you can access all of this information through one server.
You can find detailed information about CiscoWorks at the following locations:
A Management Information Base (MIB) is a structured set of data variables, called objects, in which each variable represents some resource to be managed. Simple Network Management Protocol (SNMP) MIB conceptual tables organize and distribute the information gathered from your IP telephony system.
SNMP allows CallManager to be managed with standard network management applications, such as CiscoWorks or HP OpenView. Windows 2000 Server provides an extensible SNMP agent that can be installed and run as a service. Cisco extension agents (dynamic link library, or DLL) support Cisco MIBs, which provides support for CallManager-specific data.
For CallManager nodes that run on HP servers, the network management application can get system information from the HP Insight Agent MIB. No trap is provided in this MIB, so the management application needs to poll the information that it is interested in periodically and generate its own trap when it reaches a certain threshold value. You can obtain platform-specific information such as hard disk array and power-supply status from the HP Insight Agent. ITEM uses the HP Insight Agent to provide platform-specific alerts for your IP telephony servers.
For CallManager nodes that run on IBM servers, the network management application can get system information from the IBM Director Agents. The IBM Director Agents are similar to the HP Insight Agent in that they provide platform-specific information. The IBM Director Agent can be used by ITEM to monitor the IP telephony server platform or can be used with IBM Director Server and Console to monitor the platform.
SNMP objects and traps are defined in CISCO-CCM-MIB. The CCM MIB extension agent implements the Cisco CallManager MIB. This MIB exports the data in the CallManager database and in other data sources. A trap is an unsolicited message sent by an agent to a management station in an asynchronous manner. The purpose is to notify the management station of some unusual event. The traps are sent to trap-receiving hosts configured in the Windows 2000 SNMP service. Network management applications such as ITEM can gather data that can be used for fault management and analysis purposes. Although the design of the MIB and trap is tied to CiscoWorks applications, it does not limit you from developing network management applications by using third-party software. The SNMP extension agent for the CISCO-CCM-MIB is packaged as a DLL file and bundled in the CallManager installation.
The CDP MIB extension agent implements all of the variables related to the tell side of Cisco Discovery Protocol (CDP). The MIB variables implemented are cdpInterfaceTable, cdpGlobalRun, cdpGlobalMessageInterval, and cdpGlobalDeviceId. This is the minimum CDP SNMP support that network management applications such as CiscoWorks need to discover the CallManager server. The variable cdpGlobalDeviceId is of type DisplayString (meaning an alphanumeric string) and will return the same value that CDP reports in its CDP advertisement messages.
The Cisco RIS Data Collector is primarily responsible for updating the information used by the SNMP agents, and it buffers the CISCO-CCM-MIB information that the agents process. At startup, the Cisco RIS Data Collector updates all of the relevant information by periodically fetching data from CallManager or the CallManager database. This updated information is based on interaction with CallManager and other CallManager-associated services.
At startup, the CDP SNMP extension agent interacts with the CDP driver, fetching and buffering CDP-related information.
New features and bug fixes cause the CISCO-CCM-MIB to be routinely updated. You can download the latest MIB from the following location:
This site provides detailed information about hundreds of MIBs, including CISCO-CDP-MIB and other MIBs that may be of interest to you.
CDP allows CallManager to advertise itself to other Cisco devices on the network by sending periodic messages to a well-known Multicast address monitored by other neighbor devices. Network operators and analysts use this information for configuration monitoring, topology discovery, and fault diagnosis purposes.
With CDP support, CallManager periodically sends out CDP messages or protocol data units (PDU) on the active physical interfaces. These messages contain CallManager information such as the device ID, interface name, system capabilities, and so on. Any Cisco devices with CDP support can discover CallManager by listening to these periodic messages.
CallManager provides a large amount of trace data to help you diagnose problems; however, analyzing trace logs can prove difficult for those unfamiliar with the formatting of the messages. In addition, much of the data in the traces appears in a numeric or hexadecimal format that you must decode to understand what is in the trace. For example, if a call going through a PRI gateway is disconnected, a cause code is sent indicating the reason for the disconnect, but the value in the CCM trace appears in hex, such as 0x8090. You need a way to convert this value into human-readable format—in this case “Normal call clearing.” To facilitate this type of trace decoding, use the Voice Log Translator (VLT) tool.
VLT enables you to open a CCM (SDI) trace file and decode its contents. Figure 6-15 shows the output of the VLT tool. You can see that it is decoding the Q.931 Disconnect cause code of 0x8090 to “Normal call clearing.”
VLT supports the decoding of the following protocols/interfaces:
SCCP
Q.931
H.225
H.245
MGCP
JTAPI
In addition to decoding these protocols/interfaces, VLT also enables you to perform advanced searching and filtering of trace data to help you find the root cause of a problem. For example, if you know the called party number for a call that failed, you can do a search for that phone number and VLT will display only those messages that contain that phone number. You can then tell VLT to filter by the call reference for that call to see all the messages related to the call.
VLT will work only with text-based CCM (SDI) trace files and will not work with XML traces, which is another reason why Cisco recommends using the default text-based traces. VLT is available at the following URL or search Cisco.com for “Cisco Voice Tool”:
This chapter provided information about two applications to assist with managing your network, BAT and CAR. The important thing to consider is whether the tools described can help in your daily operations. In most cases, either BAT or CAR (or both tools) can save you considerable time (and, therefore, money) in configuring, managing, and diagnosing your CallManager system.
This chapter also covered several monitoring applications, including CallManager Serviceability and the Real-Time Monitoring Tool, which can assist in monitoring your system. You should understand how to use the features in CallManager Serviceability because these will help you troubleshoot the system.