242 Tivoli Business Systems Manager Version 2.1: End-to-End Business Impact Management
[133] exit drop_received_event
[134] call first_instance(event : _4741 of_class w2k_MemAvailBytes
where [date_reception : equals 0x3de5005f,server_handle : equals
1,event_handle : equals 1])
[135] exit first_instance(event : 0x1491498 of_class
w2k_MemAvailBytes where [date_reception : equals 0x3de5005f,server_handle :
equals 1,event_handle : equals 1])
[136] call set_event_status(0x1491498,CLOSED)
[137] exit set_event_status(0x1491498,CLOSED)
[138] call set_event_administrator(0x1491498,'TBSM:administrator')
[139] exit set_event_administrator(0x1491498,'TBSM:administrator')
[140] call commit_set
[141] exit commit_set
[142] exit reception_action action0
We can see the dropping of the received event [132] and the closure of the event
[136-137]. This event now is cleared from the TEC console and this status also is
reflected on the IBM Tivoli Business Systems Manager console.
7.6 Troubleshooting
If something goes wrong and the object is not shown at the IBM Tivoli Business
Systems Manager console, we have to analyze the source of the failure.
1. First check whether the conditions have been met to cause an alert at the
endpoints where the monitors have been distributed. For DM classic we use
wlseng -l <enpoint_name> and for IBM Tivoli Monitoring or DM advanced
edition, we use wdmlseng -e <endpoint_name> -verbose.
2. If these monitors fire correctly and the network communications within the
Tivoli Management Region are working, the next step for this event is TEC.
3. When TEC receives a recognized event it is parsed by the rules engine. The
rule that will parse and act upon classic DM events and send to IBM Tivoli
Business Systems Manager is ihstdmon.rls.
4. The wtdumprl command is used to display all received events and their
disposition whether they are processed or getting parsing failed status. When
a parsing is failed, it means that either the class is not defined or there is an
unknown slot (field) or an unparsable slot. The result of wtdumprl command is
shown in Example 7-15 on page 243. The event is succesfully received.
Note: While setting up the monitors we also send a message to the Tivoli
notice board to help troubleshoot the classes if they are not defined correctly
within TEC. When everything is working, this can be disabled.
Chapter 7. TEC components integration 243
Example 7-15 Result of wtdumprl
1~1429~65537~1038357634(Nov 26 18:40:34 2002)
### EVENT ###
w2k_MemAvailBytes;
source='SENTRY';
sub_source='ClassicSentryProfile';
severity='FATAL';
origin='9.3.4.58';
sub_origin='ibmtiv8';
hostname='ibmtiv8';
adapter_host='ibmtiv8';
distrib_admin='ibmtiv8';
response_level='normal';
probe='AvailBytes';
probe_arg='';
tmr='1345392748';
dispatcher='2';
prev_value='6.80362e+008';
value='6.79887e+008';
effective_value='6.79887e+008';
collection='w2k_Memory';
info='';
monitor='Available Bytes';
units='(bytes)';
relation='(default result)';
relation_delta=' no change';
msg='Distributed Monitoring ClassicSentryProfile/Available Bytes on host
ibmtiv8 11/26/02 18:39:39 Central Standard Time
Status: >>> normal <<<
Available Bytes () no change (default result)
(Previous: 6.80362e+008 (bytes) Current: 6.79887e+008 Effective: 6.79887e+008)
';
END
### END EVENT ###
PROCESSED
5. The wtdumptr command is used to display the completion of the action
invoked from the TEC rules for an event. For example, the completion code of
invoking the event enablement exits. It can be seen from Example 7-16 that
the rule is sucessfully processed from the wtdumptr command.
Example 7-16 Result of wtdumptr
TASK;
server_handle=1;
244 Tivoli Business Systems Manager Version 2.1: End-to-End Business Impact Management
date_reception=1038357634; # Nov 26 18:40:34 2002
event_handle=1;
task_number=1;
command='../../TDS/EventService/ihstztec ';
start_time=1038357634; # Nov 26 18:40:34 2002
status=SUCCESS;
exit_status=0;
message=[ ];
6. If the received event matches the profile of an event that has been slated to
send to IBM Tivoli Business Systems Manager (in our case it was the
ClassicSentryProfile subsource), a TEC exit ihstztec is invoked. This exit also
is invoked when there is a change of status to this event to ACK or CLOSE.
Again, we confirm this from the rule trace shown in Example 7-17.
Example 7-17 rules.trace output
[1]=> rule set tbsmstatus
[2]=> rule set interapp
[3]=> rule set ihstdmon
[4] -> rule ihstdmon_ClassicSentryProfile
event : 0x14cab38 of_class TEC_Start
[5] call condition
[6] call sub_source : _621
[7] exit sub_source : ''
[8] call sub_source : '' equals ClassicSentryProfile
[9] fail sub_source : '' equals ClassicSentryProfile
[10] fail condition
[11]=> rule set tbsmstatus
[12]=> rule set interapp
[13]=> rule set ihstdmon
[14] -> rule ihstdmon_ClassicSentryProfile
event : 0x183fda8 of_class w2k_MemAvailBytes
[15] call condition
[16] call sub_source : _621
[17] exit sub_source : ClassicSentryProfile
[18] call sub_source : ClassicSentryProfile equals
ClassicSentryProfile
[19] exit sub_source : ClassicSentryProfile equals
ClassicSentryProfile
[20] exit condition
[21] call reception_action action0
[22] call
exec_program(0x183fda8,'../../TDS/EventService/ihstztec','',[],YES)
[23] exit
exec_program(0x183fda8,'../../TDS/EventService/ihstztec','',[],YES)
[24] exit reception_action action0
[25]=> rule set tbsmstatus
[26]=> rule set interapp
Chapter 7. TEC components integration 245
[27]=> rule set ihstdmon
[28]=> rule set tbsmstatus
[29]=> rule set interapp
[30]=> rule set ihstdmon
[31] -> rule ihstdmon_change_ClassicSentryProfile
event : 0x183fda8 of_class w2k_MemAvailBytes
[32] call condition
[33] call sub_source : _631
[34] exit sub_source : ClassicSentryProfile
[35] call sub_source : ClassicSentryProfile equals
ClassicSentryProfile
[36] exit sub_source : ClassicSentryProfile equals
ClassicSentryProfile
[37] call slot status set_to : ACK within [ACK,RESPONSE,CLOSED]
[38] exit slot status set_to : ACK within [ACK,RESPONSE,CLOSED]
[39] exit condition
[40] call action action0
[41] call
exec_program(0x183fda8,'../../TDS/EventService/ihstztec','-n',[],YES)
[42] exit
exec_program(0x183fda8,'../../TDS/EventService/ihstztec','-n',[],YES)
[43] exit action action0
If you examine the trace of the rules you will see from [1] to [10] an event
relating to the starting of the TEC server. This is parsed by the ihstdmon rule
but fails because the condition where the ClassicSentryProfile subsource
fails. The next event we can see parsed, represented by [14] to [24], shows
that we have an event that has met the subsource:ClassicSentryProfile.
7. The ihstztec exit when invoked (as seen in Example 7-17 on page 244 [21] to
[24] ) formats the data and passes it onto the Tivloi BSM Event Enablement
service. This in turn sends this event to the agent listener. We also can see
from lines [30] to [43] that we have sent an ACK (acknowledgement from the
TEC console) and this state change is also sent to IBM Tivoli Business
Systems Manager.
We have seen the usage of wtdumprl, wtdumptr, and the tracing of the rule base
that has been used to show the data flow from within TEC. The most common
problem you will see from the wtdumprl is that of an unprocessed event. If this is
the case check the classes (BAROC files) whose addition may have been
missed. The most useful trace is that of the rule base, as you may have made a
syntax error with the customization of a rule.
The event enablement engine itself also keeps logfiles. Four log files, two for the
task server and two for event enablement, are available in the
$BINDIR/TDS/EventService/log directory. You need to start tracing for event
246 Tivoli Business Systems Manager Version 2.1: End-to-End Business Impact Management
enablement to get meaningful information into the log file. Run the trace using
the tserver ee_utility command. To start tracing, issue the command:
$BINDIR/TDS/EventService/bin/tserver ee_utility -t
To format these log files, we use the command ihszfmt, which is located in
subdirectory $BINDIR/TDS/EventService/bin.
The next part of the data flow we examine is that of the agent listener. Verify that
the agent listener is connected to the event enablement with the command
gemeeconfig as shown in Example 7-4 on page 225.
The next point to check is the agent listener log file. The amount of information
logged in the log file depends on the loglevel, which can be set in the Windows
registry. When the loglevel is set to 0, everything will be traced. The default
loglevel setting is 2. The file is named AL + Timestamp of creation, so the file
AL200211141427.log is the agent listener log that is created on the 14
November 2002, at 14:27. This logfile, shown in Example 7-18, matches the
event that is shown in Example 7-10 on page 233.
Example 7-18 AL200211141427.log
2002/12/03
13:36:50|593|DBG|0|AL|774|B:oleasiruleengineASIRuleEngineObj.cpp|307|GetNext
Event: got message "( assert ( APMGenericEvent 99999 "IBMTIV5"
"ASIEnqueueGemMsg" 50003 "12/03/2002 13:36:50" 1038944210 2000 1004
asimcsAvailability asimpMedium 1 -1 "generictest;1.0" "ibmtiv9" "" "EXCEPTION"
"TMW_Processor" "" "" "CRITICAL" "" "" "High CPU Usage, High CPU Usage User
Priv, and High CPU Usage Process are exceeded on the processor 0." "ibmtiv9" 22
"ibmtiv9" "" "ibmtiv9" "" "" "" "" 1038944076 "1" "1" "1038944076"
"TMW_HighProcesses" "OPEN" "" 0 ) )"
2002/12/03
13:36:50|625|DBG|0|AL|774|B:oleasiruleengineASIRuleEngineObj.cpp|268|GetNext
Event: waiting for message
2002/12/03
13:36:50|625|DBG|0|AL|774|B:oleasiruleengineASIRuleEngineObj.cpp|307|GetNext
Event: got message "(run)"
2002/12/03
13:36:50|671|DBG|0|AL|774|B:asidbincludeASIDBASISQLDBC.ado.cpp|805|adCmdSto
redProc(4) - { ? = call gemsp_processAPMGenericMsg(?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) }(
/*@RETURN_VALUE=<empty>(R)*/@tbsmclass=N'generictest;1.0'/*(I)*/,
@instanceid=N'ibmtiv9'/*(I)*/, @instancename=N''/*(I)*/,
@probetype=N'EXCEPTION'/*(I)*/, @probeid=N'TMW_Processor'/*(I)*/,
@probename=N''/*(I)*/, @probeargs=N''/*(I)*/, @severity=N'CRITICAL'/*(I)*/,
@probevalue=N''/*(I)*/, @probecondition=N''/*(I)*/, @msg=N'High CPU Usage, High
CPU Usage User Priv, and High CPU Usage Process are exceeded on the processor
0.'/*(I)*/, @eehost=N'ibmtiv9'/*(I)*/, @hostos=22/*(I)*/,
@tcphostname=N'ibmtiv9'/*(I)*/, @tmr=N''/*(I)*/,
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset