Each non-functional requirement has a set of sub-qualities and metrics that are associated with each sub-quality. How does the service in question perform against the service quality targets set for the service? This question can be answered with the Enterprise Monitoring and Management (EMM) architecture and capabilities. EMM tools can help with business, service, and resource monitoring and reporting. Measurable and reportable metrics that map to the non-functional requirements are key for non-functional requirements monitoring. SMART metrics are Specific, Measurable, Achievable, Relevant, and Time-bound.
Non-functional requirements themselves can be performance targets for certain key metrics. For example, Mean Time to Restore Service (MTTRS) is a key metric for availability and recoverability (sub-quality), and number and percent of time the 30 minutes MTTRS is met or exceeded for email as a service becomes a measurable and reportable non-functional requirement metric for e-mail.
Metrics models such as the Distributed Management Task Force (DMTF) CIM metrics model with its Unit of Work (UoW) definition, base metric definition, and base metric value provide a standards-based model for defining and managing non-functional requirements-related metrics.
As per the CIM metrics model, there can be several units of work such as batch jobs, user-initiated interactive operations, completed and committed transactions, and so on associated with a service. There are also several metrics associated with each UoW.
UoW metrics and measurements are at a more granular level than service metrics. A UoW can have several associated non-functional requirements metrics, even though the CIM metrics model-related UoW metrics are primarily time taken to complete the UoW (performance dimension) and status of the UoW (availability dimension).
In ITIL parlance, Continuous Service Improvement (CSI) and Service Improvement Plans (SIP) involve measures to improve service qualities and service capability to meet or exceed non-functional requirements.
For example, service outage analysis and the application availability patterns result in availability improvement plans, particularly when IT organizations face Service Level Agreement (SLA) breaches due to unplanned outages.
Similarly, IT organizations can have improvement plans for each service quality or sub-quality discussed in this White Paper.
New and emerging technologies, processes, and organizational capabilities can directly improve certain service qualities.
Examples include replication technologies and the exponential decline in the cost of storage space (disk space measured in gigabytes), which has allowed for significant improvements in service continuity capabilities.
Another example is grid storage that offers improved storage performance and resilience capabilities over older storage methods. The NFR life cycle results in the realization and improvement of a service performance when it comes to non-functional requirements. The following table describes NFR framework:
|
NFR
|
Attributes
|
|
Performance
|
- Throughput: The ability of the system to execute a given number of transactions within a given unit of time
- Response times: The allowable distribution of time which the system takes to respond to request
|
|
Scalability
|
- Throughput: Number of maximum transactions your system needs to handle, for example, thousand a day or A million
- Storage: Amount of data you going to need to store
- Growth requirements: Data growth in the next 3-5 years
|
|
Availability
(uptime)
|
- Availability: Application availability considering the weekends, holidays and maintenance times and failures
- Locations of operation: Geographic location, Connection requirements and the restrictions of a local network prevail
- Offline requirement: Time available for offline operations including batch processing and system maintenance
- Length of time between failures
- Recoverability: Time required by the system is able to resume operation in the event of failure
- Resilience: The reliability characteristics of the system and sub-components
- MTBF : Length of time between failures
- MTTR : Length of time needed to resume operation after a failure
- Availability = MTBF/(MTBF+MTTR)
|
|
Capacity
(provisioning for growth)
|
- Throughput: Number of peak transactions the system needs to handle
- Storage: Volume of data the system will page/persist at runtime to disk. This relates to the memory/disk
- Year-on-year growth requirements (users, processing and storage)
- e-channel growth projections
- Activities or transactions supported for each type of transaction, volumes on an hourly, daily, weekly, monthly and so on
- Volumes are significantly higher during specific parts of the day (for example, at lunch), week, month or year
- Transaction volume growth expected and additional volumes you will be able to handle
|
|
Security
(define key security requirements)
|
- Authentication: Correct identification of parties attempting to access systems and protection of systems from unauthorized parties
- Authorization: Mechanism required to authorize users to perform different functions within the systems
- Encryption (data in flight and at rest): All external communications between the system's data server and clients must be encrypted
- Data confidentiality: All data must be protectively marked, stored and protected
- Compliance: The process to confirm systems compliance with the organizations security standards and policies:
-
- Resistance to known attacks (to be enumerated)
- Time/efforts/resources needed to find a key (probability of finding the key)
- Probability/time/resources to detect an attack
- Percentage of useful services still available during an attack
- Percentage of successful attacks
- Lifespan of a password, of a session
|
|
Maintainability
(the ease with which the system can be maintained)
|
- Conformance to design standards, coding standards, best practices, reference architectures and framework
- Flexibility: The degree to which the system is intended to support change
- Release Support: The way in which the system will support the introduction of initial release, phased rollouts and future releases:
-
- Coupling/cohesion metrics
- Number of anti-patterns
- Cyclomatic complexity
- Mean time to fix a defect
- Mean time to add new functionality
- Quality and quantity of documentation
|
|
Manageability
|
- System must maintain full traceability of transactions
- Audited objects and audited database fields to be included for auditing
- File characteristics: size before, size after, structure
- User and transactional time stamps, and so on
- Get notices and alerts as thresholds (for example, storage, memory, processor) are approached
- Remotely manage systems and create new virtual instances at the 'click of a button'
- Rich graphical dashboard for all key applications metrics
|
|
Reliability
|
- The ability of a system to perform its required functions under stated conditions for a specific period of time
- Mean Time Between Failures: Acceptable threshold for down-time
- Mean Time To Recovery: Time is available to get the system back up online
- Data integrity: Referential integrity in database tables and interfaces
- Application integrity and information integrity: During transactions
- Fault trapping (I/O): Handling failures and recovery
- Defect rate
- Degree of precision of computation
|
|
Extensibility
|
- Handle new information types
- Manage new or changed business entities
- Consume or provide new feeds
|
|
Recovery
(in the event of failure)
|
- Recovery process: Recovery Point Objectives (RPO) and Recovery Time Objectives (RTO)
- RTO/Restore time: Time required to switch to secondary site when the primary fails
- RPO/Backup time: Time taken to back your data
- Backup frequencies: Frequency of backing-up the transaction data, config data and code
|
|
Interoperability
|
- Compatibility with shared applications: Other systems it need to integrate
- Compatibility with third party applications: Other systems it has to live with amicably
- Compatibility on different operating systems: Different OS compatibility
- Compatibility on different platforms: Hardware platforms it needs to work on
|
|
Usability
|
- Look and feel standards--screen element density, layout and flow, colours, UI metaphors, keyboard shortcuts
- Internationalization / localization requirements-- languages, spellings, keyboards, paper sizes, and so on
- Proportion of functionalities or tasks mastered after a given training time
- Acceptable response time
- Number of tasks performed or problems resolved in a given time
- Number of mouse clicks needed to get to information or functionality
- Number (or ratio) of learned tasks that can still be performed after not using the system for a given time period
- Number of error per time period and user class
- Number of calls to user support
- Mean time to recover from an error and be able to continue the task
- Satisfaction ratio per user class and usage ratio
|
