Compliance to project plans and standards is ensured by process assurance. Process assurance performs inspections of the hardware and its artifacts to make sure it conforms to its drawings and specifications. Artifacts from the life cycle process are audited to make sure that the processes were followed, and for Level A and B design assurance, that the transition criteria specified in the hardware management plans were met.
While process assurance needs to be performed independently from the audited activities, it does not have to be an independent organization. Companies typically use a representative from the quality organization to perform process assurance activities, which satisfies the independence criteria. A small company can use people in varying roles and meet the independence criteria without a separate quality or process assurance organization.
The activities for process assurance can be performed by personnel that have engineering skills, or personnel experienced with quality assurance responsibilities. The audits and reviews do not require technical design skills, though technical competency can enhance the depth and value of process assurance. The process assurance engineer does not need to perform peer reviews of technical documents. The typical role for the process assurance engineer is to ensure that the peer reviews were performed, records of the reviews were retained in the configuration management system, and that all issues raised in the peer review were resolved.
Process assurance activities should be performed in a timely manner and not left until the end of a project. Using process assurance as a rubber stamp to approve data and hardware at the last minute can compromise product quality, compliance with the project certification basis, and the ultimate certification of the product.
While process assurance persons may not always be design engineers, they are nevertheless valuable team members. Early detection of process issues in a compliance project allows for corrective action to be defined, implemented, and measured. Taking care of deviations in a timely manner will also help ensure ultimate approval of the project.
Process assurance activities are intended to be performed by sampling the work product. While 100 percent sampling is not required, a smaller sample size should be earned rather than assumed. What this means in practice is that initial audits of engineering work products should look at work from each person on the project.
Taking samples from the most rigorous and conscientious engineer and assuming or asserting that it is a representative sample is not a thorough approach.
For example, when auditing peer reviews, the audits should take samples from each person performing a review and from each type of peer review. Stellar work on requirements peer reviews does not mean that the peer reviews of testbenches were equally proficient.
When trends, positive or negative, are detected in the samples, the trends should be used to justify decreasing or increasing the sample size. Initial audits should be broad, covering many samples of life cycle data and personnel. Once the organization has demonstrated compliant practices, process assurance can reduce the sampling to a smaller percentage. If problems arise later on in the project, the sample size should increase again to determine how widespread the issue is. Once corrective action is implemented and the quality and proficiency in the work product resumes, the audit size can again be reduced.
Audits of work products such as peer reviews, problem reports, baselines, releases, et cetera should encompass any sub-tier suppliers or work outsourced to other organizations.
Note that the sampling concept is not the same for a first article inspection. The first article inspection is a comprehensive review of the production data. Since it is the first production article, it is a sample of the manufactured articles to follow. The manufacturing or production quality control system takes over for the on-going aspects of production to ensure that the subsequent hardware produced is the same as the first article.
DO-254 Section 8.1 and especially Section 10.1.6 could imply that process assurance is responsible for conformance or conformity. Process assurance can perform inspections of hardware to ensure that it was built according to drawings and specifications. Conformity for FAA purposes, however, is a formal process performed by an FAA representative or an appropriately delegated individual (or organization).FAA conformity requires a request for conformity, conformity delegation, the conformity inspection, and a statement of conformity. The conformity process is conducted at the system or LRU level which may be the same scope or even higher level of system integration than the aspects covered by DO-254. Project system level certification plans typically describe the plan for conformity and delegation.
Unless otherwise also appropriately delegated by the FAA, the assigned process assurance personnel do not carry out FAA conformity. Process assurance personnel do inspect hardware and associated drawings to ensure that the hardware that will be used in the FAA conformity is manufactured consistent with its drawings.
Process assurance can also perform an in-house conformity for a PLD device. The activity should ensure that the device type and part number is correct, the programming data for programmable devices is the correct part number and version, the programming procedures are documented and followed, and that any device programming verification such as a programming file checksum is confirmed.
Process assurance should perform at least two different types of audits: audits tied to events or activities, and periodic audits. Event specific would include audits performed when phase transition criteria have been met, audits for baseline content, and audits of test readiness. Periodic would include audits of configuration management activities and deviations.
Specific checklists containing the audit criteria are a good method to perform repeatable and documented audits. The Hardware Process Assurance Plan can list the criteria or include the checklists directly or by reference. Once checklists are established for a project, the content and criteria should not be modified without approval from the certification authority.
The hardware process assurance (HPA) audits for a project can be summarized as shown in the example in Table 8.1.
Audits should record the configuration identification of all life cycle data inspected. An audit for a peer review should list the data item that was reviewed and the associated peer review checklist. The audit should check whether all paperwork was filled out correctly, peer review actions were completed, and agreed updates to reviewed documents were made in accordance with the peer review comments. The auditor can also look for any additional changes made to the data item that was the topic of the peer review to make sure no additional changes, beyond the scope and intent of the peer review actions, were made.
A spreadsheet of all project audits, their related status, and pointers to the completed audit reports is a powerful tool for organizing and tracking process assurance activities and progress.
AUDITS OF CONFIGURATION MANAGEMENT
All aspects of the Hardware Configuration Management Plan should be audited by process assurance. This includes:
• Verifying that life cycle data have unique identifiers
• Verifying that baseline contents are correct and complete
• Change control procedures are followed
• Problem reporting is conducted in accordance with the plan
• Changes made to life cycle data is reviewed
• Released documents are stored in the correct format in their respective repository
• Periodic backups are performed on servers that host project data and applications
• Off-site storage is used for backup data or disaster recovery
• Data can be retrieved from servers and applications
• Data can be retrieved and restored from off-site storage
• Data in archives is maintained as long as the equipment is used in service on aircraft
• Tapes or disks used for backups or archives are rotated or updated as needed
• Data storage systems are tamper proof
Process Assurance Audits
Some of the above configuration management tasks could be performed by information services or information technology organizations within a company. Process assurance can look at logs of backups or manifests of tapes delivered to off-site storage facilities. Annual or bi-annual visits to archives for an audit can also be performed.
Problem reports should be audited for compliance to the Hardware Configuration Management Plan. Process assurance can look at new problem reports during regularly scheduled change control board meetings. More detailed inspection of problem reports should check that all fields in the problem report are filled in correctly and that the work is progressing toward resolution. The problem report should also be checked to confirm that the proper configuration identification of all life cycle data affected by the problem when the problem is initially detected is recorded. Process assurance can then later check that the configuration identification of all life cycle data updated by the resolution of the problem was recorded. The completion of verification activities such as peer reviews or regression tests cited in the resolution of the problem should also be checked.
Another helpful task for process assurance is to assist in in-house dry runs of FAA SOI audits before the formal audit with the certification authority takes place. The respective SOI audit checklist from the FAA Hardware Job Aid can be used as a guide to make sure all the activities for the stage of development are complete, the associated verification activities have been performed, and all life cycle data is managed in the configuration management system. Corrective actions can be formulated and implemented prior to the authorities discovering problems during the course of the audit. The dry run activity also positions process assurance as a knowledgeable and useful asset in the formal audit.
The most common term for the inspection of hardware at the end of the development life cycle as it transitions to the production environment is First Article Inspection (FAI).The FAI is an examination of the components, assemblies, and their drawings. The first in the First Article Inspection refers to the hardware item being built with the production processes before the serial production cycle begins. In other words, this is the first hardware off the production line.
The FAI ensures that the documentation needed for manufacturing and serial production of the hardware is correct. The inspection also ascertains whether the hardware was built in accordance with the documentation, processes, and procedures.
Airframers and customers may have detailed requirements for FAI. Process assurance should familiarize themselves with customer FAI requirements and help their company adapt and prepare for the event.
Any non-compliance with agreed and approved hardware management plans and standards needs to be first detected, then agreed and tracked. Process assurance is the keeper of the deviation process.
Deviations should be recorded and authorized by engineering and management as required. Corrective actions defined in the resolution of the deviation should be implemented and tracked to ensure that the agreed action is undertaken.
Deviations should also have an agreed escalation process to elevate visibility to management when corrective actions are not defined or not followed. Process assurance should have the autonomy to bring issues to managements’ attention without fear of reprisal.
Airframers and customers may have detailed requirements for deviations. Process assurance should familiarize themselves with customer requirements for deviations and help their company comply with the expectations.
Work packages and sometimes even whole development projects are sometimes shared with other organizations for schedule and/or budget reasons. The organizations can be another division within a company or a different company altogether. The companies may also have geographic differences—anywhere from a different city to another state or different country.
Process assurance needs to be involved with audits of work done for the project, regardless of the location of the engineers. Some projects may have specific certification requirements for performing on-site audits of work outsourced to another company or division within a company.
When process assurance activities are outsourced to the sub-tier supplier, process assurance at the originating organization still needs to be involved to audit the process assurance at the sub-tier supplier. The process assurance activities at the sub-tier could use the Hardware Process Assurance Plan as the originating organization or have their own in-house Hardware Process Assurance Plan. Regardless of which plan is used or which organization performs the audits and inspections, process assurance at the originating organization will have responsibility and accountability for meeting their DO-254 related objectives.