| A draft version of AS9101D is out for final review | | | | from the section highlight where this audit methodology |
| before publication. It provides tremendous insight as to | | | | focuses. |
| how the new AS9100C standard will be audited. This is | | | | 4.1.2.2 Organizational Leadership |
| an excellent document which really only describes | | | | There should be an interview(s) with top management |
| normal/best auditing practice but hopefully it will | | | | to evaluate the establishment of policy, objectives and |
| revolutionize AS9100 and then probably the rest of the | | | | about commitment. - This is not the only place where |
| ISO industry. It mandates behavior that is likely to | | | | an "interview with top management" is discussed. It is |
| ensure better and more effective auditing - so long as | | | | hard to believe but perhaps some auditors have not |
| the auditor and registrar community is capable. Training | | | | been insisting on meeting top management. It will be |
| has been mandated for auditing the new AS91000C | | | | hard to avoid now. |
| standard in an attempt to ensure competency and it | | | | 4.1.2.3 Quality Management System Performance and |
| will to some extent be based on AS9101D - despite | | | | Effectiveness |
| the fact that the training is already being developed. | | | | Methodology subjects include a review of the |
| Here is a summary of the key points and a link to the | | | | processes for complaints, internal audits, stakeholder |
| full review of this draft. | | | | feedback, nonconformances, preventive actions, |
| Process Approach - Strong emphasis on an | | | | management review meetings and performance to |
| understanding and use of a process approach by | | | | targets. These subjects are going to be addressed in |
| organizations. If your documents are not process | | | | many other areas (the methodology does not say |
| based you should perhaps worry. | | | | how they will be met). The key will be to ensure they |
| Auditing Methodology - Strong emphasis on 6 specified | | | | are linked where appropriate and evidence complete |
| methods of auditing which outline key issues which are | | | | to demonstrate the methodology has not been ignored. |
| "recommended" to be included in an audit. | | | | 4.1.2.4 Process Management |
| More audit time needed - It is "expected" that the | | | | "The audit team should conduct quality management |
| previously specified "total" audit time is now applied as | | | | system audits using a method that focuses on |
| "on-site" time. Time will be added for off-site activities | | | | process performance and effectiveness; this ensures |
| which are also about to increase. AS9101D specifies | | | | that priority is given to the following:a) reviewing the |
| lots of extra information needed prior to audits and | | | | organization's processes, their sequence and |
| additional information and activities during the audit | | | | interactions, and performance against requirements....." |
| which is going to ensure a thorough and effective audit | | | | Also mentioned, the auditor is to look for defined |
| - but also a longer and more intensive audit. | | | | measures with a focus on processes that directly |
| Objective Evidence Record (OER) - although there | | | | impact the customer. |
| was talk about removing the laborious checklist that | | | | 4.1.2.5 Process Performance and Effectiveness |
| was previously mandated - it is still there in the current | | | | Measurements that indicate or directly measure the |
| draft standard. While this is potentially a barrier to | | | | effectiveness of processes. The example quoted is |
| process based auditing (which will cause much | | | | KPI's (Key Performance Indicators). In truth ISO has |
| confusion to some auditors) it is the only way that | | | | always stressed the importance of this. |
| anyone will be able to see that the auditor has done a | | | | 4.1.2.6 Continual Improvement |
| good job of assuring every requirement of the | | | | "The audit team should evaluate the organization's |
| standard has been audited. Without this or an | | | | interrelated processes and activities for continual |
| equivalent audits will simply fail. | | | | improvement of the quality management system, its |
| Process Effectiveness Assessment Report (PEAR) - | | | | processes, their conformity, and effectiveness in order |
| Auditors are now required to assess the | | | | to: |
| effectiveness of processes in an organization. Not all | | | | ensure focus on issues that are important to the |
| processes will be subject to this assessment but | | | | organization, their customers" |
| certainly all important ones. This is an obvious need | | | | Many other items are listed in the continual |
| during an audit but this is going to be beyond the ability | | | | improvement methodology but this is where the focus |
| of many auditors who just audit to checklists. The way | | | | is. Note also that improvement is a process and not |
| to ensure you stay clear of problems is to do this on | | | | incidental and unrelated improvements that might have |
| behalf of the auditor and make it indisputable. | | | | occurred. And the focus is on things that are important. |
| Perhaps the final note to mention here is that there are | | | | More Information Required by Registrars |
| rumors that AS9101D publication is delayed. This wont | | | | Before the audit, registrars must obtain more |
| cause a problem to the overall progress of AS9100C | | | | information in order to be able to "take into account" |
| release and upgrading as the aforementioned | | | | results of internal audits, performance measures, |
| sanctioned training is not expected to be "available" | | | | previous Management Review results and the |
| until April. Unfortunately nobody really knows the | | | | proportion of aerospace (and defense) customers |
| timescales as the key milestones are dependent upon | | | | compared to total revenue. |
| the results of committees voting. | | | | All of this information must be obtained before the |
| Here is a more detailed description of what you might | | | | audit starts so that planning can be conducted |
| have to do and how your audit might be performed. | | | | appropriately. Registrars will have to spend more time |
| The Process Approach | | | | on this activity (and it will have an impact on their |
| The process approach has always been "out there" | | | | customer service, response and probably costs. |
| as a concept when developing management systems. | | | | During the Stage 1 audit they are required to review |
| Cavendish Scott, Inc. has been using the process | | | | risk assessment methods, customer satisfaction data, |
| approach since 1985. We didn't call it that but it just | | | | special processes, preventive programs e.g. FOD, in |
| made sense that the right way to build a management | | | | addition to everything that they normally cover in a |
| system was to structure it around the organization and | | | | stage one (e. g. internal audits). Again more time and |
| not copy the requirements of a standard. Unfortunately | | | | perhaps more cost (but this should assure audits that |
| many organizations saw fit to base their management | | | | get to stage 2 are more successful). |
| system on the standard, structuring it around the | | | | Stage 2 Audits are required to include a review of all |
| numbering and titles of the standard and not really | | | | new [aerospace] customers, a review of customer |
| describing their own processes. Somehow this was | | | | satisfaction, special processes, continual improvement |
| satisfactory for the registrars. In 2000, ISO tried to | | | | of the QMS and an interview with top Management. |
| rectify this by describing the process approach. Then | | | | This is not particularly a significant increase over |
| they back-tracked declaring that although a process | | | | previous "expectations" although some additional effort |
| approach is contained within their model, they were not | | | | will be required. |
| prescribing any method and that organizations already | | | | Process Efficiency Assessment Report (PEAR) |
| ISO registered would not need to rewrite their | | | | This requirement is new. It is hoped that sanctioned |
| procedures. Currently ISO is in decline (there are less | | | | training will include substantial training for auditors as |
| certificates in the US than there were last year) and | | | | judging the effectiveness of a process that you get to |
| one of the reasons is that organizations saw little | | | | see for a matter of minutes (rarely hours) is quite a |
| benefit - and so did their customers. Part of the blame | | | | task. However, the form that records the PEAR |
| for this is that a system based around the | | | | provides a simple approach and this is where well |
| requirements of a standard is difficult to maintain and is | | | | organized companies will address this issue before the |
| unlikely to yield benefits. Further, registrars were | | | | audit and make the auditors life easy. Choosing the |
| reluctant to ask organizations to change - for fear of | | | | right measures for effectiveness will be important. |
| losing business and accreditation agencies too didn't | | | | Inappropriate measures are likely to be dismissed but |
| want to rock the boat. Now for instance the | | | | measures that take too much effort may be |
| accreditation agencies are getting tough on root cause | | | | unpopular. The following extract outlines the |
| corrective actions despite the fact that this has been | | | | importance of the PEAR. |
| an issue since the original issue of the standard at in | | | | "The results of effectiveness shall be recorded on the |
| 1987. However, the accreditation agencies still don't | | | | PEAR (see Appendix C) for each audited product |
| press registrars to insist on a process approach. | | | | realization process. The level of effectiveness for |
| Clearly the aerospace industry is not putting up with | | | | each process shall be recorded on the PEAR |
| the inadequacies of the accreditation or registration | | | | (statement of effectiveness level). If the level of |
| processes. AS9101D is their attempt to address the | | | | effectiveness has been classified as a '2' or a '1', this |
| process model issue. This is a direct quote from the | | | | shall result in a nonconformity being issued against |
| draft.n0.2 Auditing Approach | | | | 9100-series standards clauses 4.1.c and f (see clause |
| This standard supports the engagement and | | | | 4.2.4)." |
| evaluation of an organization's quality management | | | | Note that if you are issued a PEAR with a level 3, it is |
| system process approach, as required by the | | | | a reasonable expectation that during the next audit, |
| 9100-series standards. When evaluating an | | | | that process will be at a level 4. "Appropriate" actions |
| organization's quality management system, there are | | | | should ensure that. |
| basic questions that should be asked of every | | | | The following is a listing of the key fields in the PEAR |
| process, for example: | | | | form. At the bottom is the classification of the |
| - Is the process identified and appropriately defined? | | | | effectiveness level mentioned above. |
| - Are responsibilities assigned? | | | | Process Name: |
| - Are the processes implemented and maintained? | | | | Process details, including associated process |
| - Is the process effective in achieving the desired | | | | interfaces: |
| results? | | | | Applicable 9100/9110/9120 clause(s): |
| This, the new auditing methodology (see details later) | | | | Organization's method for determining process |
| and requirements throughout the AS9101D standard | | | | effectiveness: |
| are pressing for organizations to understand a process | | | | Auditor observations and comments supporting |
| approach. If your system is based on the standards | | | | process effectiveness determination: |
| requirements it does NOT demonstrate that you | | | | Statement of Effectiveness Level: |
| understand the process approach, implemented | | | | The process is: |
| processes and process based control. Given this it is | | | | |
| hard to understand how an auditor will not generate | | | | 1. Not implemented; planned results are not achieved. |
| nonconformances on the effectiveness of the system. | | | | 2. Implemented; planned results are not achieved and |
| If there is no procedure (or flowchart or process | | | | appropriate actions not taken. |
| diagram or something) describing your production | | | | 3. Implemented; planned results are not achieved, but |
| process, for instance, then you have not met the 4.1 | | | | appropriate actions being taken. |
| requirement to define your processes. Whether | | | | 4. Implemented; planned results are achieved. |
| nonconformances get generated remains to be seen. | | | | Objective Evidence Record |
| Auditors have accepted poor process definitions for | | | | Although there were rumors that the checklist was |
| years. Given that without good process definitions, | | | | going to be removed, it is still included in the current |
| measurement, objectives and improvement are not | | | | draft although in a different format. This is a good thing |
| easy to achieve, these nonconformances are long | | | | - despite what many auditors claim. Without this, there |
| overdue. | | | | is insufficient evidence to demonstrate what was |
| Auditing Methodology | | | | audited. If any problems occur it would be impossible to |
| This standard identifies 6 approaches that "CAN" be | | | | follow up and see what the auditor looked at and |
| used to conduct audits. The note following that | | | | possibly clear them of blame. Registrars are allowed |
| statement implies that this is expected as a minimum. | | | | to have their own checklist so long as it meets the |
| This probably means that registrars cannot ignore the | | | | same intent. |
| methods or they will get in trouble. They will have to | | | | Nonconformances |
| generate tools to help them ensure the audit utilizes | | | | "Recurrence of the same or similar nonconformity |
| these methods (as a minimum) and generates | | | | found during consecutive audits at a particular site |
| evidence to prove they have. It is complicated to | | | | location shall be considered as a failure of the |
| integrate these methods into the audit approach | | | | corrective action process (see 9100-series standards |
| currently used which almost exclusively relies on the | | | | clause 8.5.2) and shall result in a major nonconformity |
| existing AS9100C checklist. Consequently expect | | | | being issued" |
| some registrars and/or auditors to simply do these | | | | Inadequate corrective action will no longer be tolerated |
| methods/approaches in addition to the checklist. | | | | in aerospace quality management systems. |
| Perhaps the mandated training will address integration | | | | Conclusion |
| of these methodologies with the checklist but even so | | | | Many of the issues introduced here represent a |
| it is a difficult concept. | | | | substantial change in auditing content and approach. In |
| The following extract includes the introduction to this | | | | many respects this is disappointing because what the |
| Audit Methodology and then lists the he six | | | | standard is calling for is mainly good auditing practice |
| methodologies by title and gives a brief (not verbatim) | | | | by competent and capable auditors. Auditors are |
| description including extracts. Some of the | | | | witnessed and it is possible that many auditors will be |
| methodologies include lengthy descriptions with many | | | | prohibited to audit if they are unable to demonstrate |
| items listed. Here they are summarized to give an | | | | best practice auditing. This too is a good thing as an |
| example of what they are about. | | | | industry as important as the aerospace industry |
| 4.1.2 Audit Methodology | | | | deserves good auditors to ensure effective quality and |
| The following approaches identify different audit | | | | to drive improvement up and costs down. That is not |
| methods that can be used, as appropriate, to conduct | | | | the case at the moment and this standard has the |
| audits. | | | | opportunity to make substantial, meaningful and |
| NOTE: The identified methods are not intended to be a | | | | beneficial change throughout the aerospace industry |
| complete listing, but represent a significant contribution | | | | and the ISO industry in general. |
| for auditors to evaluate quality management system | | | | Organizations will need to carefully prepare for |
| conformity and effectiveness. Use of these methods | | | | AS9100C. Not necessarily because the standard has |
| will help transition from clause based auditing and put | | | | changed much but because the auditing approach is |
| focus on the actual processes, their effectiveness, and | | | | getting more sophisticated and better. This will |
| their ability to meet the quality objectives. | | | | represent some additional cost and effort for |
| 4.1.2.1 Customer Focus | | | | organizations but with the loss from the supplier base |
| "The audit team should assess whether customer | | | | of those that didn't do it, and the potential benefits that |
| satisfaction is adequately evaluated and appropriate | | | | can be achieved with ISO/AS with the right focus, |
| actions are taken". "Customer feedback is a process | | | | companies who put that effort in early are likely to |
| and should be audited as a process" - these extracts | | | | reap the benefits. |