Program Basics and Background

• What is the Case for Quality (CfQ)?

  
  The FDA launched the Case for Quality (CfQ) in 2011 following an in-depth review of device quality data and feedback from both FDA and industry stakeholders.
  
  The FDA then enlisted the Medical Device Innovation Consortium (MDIC) in 2014 to foster a new level of collaboration between the Agency and industry through a Broad Agency Agreement.
  
  The Case for Quality aims to elevate the focus of all medical device stakeholders from baseline regulatory compliance to sustained, predictive practices that advance medical device quality and safety to achieve better patient outcomes.
  
  Participants in the Case for Quality include FDA and a growing number of medical device companies, healthcare providers and payer organizations, as well as trade associations and educational institutions.
  
  The Case for Quality offers a unique forum for medical device stakeholders to work together on an umbrella of initiatives. One such initiative is the Case for Quality Voluntary Improvement Program (CfQ VIP).
  
  Read the detailed descriptions of the Case for Quality on:
   

  • The FDA Case for Quality Landing Page
  • MDIC Case for Quality Landing Page

  
  Additional Resources:
   

  • Watch Dr. Jeffrey Shuren, director of the FDA’s Center for Devices and Radiological Health, affirm the FDA’s commitment the Case for Quality and their goal to improve patient outcomes.
  • Watch MDIC’s overview video of the Case for Quality.
  • Review outcomes from the previous MDIC Case for Quality Forums.

  In summary, the Case for Quality aims to elevate the focus of all medical device stakeholders from baseline regulatory compliance to sustained, predictive practices that advance medical device quality and safety to achieve better patient outcomes.
  
  Participants in the Case for Quality include FDA and a growing number of medical device companies, healthcare providers and payer organizations, as well as trade associations and educational institutions.
  
  The Case for Quality offers a unique forum for medical device stakeholders to work together on an umbrella of initiatives. One such initiative is the Case for Quality Voluntary Improvement Program (CfQ VIP).

• What is the Voluntary Improvement Program (VIP)?

  
  The Case for Quality Voluntary Improvement Program (CfQ VIP) was announced as a pilot (Voluntary Medical Device Manufacturing and Product Quality Pilot Program) in a FDA Federal Register Notice on December, 28, 2017.
  
  CfQ VIP leverages a proven maturity model, the CMMI framework, by which medical device organizations may measure their capability to manufacture high quality devices which improve patient outcomes. By understanding areas for improvement against a structured set of best practices, organizations can then drive targeted continuous improvement activities throughout their facility.
  
  In recognition of manufacturers who participate in this independent appraisal of capability, FDA will adjust that manufacturer’s regulatory engagement activities and streamline some submission requirements.
  
  For additional information, see:

  • MDIC's CfQ VIP Landing Page
  • The FDA's CfQ activities Landing Page (first section)
  • CMMI Institute’s 4-page program overview
  • The FDA's pilot summary White Paper

   

• How does CMMI® fit into the CfQ VIP?
   

  The Capability Maturity Model Integration (CMMI) is a capability improvement model that can be adapted to solve any performance issue at any level of the organization in any industry. The Model provides guidelines and recommendations for helping your organization diagnose problems and improve performance. Used by over 10,000 organizations from more than 100 countries all over the world, CMMI helps you identify and achieve measurable business objectives.
  
  This FDA pilot leverages the CMMI framework as the standard maturity model by which medical device organizations may measure their capability to produce high quality devices for improved patient outcomes. FDA will adjust their engagement activities and submission requirements in recognition of pilot participation, including a discovery appraisal. We will not be using “maturity levels” in this pilot. There are no incompatibilities between quality system regulations and the CMMI framework, however, the appraisal will be of the entire facility rather than just the QMS.
  
  See testimonials from existing pilot participants:

  • Edwards Lifesciences
  • Boston Scientific
  • Medtronic and Baxter
  • Steris and CeloNova
  • Biosense Webster (a J&J Company)

• Why did CDRH choose CMMI for this activity?

  
  MDIC commissioned Deloitte & Touche, LLC to conduct research concerning the use of maturity models in other industries. Their report concluded that Capability Maturity Model Integration (CMMI) should be leveraged for this voluntary pilot program. CMMI is a framework of best practices that includes an appraisal method to ensure objectivity and consistency. This model has been successfully used in various industries —including information technology, healthcare, automotive, defense, and aerospace— to consistently deliver high quality products and reduce waste and defects.

• What is expected from participating and non-participating companies?

  
  Companies choosing not to participate will see no change  in their engagement with FDA.
  Companies may request to enroll into the program through an online application process.
  
  Minimum enrollment requirements:
   

  Part of lifecycle for medical devices distributed in the U.S. (Class I, II, III)

  • This can be any facility which “designs, manufactures, fabricates, assembles, or processes a finished device.”
  • This can include, but is not limited to, “those who perform the functions of contract sterilization, installation, relabeling, remanufacturing, repacking, or specification development, and initial distributors of foreign entities performing these functions.”

  
  Companies should have a:

  1. prior compliance history or profile (FDA inspection/audit or MDSAP),
  2. site registration, and
  3. listing with the FDA.

  
  Facilities must not be under Official Action Indicated (OAI) status or subject to a judicial action. Voluntary Action Indicated (VAI) is acceptable.

  • Companies with OAI can become eligible when they have provided the FDA with confirmation that appropriate corrective action has been implemented, and those actions have been verified by the FDA during a follow up inspection .

  
  Companies may want to seek additional eligibility information if they are

  • a new manufacturer,
  • a start-up company,
  • without a compliance profile,
  • not marketing products in the U.S.

  
  Interested sites may contact the CDRH CfQ team at: [email protected]
  
  Expectations of Participants:
   

  • Annual subscription fee of $5,000 to be paid once you are accepted into the pilot program.
  • Completion of a Medical Device Discovery Appraisal within the target of 90-days and payment of the associated delivery costs.
  • Submission of metrics at predefined frequencies to confirm progress and improvement from initial appraisal results.
  • Sustained favorable compliance profile/history against regulations

   

   

• How is the Appraisal different from an FDA inspection?

  
  The CMMI Medical Device Discovery Appraisal (MDDA) evaluates an organization’s capabilities and activities that are of value to the facility and does not explicitly investigate compliance against the CFR. The MDDA collects evidence from interviews with people who perform the work and not from who usually manage audits and inspections from a “front room.” The purpose of the MDDA is to drive a conversation for how the facility can accelerate prioritized improvements, not address every gap in a corrective action list.

What’s the Benefit to my Organization?

• Why should my company participate? What are the tangible benefits from CDRH? When do I recognize them?
   

  CfQ VIP can be a method to drive improvements such as:
   

  • Increased manufacturing and product confidence
  • Faster time to market
  • Better information to drive regulatory decisions
  • Improved resource allocation
  • A continuous improvement focus on what is most important to patients

  
  As the result of additional granularity and the commitment to transparency, FDA is modifying certain regulatory activities to support participants in the Program to bring innovative devices to market. FDA will forgo routine and risk-based inspections for those facilities that have committed to completing a Medical Device Discovery Appraisal by signing a Statement of Work for the appraisal with CMMI Institute.
  
  Additionally, after appraisal results have been submitted to the FDA, participants will be eligible for streamlined submissions and reviews for the following PMA supplements:
   

  • 30-Day Notices
  • Site Transfer
  • PMA Original manufacturing module
  • FDA may also forego pre-approval inspections

  
  The FDA will also benefit from this program by potentially reducing the internal resources required for evaluation of inspections, improving consistency of the review, and having access to enhanced data analytics of quality performance. The increased FDA capacity can be used for manufacturers who have not demonstrated high capability and to engage earlier in the development life cycle of new products.
  
  The benefits provided can also accelerate the implementation of quality improvements at a faster rate and reduce time to market of product improvements and enhancements for patients.
   

• What does an appraisal cost?

  
  The actual appraisal cost varies depending on the scope of the appraisal, which is determined by examining the variety, complexity, and volume of work done at your site, and sampling products and projects that are representative of those work activities. The enrolled organization is expected to pay all associated costs to CMMI Institute, and should negotiate those costs with CMMI Institute, not FDA or the designated Appraisal Team Lead. The scope of the appraisal will generally be determined after a few conversations with your designated Appraisal Team Lead.

• Will I need to engage with the FDA?

  
  FDA will review all applications and send acceptance letters. Participants should openly communicate with FDA reviewers early and often when using the modified submission documents. Questions can be submitted to [email protected].

Appraisal Activities

• What does the site need to do to prepare for the appraisal? What people from the site are expected to participate in the appraisal?

  
  The Appraisal Team Lead (ATL) will work with the Appraisal Sponsor and Site Coordinator to determine when interviews will be conducted with which group of people. The appraisal team will need at least one conference room for the duration of the onsite period, which is typically 5-days, but which could be longer if appraising multiple sites. Do not manage a front room nor back room. There is no “pass” or “fail”, so tell staff that the more open and honest they are with the appraisal team, the more accurate and useful your appraisal results will be. Remember, the purpose of this activity is to help facilities identify improvement opportunities to accelerate their capability to manufacture high quality devices.

  • Appraisal Sponsor: High-level individual that can support the appraisal and follow-on activities by removing roadblocks and securing resources or funds.
  • Site Coordinator: Individual that will support logistics such as rides, internet, lunch, scheduling appraisal participants, conference room(s), tours, etc.

• How many appraisers participate in the appraisal?

  
  Each appraisal will require a team of at least two, the Appraisal Team Lead (ATL) and an Appraisal Team Member (ATM). To date, most appraisals (65%) will only need a team of two (2-total), but this varies depending on the scope of the appraisal, and some require additional team members to support the work load.

• Can I provide my own appraisers?

  
  Yes! Personnel from participating organizations can become certified as an “Internal Appraisal Team Member (ATM)” and thereby become eligible to serve on the appraisal team. There are a series of steps in the certification pathway for Internal ATMs, including an application, training, exam, observation, and practice. For more information, see the Internal ATM Certification Pathway [link]. It should be noted that all ATLs must be provided by CMMI Institute and be independent of the site.

• How frequently do I need to do a regular appraisal?

  
  At present, annually from the date of your initial appraisal.

Appraisal Results

• What is the heat map tool?

  
  The appraisal tool heat map is a standardized workbook for collecting interview data and documented evidence during the appraisal. Specifically, the heat map is a roll-up of the appraisal results against the model’s practice areas that are under consideration for that appraisal. It is intended to be an easy-to-read visual representation of your organization’s current standing against the CMMI model.
  
  NOTE: There is no minimum score required to “pass” the appraisal. This activity is not a pass/fail exercise, but is rather designed to facilitate ongoing, targeted improvement efforts within the appraised organization.
  
  The FDA will only be provided the cumulative percentages of this heat map, and not the interview information, documented evidence, or results in each practice.

• What will be shared and with whom?

  
  All appraisal results will initially go to the CMMI Institute, which will review the deliverables for quality assurance (approximately 10 - 30 days), then anonymize the data and submit a subsection of the heatmap to the FDA.
  
  The PMO will anonymize and aggregate the data for trends and baseline purposes, those results will only be shared with organizations participating in the pilot. This aggregate, deidentified information could enable participants to measure their capability against industry. Analysis of those trends might be made public but have not at present.

• Am I obligated to correct or improve my organization’s appraisal results?

  
  This is not an inspection/audit. The feedback provided by the appraisal is intended to help improve the effectiveness of the quality system and improve the quality of the medical device product in the hands of the user. The feedback should be prioritized by the organization. The first priority should be impact to patient and the second should be business value.
  
  This pilot encourages organizations to focus their improvement efforts on the areas that are the most critical or impactful to their facility’s needs and objectives. We intend to measure improvement through the Performance Report, wherein quality metrics are expected to go up and down, but trend positively in the long-run. There are currently no expectations for how much improvement should be made in any given timeframe.
  
  Although an appraisal against the CMMI framework will result in characterizations of an organization’s capability, there is no minimum “score” to achieve for continued participation. This activity is not a pass/fail exercise, but is rather designed to facilitate ongoing, targeted improvement efforts within the appraised organization.

Consistency in the Appraisal

• Can I extend the scope of my appraisal to include other CMMI practice areas?

  
  All initial, baseline appraisals must include a standard, predetermined set of 11 Practice Areas (PAs), unless an exception is approved by FDA; however, facilities are able to include additional PAs to their baseline appraisal. In reappraisals, facilities may tailor which Pas to include or exclude as they see fit, except for the core PAs which are always required:

  • Governance (GOV): considers leadership’s ability to set strategy and goals, communicate them to the rest of the organization, and provide resources and training required for the team to be successful.
  • Implementation Infrastructure (II): considers the facility’s habit and persistence around processes so that in times of high stress or turnover, work still gets done as intended and processes aren’t abandoned.
  • Managing Performance and Measurement (MPM): considers the facility’s ability to track and measure success against the goals aligned with strategy.

• Who are the people performing the appraisal? Can I pick who they are?

  
  MDDAP appraisers that have been vetted by the CMMI Institute PMO based on the quality of their appraisal reviews, their knowledge base, and their alignment in the efforts of this pilot to focus on continuous improvement.

• How is CMMI institute providing oversight to appraisal methodology and appraisers?

  
  The CMMI Institute has a historically proven model, specific qualifications and certifications for lead appraisers, and a clearly documented methodology for executing appraisals. In addition, for this Medical Device Discovery Appraisal Program, specific expectations and training have been defined collaboratively by the CMMI Institute, the FDA, MDIC, and industry partners. Appraiser and appraisal team performance are regularly monitored by the CMMI Institute Program Management Office (PMO).

• What happens if I want to express concerns about my appraiser or the appraisal team?

  
  If there is concern before the appraisal, please reach out to the CMMI Institute PMO at [email protected] and the Institute will try to address those concerns then (whether directly with the appraisal team, or by assigning your facility a new team). If there are concerns after the appraisal, please also reach out via that email and we will address those concerns.

Next Steps after the Appraisal

• What happens if I disagree with a finding?

  
  First, the results are based on validated interviews with your employees who are doing the work. Second, we expect that most organizations, if not all, will identify gaps that will help them accelerate their existing improvement initiatives. Third, scores do not impact a facility’s standing in the Program, the appraisal results have no impact to FDA’s interactions with the site and therefore do not lead to any regulatory decisions by the FDA. If there is some other issue with the way the appraisal was conducted, the CMMI Institute PMO’s Quality Analysts will be able to determine that issue and work with the ATL to resolve it. If there is still a concern, please contact the CMMI Institute PMO at [email protected]

• What is the check point process?

  
  The goals of the check point process are to (1) encourage continuous process improvement and (2) encourage engagement in the program. Check points with the CMMI ATL occur at least every quarter but be more frequent if the organization chooses. The Performance Report will be discussed in the first check point with the first draft of metrics expected during the second check point. Check point occurs on the phone/digitally over the course of 1-2 hours.

• What is the Performance Report for used for?

  
  There are three main purposes for these metrics. First, they will help us to understand how industry measures its performance and product quality. Second, insight into these trends will help the Program consider which activities might support industry to improve quality (ex: regulatory modifications, refocus of PAs). Third, reviewing long-term trends will inform whether improvement efforts against the MDDA leads to improved organizational and product performance.

Everything about Enrollment

• Do I enroll my organization or a facility?

  
  Each application is intended to be for a specific facility identified by their FEI#. Organizations can enroll multiple facilities with multiple applications. For value streams represented across multiple facilities, or multiple facilities that are managed by the same team, an appraisal that spans multiple facilities can be considered. This “multi-site appraisal” approach will entail a higher appraisal cost but represents an overall cost savings per site.

• How much does it cost to enroll?

  
  There is a standard, annual $5,000 Program Fee for every facility enrolled, as well as a separate, annual Appraisal Fee. The cost of the appraisal varies based on the variety, complexity, and volume of the work conducted at the facility to be appraised. Year over year, organizations typically see flat or reduced annual costs, but it depends on how they tailor future appraisals [see “What are the expectations for reappraisals?” in the section, “Next Steps after the Appraisal”]. Some organizations have seen enough value from these activities that they have pushed for even more engagement than what is expected in the Program.
  For organizations registered with a Small Business Designation number from FDA, the standard, annual Program Fee is reduced to $3,000 per facility, and the Appraisal Fee is similarly reduced.
  For more information about the cost of participation, please contact [email protected].

• How will I know if I have been approved into the program?

  
  If you are approved into the program, the FDA will send you a letter of approval (via email) within 5 business days of receiving your application.

• What if I am not approved into the program?

  
  If you are denied from the program, the FDA will send you a letter of denial (via email) within 5 business days, containing additional information about the reason(s) for ineligibility, and the steps for resolving any issues.

• What is the longest that I can wait before my appraisal?

  
  The current target is for a facility to complete the appraisal with 90-days from the date they were approved into the program. If an extension is required, the request will be considered by the CMMI Institute PMO and the FDA.