Quality Assurance
Before diving into your analyses or product development steps - you’ll want to assess the quality of your data according to quality assurance and quality control processes and standards that are relevant to your project.
Impactful and effective quality assurance requires setting up systems and thinking about quality long before the data analysis phase - be sure to start thinking about project and data quality early and take the time to get effective systems in place. See the sections below for more guidance and resources on how to do so with equity in mind.
Quality Assurance/Quality Control (QA/QC) Grounding
According to the Water Boards Quality Assurance/Quality Control Website:
Quality assurance and quality control are components of a quality system. A quality system is a structured system that describes the policies and procedures for ensuring that work processes, products, or services satisfy the user’s specifications and expectations. A quality system is the means by which an organization manages its quality aspects. These aspects are Project Management, Data Generation and Acquisition, and Assessment and Oversight.
Quality Assurance (QA) involves an integrated system of management activities that involves planning, implementation, documentation, assessment, reporting, and quality improvement to ensure that a process, item, or service is of the type and quality required for the project needs.
Quality Control (QC) involves a system of technical activities that measures the attributes and performance of a process, item, or service against defined standards to verify that they meet the stated requirements established by the customer. QC also involves analytical frequency requirements and control limits.
QA/QC with an Equity Lens
Setting up your QA system with an equity lens requires thinking through the systems that are already in place and assessing the extent to which they are supporting the advancement of equity objectives associated with your project. Consider reviewing the pages associated with the aforementioned QA activities to see if your systems are on track:
- Planning: Planning, Data Preparation
- Implementation: Data Collection, Data Processing, Data Analysis, Data Visualization, Preservation & Storage
- Documentation: Describe (aka Documentation)
- Assessment: Discover & Integrate (aka Evaluation)
- Reporting: Data Sharing
- Quality Improvement: If some or all of your QA systems are not in support of your equity objectives, think through what changes you/your team can make to your process NOW to better operationalize equity.
Setting up your QC system with an equity lens requires thinking through how we’re measuring quality/performance and the standards we’re using to assess quality.
How do our current performance measures help us assess the extent to which we’re meeting project objectives?
Do any of our current performance measures make it difficult for us to achieve our equity objectives? If so, how might we modify or eliminate them?
What additional performance measures would better help us assess the extent to which this data-intensive project is meeting the project’s equity objectives?
Do the standards, thresholds, or limits we’re using align with the needs of those facing inequity or disparities - are or likely will be affected by a policy, program, or process? If not, how might we modify the them to better align with those needs?
From the data science perspective, conducting QA/QC of the data requires checking values of the dataset against the aforementioned standards to understand the quality of the data.
Is the metadata and vocabulary associated with each dataset complete, easy to access, and simple enough to be understood by a variety of audiences?
Are the standards we’re using to check the quality of our data sufficient and relevant to our project objectives and/or in support of advancing equity outcomes? If not, why not? How should our project standards be revised so they can be more supportive of equity outcomes?
If data is of insufficient quality and needs to be excluded from future steps - where and how will documentation summarize what measures/standards were used to make such determination and which data were excluded?
Resources
- Water Boards QAQC Webpage
- SWAMP Bioaccumulation Monitoring Program Training Series
- Data Quality and Data Management - Slides 67-80 | Recording Part 3
- Quality Assurance & Quality Control - Slides 5 - 38 | Recording Part 1