What Is the Response Quality Score?
The Response Quality Score is a comprehensive indicator that evaluates the reliability of response data through an algorithm developed in-house by Opensurvey. This score is calculated based on two core elements:
Pattern-based score: Measures response sincerity by analyzing response patterns such as response time and straight-lining.
AI response quality score: AI directly analyzes open-ended response content to evaluate its relevance to the question and the quality of the answer.
When the 'AI Response Quality Inspection' feature is enabled at the time of collection group creation, a final Response Quality Score is calculated that takes both scores into account. Even when the feature is turned off, you can still check response quality through the pattern-based score.
Why Is the Response Quality Score Needed?
1️⃣ When you need to verify whether responses are meaningful
In online surveys, some respondents may answer without fully reading the questions, or enter meaningless content.
"Open-ended questions received a lot of responses like 'haha', 'don't know', or 'none'.
How should I assess these responses? 🤔"
By using the Response Quality Score, AI evaluates the sincerity of responses by considering both the response content and the intent of the question. This allows you to assess objectively how carefully each response was written.
2️⃣ When you need to reduce data inspection time in large-scale surveys
The more responses are collected, the more time is required for data inspection. Especially when there are many open-ended responses, it takes considerable time to review all of them individually.
"There are thousands of responses, and it takes too long to inspect the data.
Is there a way to conduct the inspection process more efficiently? 🤔"
When AI Response Quality Inspection is enabled, AI assesses quality first, allowing you to quickly identify responses that need review and reducing inspection time so you can focus more on the analysis itself.
Response Quality Score Calculation Criteria
The Response Quality Score is calculated as follows, depending on whether 'AI Response Quality Inspection' is enabled when setting up the collection group.
AI Response Quality Inspection ON: Calculated by combining the pattern-based score and the AI response quality score
AI Response Quality Inspection OFF: Calculated using only the pattern-based score
AI comprehensively evaluates answers that are clearly unrelated to the question, responses consisting of meaningless single characters or special symbols, and content that is logically difficult to interpret, by analyzing both the intent of the question and the response content together. The score is deducted each time a problematic element is detected, and the final score is classified as follows:
Reliable: 70 points or above
Review recommended: 60 points or above and below 70 points
Suspected insincere: Below 60 points
How to Check the Response Quality Score
Step 1. Select the survey for which responses have been collected, then navigate to the [Responses] tab.
Step 2. In the Responses screen, you can check the 'Overall Quality Score' and 'Quality Assessment Basis' columns. For responses below 70 points, you can check the specific reasons and related questions in the 'Quality Assessment Basis' column.
💡 Usage tip | If a quality inspection is in progress, the Response Quality Score will be displayed as 'Calculating' and the Quality Assessment Basis as 'Analyzing'. Quality inspection typically completes within an average of 5 minutes.
💡 Usage tip | If you want to review the actual response content in detail, click the [Download] button to download the response data and review it together.
Have your questions about the Response Quality Score been resolved?
If you have questions about why a specific response received a low score during the data cleaning process, or if you need help setting cleaning criteria, please feel free to contact us at any time via the [Help Center icon] in the bottom right corner of the screen.
Our team will do everything we can to help you resolve any difficulties you're experiencing.