What Is the Sincerity Score?
The Sincerity Score is a sincerity indicator calculated by analyzing response patterns, designed to improve the reliability of collected survey responses. It quantifies responses that may be insincere, and can be used as an objective standard for data cleaning.
Score range: 0–100 (higher scores indicate more sincere responses)
Calculation method: Final score calculated by comprehensively evaluating multiple factors including response time, text quality, and answer patterns
Why Is the Sincerity Score Necessary?
1️⃣ Ensuring data quality by identifying insincere responses
Responses that may be insincere — such as meaningless text input or repetitive answers following the same pattern — can be identified, improving the reliability of data used for analysis.
2️⃣ Establishing score-based criteria for data cleaning
Even when it is difficult to review every response individually, you can use the score as a basis for determining cleaning targets, making the standards for data cleaning clearer.
Additionally, the Sincerity Score can be used to set a threshold score for [one-click cleaning], making it easier to exclude insincere responses, erroneous responses caused by misunderstanding questions, and outliers that deviate from overall trends during data cleaning.
3️⃣ Improving analysis efficiency by reviewing score distribution
Without checking each of the many responses one by one, you can quickly identify outliers or suspicious responses through the score distribution, improving the efficiency of the data review and analysis process.
Sincerity Score Calculation Criteria
The Sincerity Score is calculated by comprehensively analyzing multiple response patterns to assess the reliability of a response. The system evaluates response sincerity based on the following factors and reflects them in the score calculation.
Response speed: If the total response time is excessively short relative to the number of questions, the possibility of not having read and answered sufficiently is factored into the score.
Text response quality: If meaningless phrases such as "anything," "yes yes," or random characters are entered in open-ended questions, the response quality may be judged as low.
Text response length: If the length of an open-ended answer is excessively short, the possibility that it does not constitute a sufficient answer to the question is reflected in the evaluation.
Rating response patterns: If only the same score is repeatedly selected across multiple rating-type questions such as NPS or scoring questions, this may be identified as a low-discrimination response pattern.
Multiple-choice selection patterns: If indiscriminate selection patterns appear — such as selecting every available choice in a multiple-select question — this may affect the score calculation.
How to Check Sincerity Metrics by Item
The detailed metrics underlying the Sincerity Score (such as response time and text length) can be viewed using the Load Variables feature. Follow the steps below to load the metrics you need.
Step 1. Click the survey you want to check, then navigate to the [Analytics > Response] tab from the top menu.
Step 2. On the 'Response' screen, you can view the list of individually collected responses. Click the [Load variables] button in the upper right of the screen.
Step 3. Select 'Metadata,' then click the [Show hidden variables] checkbox at the bottom of the list. The detailed sincerity-related metrics calculated by the system will then also be displayed.
Step 4. Referring to the meaning of each item, select the checkbox for the metric you want to view, then click the [Select] button at the bottom. The metric will be added to the response list.
sincerity_score: Final calculated response sincerity score (0–100)
resp_elapsed: Total response time
resp_avg: Average response time per question
eval_var: Variance in rating responses (used to identify responses where scores show almost no variation)
text_len_avg: Average length of open-ended responses
text_insincerity_cnt: Number of text responses judged as insincere
Step 5. The columns added to the response list let you check detailed sincerity-related metrics for each response. Using this information, you can select responses that fall below a certain threshold or determine which responses require data cleaning.
💡 Usage tip | The standard for a 'sincere response' may vary depending on the company or research purpose. It is best to start by reviewing text response content and response speed together, rather than judging by score alone. Based on this, set an appropriate threshold score suited to the characteristics of your project.
Have your questions about the Sincerity Score been resolved?
If you're wondering why a specific response received a low score during the data cleaning process, or if you need help setting cleaning criteria, please contact us anytime via the [Customer Support icon] in the bottom right corner of your screen.
Our team will do its best to help you resolve any difficulties you're experiencing.