📌 The order randomization feature is available on all plans.
What is Randomizing the Order of Blocks and Questions?
Randomizing the order of blocks and questions is a feature that sets blocks or questions included in a survey to be displayed to respondents in a different order every time.
Why should you use Randomizing the Order of Blocks and Questions?
1️⃣ Increase data reliability by reducing order effects
In a survey, an 'order effect' can occur where the presentation order of questions or choices itself affects the response. In this case, responses different from actual perceptions or experiences may be collected, causing bias in the results.
For example, a response to a question presented earlier may become the judgment criterion for a later question, or if similar questions are placed consecutively, responses may be made insincerely due to decreased concentration or fatigue as the survey progresses.
To minimize these effects, a randomization feature is provided that exposes the presentation order of blocks or questions randomly, helping to collect more balanced and reliable data.
2️⃣ Necessary logical flow and context can be maintained as is
Even when applying randomization, there may be cases where the logical flow or context of the survey must be maintained. In such cases, you can set it by grouping related questions into one group and randomizing only the order between groups or randomizing the question order only within a group.
Through this, flexible survey design is possible, reducing response bias caused by order while maintaining the understandability of the survey.
How to Randomize Block Order
📌 Block order randomization can only be applied to general blocks and is not applied to screening blocks.
Randomizing individual blocks
This is a method of randomly arranging the order of all individual blocks included in the survey. Please set it using the method below.
step 1. Click [Set block order randomization] at the bottom of the question list on the left.
step 2. Select Randomizing individual blocks as the randomization method and click [Confirm].
step 3. Once the setting is complete, 'Block order randomization ON' is displayed at the bottom of the question list on the left, and you can check the setting conditions by hovering your mouse over it.
Randomizing blocks by group
This is a method of designating related blocks as a group and randomly arranging the order between groups or within groups. Please set it using the method below.
step 1. Click [Set block order randomization] at the bottom of the question list on the left.
step 2. If you select Randomizing blocks by group as the randomization method, you can designate groups. After selecting the blocks to be grouped, click the [Add] button to create a group.
💡 Usage tip | If you want to unassign a group, please use [Reset group assignment].
💡 Usage tip | Non-consecutive blocks cannot be grouped.
Example: Grouping Block 3 and Block 5 → Impossible
Example: Grouping Blocks 3, 4, and 5 → Possible
step 3. The created groups are displayed in the list below. Please set them according to the desired randomization method.
Randomize block order within group: To randomly arrange the blocks inside a group, activate the toggle to the right of the corresponding group.
Randomize order between groups: To randomly arrange the presentation order between groups, activate the [Randomize order between groups] toggle at the bottom.
Both randomization options can be set simultaneously. In this case, both the order between groups and the blocks within each group will be presented randomly.
💡 Usage tip | If you want to turn off the randomization setting, change the randomization method to Do not set in the settings screen. You can readjust the settings at any time as needed.
step 4. Once the setting is complete, 'Block order randomization ON' is displayed at the bottom of the question list on the left, and you can check the setting conditions by hovering your mouse over it.
How to Randomize Question Order within a Block
📌 Question order randomization can only be applied to general questions and is not applied to screening questions.
Randomizing individual questions
This is a method of randomly arranging all questions without setting groups. Please set it using the method below.
step 1. Click [Randomize question order within block] on the right of the block containing the questions to be randomized, or select Randomize question order within block from the More (⋮) menu of the block.
step 2. Select Randomizing individual questions as the randomization method and click [Confirm].
step 3. Once the setting is complete, a message saying 'The order of all questions in this block will be exposed randomly.' is displayed at the bottom of the block.
Randomizing questions by group
This is a method of designating related questions as a group and randomly arranging the order between groups or within groups. Please set it using the method below.
step 1. Click [Randomize question order within block] on the right of the block containing the questions to be randomized, or select Randomize question order within block from the More (⋮) menu of the block.
step 2. Select Randomizing questions by group as the randomization method, then designate the questions to be grouped in 'Group assignment' and click the [Add] button to create a group.
💡 Usage tip | If you want to unassign a group, please use [Reset group assignment].
💡 Usage tip | Non-consecutive questions cannot be grouped.
Example: Question 3 and Question 5 → Impossible
Example: Questions 3, 4, and 5 → Possible
step 3. The created groups are displayed in the list. Please set them according to the desired method.
Randomize question order within group: To randomly arrange the question items inside a group, activate the toggle to the right of the corresponding group.
Randomize order between groups: To randomly arrange the presentation order between groups, activate the [Randomize order between groups] toggle at the bottom.
Both randomization options can be set simultaneously. In this case, both the order between groups and the questions within each group will be presented in a randomly arranged order.
💡 Usage tip | If you want to turn off the randomization setting, change the randomization method to Do not set in the settings screen. You can readjust the settings at any time as needed.
step 4. Once the setting is complete, the setting conditions will appear at the bottom of the block.
💡 Usage tip | If randomization settings are used together with question display logic, question skip logic, piping conditions, etc., the randomization feature may be restricted if it does not fit the survey flow.
💡 Usage tip | Even when adding question display logic, question skip logic, piping conditions, etc., to questions where randomization has already been applied, the randomization setting may not be applied if it does not conform to the survey flow.
💡 These features are also available in the Dataspace Editor screen.
If you want to check if the randomization settings have been applied correctly, please utilize the [Preview] feature at the top right. Each time you run the preview, blocks or questions are presented in a random order, allowing you to check in advance the survey flow that actual respondents will see.
Frequently Asked Questions
Q. Can users directly set the order of randomized blocks or questions, or check in what order they were shown to respondents?
Q. Can users directly set the order of randomized blocks or questions, or check in what order they were shown to respondents?
A. Randomization is automatically applied according to the internal system logic. Therefore, users cannot directly specify the exposure order or ratio, nor can they separately inquire about the actual order in which they were presented to each respondent. Please note that the randomization feature is intended to reduce response bias caused by order.
Do you fully understand the randomization settings?
If you still have questions after reviewing this guide, please feel free to contact us at any time through the [Customer Center icon] at the bottom right of the screen.