Surveys and experience sampling are critical components of most studies. Therefore, it's very important to make sure they are programmed correctly into the Ethica Study, prior to the deployment. In this article, we explain how you can test the flow and triggering logic of your survey.
For each survey, it's necessary to test two elements: the survey flow (such as questions, paginations, skip patterns, or average completion time), and survey triggering logic (when the survey is expected to be prompted). The following sections describe how you can test each of these components in details.
Using Survey Preview
The fastest method to test the survey flow is via the Preview tab on the Survey Editor. The Preview tab allows you to check your survey flow while programming it, without leaving your browser. The aim in designing the Preview tab is been to emulate the Ethica app's behaviour as closely as possible, so you will have a feeling how participants will experience your survey.
The Preview tab simulates a smartphone's user interface and allows you to see how your surveys will look like on participants' Android or iPhone device. All question configurations, enabled/disabled status, page orders, skip patterns, and other elements of your survey are handled in the Survey Editor's Preview, as similar as possible to the Ethica app.
While the Survey Preview simulates conditional expressions based on current survey's questions, it cannot evaluate expressions based on questions from other surveys in the current study. References to questions from other surveys in your criteria can yield unpredictable behaviour in the Survey Preview.
Using Ethica app
While Survey Preview is very helpful to quickly evaluate the survey behaviour, it's always helpful to test the survey flow on an actual phone before deploying it. Assuming you already have access to an iPhone and an Android device, you can test your survey by registering in your study as a participant.
The challenge here is that depending on the triggering logic of your survey, Ethica app might not present the survey to you right away (for example, for scheduled trigger surveys), or it might not present it more than once (for example, for baseline surveys). To overcome this limitation, you can temporarily change the Survey Type to Generic and Triggering Logic to User Triggered. This way, when you register in your study as a participant, Ethica app will present a button for the survey, which you can use to start the survey as many times as needed and respond. When you are comfortable with the flow of the survey on the phone as well, you can change the logic back to the original Survey Type and Triggering Logic.
While testing the survey on the phone, any changes made via the Survey Editor has to be published to the phone. This is because the phone always downloads a copy of the survey content and will use it. As you change the content on the website, the downloaded content will be outdated and has to be updated before testing it again.
When you finished editing the survey and pressed Publish, you can update your phone's survey in one of the following ways:
When the Survey Editor asks whether you want to "update current participants' devices", you can choose either "Yes. Update devices for all currently-enrolled participants." or "Yes. Only update devices for the following participants." Both these options will refresh the content of your phone.
Using your phone, you can open the Ethica app, go to Settings, and tap of Update Studies.
Both of these methods will reload all study configuration from the server and receive the latest version from the website.
Note that before updating your study on your phone, make sure you do not have pending survey sessions (survey sessions awaiting your response through the phone).
Survey Triggering Logic
Testing the triggering logic of the survey, when it's set to "Scheduled Trigger", is slightly more complicated. Initially, as you set the schedules under which the survey should be prompted, the Survey Editor shows you a set of examples which the survey could be triggered, assuming the participant joins on the specified date (as shown in the image below).
When you test this logic on the phone, there is no sign on Ethica app indicating when you should expect the survey to be triggered. But Ethica does send the data on triggering schedules to the server. These data show for a given participant when the survey is expected to be triggered. As of now, Ethica does not show this data on the online dashboard. We are planning to add it as soon as possible.