Using Open Data Kit (ODK) in the field

When in the field conducting environmental or ecological surveys, a record of events and subjects is often collected for future reporting and analysis. This has traditionally been a physical record, collected using pen and paper. In recent years, physical records are being digitised, and the process of data collection itself is now often digital. Digital records of events and observations in the field can now be collected using forms on mobile phones, even when the devices are offline.

OpenDataKit (ODK) is a suite of open source tools that aids in the collection and storage of survey data, including audio, video and imagery among others. The form can be customised as necessary; for example, it can have mandatory questions, allow only specific inputs as responses and loop question threads. These features help reduce the inherent error in collecting data in the field. In this blog post, I will explain the customisation features and provide the steps to build a basic form in ODK.

ODK Flow Illustration

Getting started

There are two ways to create customised survey forms for ODK.

  1. Using ODK Build.

  2. Using XLS Form.

ODK Build is a browser-based tool that allows users to drag and drop blocks together to create forms. While it is a great way to begin using ODK, it is not as powerful or customisable as using the XLSForm method.

To use XLSForm, one needs access to the XLSForm software itself, which is free and open-source (Online version here and offline version here). Essentially, the user creates a form using spreadsheet software that outputs the .xls format, which can be accomplished with both MS Excel as well as with Google Sheets. This form is then processed through the XLSForm software, to create the mobile-friendly survey file (in .xml format). The form itself has to comply with a set format and syntax; for example, ‘type’, ‘name’ and ‘label’ are mandatory columns that must exist for the spreadsheet to be successfully converted into a form, while other columns are optional. 

The basic format required is detailed in the XLSForm documentation (XLS Form document). In addition, I found this sheet called widgets quite helpful, as it provides an exhaustive list of options to populate the ‘type’ column.

Mock form using XLSForm

In this section, I’m sharing the steps of building a simple ODK form using XLS form. I’m assuming there’s a plot of trees owned by the surveyees. Now, say one wants to conduct a survey, to collect data of individual trees in the plot and data about the plot. I’ve listed a few probable questions which serve as good examples to build a customised form.

  1. Open a new workbook/spreadsheet. Name the first three columns as ‘type’, ‘name’ and ‘label’. Name the worksheet as ‘survey’. All the column and worksheet names are case sensitive. The following question uses a ‘start-geopoint’ widget. This forces the android phone to start location service.

2. Using two more columns: ‘constraint’, ‘constraint_message’.

3. Add another column named ‘required’. Required indicates that it is a mandatory field/question. This question uses a text widget as shown under the type column.

4. Ask the following question only if the plot size is > 500 sq km and for that add another column ‘relevant’.

The above question is asked only if the plot size (previous question) is more than 500 - a conditional question. There are many ways of asking such questions. For example, one can repeat a set of questions based on certain conditions and other such. This is a type of dynamic form as questions change based on conditions.

5. Ask if the plot is owned or rented - a yes or no question - this is to give an example of using ‘choices’ worksheet. Leave the other columns (constraint,relevant, etc.) blank when not required.

Create the choices worksheet as shown in the form definition template. Enter the following in the ‘choices’ worksheet. 

Also take note of the ‘settings’ worksheet Fig3 - the ‘version’ column of this worksheet must be different for every draft of form for it to reflect the changes in the ODK Collect App.

Fig. 01 Survey Worksheet

Fig. 02 Choices Worksheet

Fig. 03 Settings Worksheet

Mock form using ODK Build:

The easier way of building forms is by using ODK Build. Forms can be build by dragging and dropping appropriate widgets. One example is given below:

Fig. 04 ODK Build Mock Form

Use ‘Add New’ at the bottom left of the screen to add new rows of widgets. This toolbar contains the different types of widgets and is similar to populating the ‘type’ column of XLS Form. The properties of the widget can be described at the Property Dialogue box of each widget which opens to the right of the screen. Data name is similar to the ‘name’ column in XLS Form. Other conditions and columns like relevant, required, constraint etc can be added using the Property Dialogue Box. An example is shown above in Fig4. After the form is built, one can export the form to XLS Form or XML Form using the File menu as shown below in Fig5.

Fig. 05 ODK Build - File Menu

Although syntax dependent, I found the XLS Form method to be more robust in terms of building the form and rectifying the errors. The XML form obtained through ODK Build is not necessarily error free. If you are using the XML Form from ODK Build without going through the Form Validation step described below, there is a possibility that the form will have errors in the ODK Collect App or while collecting the data. Hence, it is advisable to download the XLS Form from the ODK Build and follow the steps below to validate the form and then input it to the ODK Collect Mobile application.

Form Validation:

After the completion of the form, go to ODK Online and choose your .xlsx format form for submission. If you have built the form in google sheets, you must download it in .xlsx format and run it through the validation. Any errors in the form are displayed here after clicking on submit. Rectify the errors and submit again until ‘Preview in Browser’ and ‘Download XForm’ options appear as shown in Fig6. A point to note here is that the form is debugged line by line in ODK Online. Hence, each time an error pops up it must be rectified and submitted again. All the errors are not shown at once so it would be a good idea to make the .xlsx format as default for the form as mentioned earlier. Check the preview and validate the form. Then, click on ‘Download XForm’ and the XML file of the form will be downloaded. 

Fig. 06 Form validation

Connecting to Google Drive:

Once the form is ready to be used for data collection, we need to connect it to the ODK Collect Mobile App. For that, go to your google drive and create a folder for ODK forms. Upload the previously downloaded xml form in it. Then create an empty google sheet in which the responses of the survey will be collected. 

ODK Collect Mobile App set up:

Download and install ODK Collect from Google Play Store. Please note that, ODK Collect App is only Android compatible. 

Fig. 07 Server settings

  1. As soon as you open the app after installation, the server settings must be configured as shown in Fig7 after selecting ‘Manually enter project details’.

  2. Type - must be selected as Google Drive, Google Sheets if it's not set by default.

  3. Google account - Enter the google account of the google drive which will be used to collect and store survey data.

  4. Fallback submission URL - share the URL of the empty google sheet which was created in the google drive in the above step. This step ensures that all the collected data is sent to the mentioned submission URL.

  5. If using ODK Build, then one can set the submission URL in the Edit menu → Form Properties as shown in Fig8. 

  6. One can also use the QR code method to configure the settings 8

Fig. 08 Form Properties - ODK Build

Survey from Mobile:

Once the settings are configured, it is time to get the form and collect data. For that, 

Fig. 09 Main menu

Fig. 10 Select xml.

Fig. 11 Fill Blank Form

  1. Select ‘Get Blank Form’ from the Main menu in Fig9.

  2. Under that select the ODK folder created in the previous step - Select the xml form and click on ‘Download selected’.

  3. When you receive the ‘download successful’ message, go back to the main menu in the form and now select ‘Fill Blank Form’ from the Main Menu. Once the form is filled, go to ‘Send Finalised Form’ in the main menu and click on Send, alternatively you can also automate the sending of forms under - Account - Project Settings - Form management - Form submission Options.

  4. Go to your empty google sheet in the ODK folder and check how the responses have been recorded. Take note of the column names - these are the names entered in the ‘name’ column in your ‘survey’ worksheet. Tweak the survey form (column names, relevant questions etc) in accordance with the response sheet so that post survey analysis is free of confusion. Exploit the ‘calculation’, ‘appearance’, ‘hint’ and other columns which will make the survey precise.

Other essentials on ODK:

The official use cases and examples of all widgets are provided here. If you want to design a basic form using drag and drop functionality - ODK build is the way to go. However, this is not as customizable as the XLSForm method. Read more on ODK Central for setting up ODK on a larger scale.

 

Conservation Cartography with Mongabay-India (Part III)

In 2021, we began collaborating with Mongabay-India to enhance their stories with our spatial analysis expertise. Our joint goal is to use diverse storytelling formats to expand conservation awareness and reach a wider audience. This blogpost documents the third set of articles we’ve worked on together between March 2022 - May 2022

A decade on, India’s first solar park has many promises left to fulfil

The 5,384 acre Gujarat Solar Park in Patan.

The Gujarat Solar Park(GSP), India’s first utility-scale solar park, sets an example of how renewable energy development, if not done right, can lead to more environmental harm than good. The article discusses the socio-economic impacts of GSP, 10 years after its establishment.

Other solar parks in Gujarat’s Kutch district.

Citizen science data recorded at Mahatma Hill, Pune, on April 2, 2022.

Pune’s Mahatma hill, an offshoot of the Western Ghats, is home to around 129 species of resident and migratory birds, though recent years have seen a decline in their numbers. In the past two years, increased anthropogenic interference in the hill’s ecosystem are affecting the avian habitats here. 

We worked on a map collating information on bird diversity present in Mahatma Hill using iNaturalist data.

Expansion of windmills in Kachchh impact unique thorn forest and wildlife

OpenStreetMap data indicating 2053 windmills within Kachcch district in Gujarat.

Sangnara village in Kachchh depends mainly on agriculture and pastoralism for livelihood. The article brings forward the villagers’ interest that windmill projects planned within the village boundary be scrapped owing to the impact of windmills on Kachchh’s sensitive landscape- on local ecology, water sources and interpersonal relations.

We worked on the locator map for Sangnara and the location of other windmills in Kachchh.

The wind farm paradox in southern Tamil Nadu & Conserving biodiversity as wind farms expand in Tamil Nadu

Windmills in Tirunelveli district and near Muppandal in Kanyakumari district in southern Tamil Nadu.

Tamil Nadu is home to about 25 percent of India’s total wind energy capacity and remains an important part of India’s renewable energy target. The articles discuss the effect of the rising number of wind farms on biodiversity, and the social impacts of the same.

Wrong trees in wrong places wastes tree plantation budget, finds study

Tree planting is widely promoted as a natural solution to restore forests and absorb carbon from the atmosphere. India places heavy emphasis and budgets on tree planting in its global climate change goals. This article reports on the trend of planting the wrong trees in the wrong places, leading to financial losses.

(Note: This is the third blog in the series on our collaboration with Mongabay-India. Read the first blog here, and the second, here)

How-To: Carry UAVs through Indian airport security in 2022

TL:DR - Pack the UAVs in check-in luggage. Pack the controllers + batteries in cabin baggage. Empty out the contents of the cabin baggage into the tray while passing through security.

 

Note: This is a brief post describing our learnings regarding the transport of UAVs through Indian airport security in 2022. It is written in the hope that it helps other Indian UAV users navigate airport security without delay. I don’t think this will apply to non-Indian UAV operators, especially if they don’t present as Indian.



At Technology for Wildlife Foundation, one of our core operations is the use of robots (both Unmanned Aerial Vehicles and Unmanned Underwater vehicles) for conservation data acquisition purposes. For some projects, our partners send us data they’ve collected using their own devices. However, for others, it is imperative that we be on site with our equipment. This occasionally necessitates the transport of our robots across the country. For sites close to our base in Goa, India, we travel by either road or by rail. For these modes of transport, our primary concern is to package the equipment securely to avoid damage during transit. While traveling by air, however, we need to put much more thought into transporting our equipment.

 

As of November 2021, UAV users in India have a clear set of guidelines to follow, in the form of the Drone Rules 2021. As UAVs have become more mainstream, the security establishment is also formalising and mainstreaming processes around UAVs. To transport drones within the country, the Central Industrial Security Force (CISF), who manage security at most of India’s airports, now seem to have guidelines on how to process UAVs at domestic airport security checkpoints.  

 

Based on whether baggage is being carried in the cargo hold or in the passenger cabin, there are two categories of baggage on flights: cabin, which accompanies the passenger, and check-in, which goes into the hold. In brief, drone batteries and controllers (which contain fixed batteries) must be carried in the passenger cabin, while the drones themselves (without any batteries) must be carried in the cargo hold. If the controller batteries are removable, the controllers can also go into check-in baggage, which may be required depending on the size and weight of the controller. UAVs with fixed batteries cannot be carried on domestic airlines.

 

Pack the drones carefully, in a locked piece of luggage, as they will be out of sight passing through the luggage handling process, which can be rough on fragile items. The check-in luggage is deposited at the counter. In the past, with other robotic devices, we have informed the check-in staff that there are complex devices within the luggage. I personally have been called to check-in luggage security to verify what exactly the device is. Informing the check-in staff that the luggage contains UAVs without batteries is not required by regulations. While it may be helpful, it may also invite additional unnecessary scrutiny and is not something that we have felt the need to do regularly.

 

When going through security with our cabin baggage, we place every single piece of electronic equipment into the security tray that passes through the conveyor belt. When security staff have enquired as to the purpose of the devices, a straightforward answer of either “batteries”, or “drone batteries and controllers, but the drones have been checked-in”, has sufficed so far. We also carry paperwork that describes how the drones are to be used and have been used in the past; for us specifically, these consist of permission letters from the Forest Department.  

 

At some point in late 2021 or early 2022, posters depicting what cannot be carried as cabin baggage have been expanded to include drones as an additional item at the bottom of the poster. We’ll update this post with a photo of the poster the next time we have the opportunity. In the meanwhile, do let us know about your own experiences transporting UAVs by air in the comment section.