COMPUTER ASSISTED PERSONAL INTERVIEWING (CAPI)

THE BERMUDA EXPERIENCE

A PILOT STUDY FOR THE 2004 HOUSEHOLD EXPENDITURE SURVEY

A Paper Presented to:

THE 29^TH MEETING OF THE

STANDING COMMITTEE OF CARIBBEAN STATISTICIANS

The Fairmont Hamilton Princess Hotel

November 24^th 2004

By:

Desmond L. Trott - Census & Survey Officer

Andrew M. Simpson - Trainee Statistician

Bermuda Government

Department of Statistics

COMPUTER ASSISTED PERSONAL INTERVIEWING

The Bermuda Experience

1          INTRODUCTION

1.1       Purpose of Report

The purpose of this report is to outline the experiences of the Bermuda Government Department of Statistics in using handheld computers as an optional method for survey data collection. Computer Assisted Personal Interviewing (CAPI) involves the use of a portable computer to collect, store and transmit data related to a personal survey interview. The Department tested this interviewing method during the conduct of the decennial Household Expenditure Survey (HES) from May to August 2004.

This document provides an overview of each phase of the HES-CAPI pilot study from preparation to performance. It is hoped the information shared in this paper provides key insight for those national statistical offices that may be considering using CAPI as an alternative method for data collection.

1.2              The Main Phases of the Pilot Study

a)      Preparation

·         software and hardware selection

·         training in use of survey software

b)      Planning

·         questionnaire digitization

·         interviewer training

c)      Processes

·         data collection and interviewer feedback

·         data processing

·         data editing

d)      Performance

·         lessons learnt

·         project cost

·         future applications

1.3       Why CAPI?

The Bermuda Department of Statistics has over 30 years experience in collecting data and is responsible for the majority of government statistical activities. Though relatively small, the department faces challenges similar to those of larger statistical agencies. These include: scarce storage space, rising printing costs and an increasing need for timely and reliable data. The department is constantly seeking new methods and procedures to collect and disseminate information more efficiently. In light of these efforts to improve, an exploration commenced to investigate new and more cost effective means of conducting surveys.

Given a worldwide increase in the use of information technology combined with an 'E-Government' initiative newly launched by the Bermuda Government, the logical place to start was in using computers. Research into latest technological advancements revealed that countries, such as the United States of America, Australia, Canada, New Zealand and Oman had experimented with CAPI in the administering of surveys. Their experiences reported three common benefits in survey operations:

·         better data quality,

·         improved timeliness, and;

·         cost effectiveness.

Historically, Pencil and Paper Interviewing (PAPI) has been an effective method of collecting data in Bermuda. CAPI, however, had the capacity to increase the quality and efficiency of data collection and processing. The ultimate goal was to integrate CAPI into the department's data collection program so the benefits of the technology could be garnered. A staged developmental approach was seen as the best method for the incorporation of CAPI.  Therefore, the immediate objectives of this pilot study were three fold:

·         to evaluate the use of technology in survey field work;

·         to determine whether the standard of data quality increases with use of technology; and,

·         to assess whether the time taken to conduct an interview and process the data is reduced using technology;

2                    SOFTWARE AND HARDWARE SELECTION

2.1              Software Selection

An initial concern was the size and complexity of the Household Expenditure Survey questionnaire and whether a handheld device and/or software could cope with a questionnaire of that magnitude. The survey data collection instrument contained 17 sections, which when combined, totalled 450 individual questions. The ability to include skip instructions, define values and incorporate rules were important criteria. The most important requirement however, was to obtain easy-to-use software for use by the questionnaire designers and ultimately the field interviewers.

Large statistical agencies often have the expertise to create specialized software. Due to the lack of this skill in-house, the Department of Statistics researched external sources and selected an appropriate software vendor to facilitate the initiative. Search results provided a number of software companies experienced in computer assisted surveying. Many of these businesses however, specialized in specific areas of surveying such as land surveying and building inspection. Hence, a process of elimination was implemented to determine a potential supplier whose software met the stated criteria.

2.2       Snap Survey Software

After much research and negotiation the Snap Survey Software, developed by Mercator Corporation, was selected for use in the pilot study.  This organization, which specialized in market research, had developed a fairly flexible software package that could be used for a variety of survey applications.

Snap offered a fully integrated survey software which had the ability to create paper surveys with scanning capabilities, e-mail and internet surveys, computer assisted telephone interviews (CATI) and CAPI.  The software also included a facility to create frequency tables, cross tabulations, charts and a full range of descriptive statistics. A strong selling point for Snap was its compatibility with other survey software and its ability to save data in a variety of formats including Triple S, ASCII and SAV.

A sample of the paper questionnaire was sent overseas to Mercator Corporation for review and to help determine whether the Snap software was suitable for the HES application. Although the software had not been used previously to design household expenditure survey questionnaires, it was ascertained that the HES survey questions required the same computer programming techniques as with questions for any other survey. The software had the ability to record text, numeric codes and dollar values. Also, the skip instructions could be followed automatically using a 'routing function'. Snap allowed for up to 65,000 questions in a particular survey. The 450 HES items, in terms of software question capability, did not present a problem.

A product specialist from Mercator Corporation scheduled an on-line software demonstration for the department to give an overview of the many features and ease of using the software. In addition, the company provided a free evaluation copy of the software for review by the department. The offer purchase package for the Snap product included 12 months of service and maintenance, unlimited technical support and free upgrades.  It was agreed that the package was attractive but more importantly the software could be used for the conduct of other surveys in the Department's work programme.

2.3              Hardware Selection

Through research it was discovered that other countries were using a variety of portable computers for CAPI applications. The type of devices used included laptops or notebook PCs, handheld PDAs, and Palmtop PCs. The laptop cost averaged about $2000 US, compared to a range of $200 - $400 for the handheld and palm type computers. In addition, purpose built pen-type survey computers had been developed and were priced in the upper $2000 range. In the interest of higher cost benefit, the less expensive handheld computers were more cost effective for the pilot study.

The decision to select a software vendor first made hardware selection easier. A Snap sales associate advised the department that the software was not currently compatible with the Handspring or Palm supported computers. Snap was only compatible with handheld computers on a Windows CE platform with the following requirements:

·                Processor: MIPS3, SH3, APM

·                Windows CE: 2.11 or higher

·                Memory: 16 MB RAM or higher

·                ActiveSync: 3.0 or higher

·                Outlook 97 or higher

The Sales Associate provided a list of recommended handheld computers along with previous interviewer experiences with some of the machines. This proved very helpful in selecting the hardware.

After reviewing the options, the Dell Axim X3 PDA was selected for the project.  The Axim featured a colour screen, a pointing tool (stylus) for data entry and was equipped with all the software requirements. Also available were detachable fold-up keyboards onto which the computer could be mounted to provide an additional option to enter data. These were also acquired.

The licensing agreement offered by Snap covered only five PDAs. Any quantity greater than five incurred additional licensing costs. However, five PDA's were sufficient for the pilot study which was a subset of the main data collection for the Household Expenditure Survey.

3                    TRAINING IN USE OF SNAP SOFTWARE

3.1              In-house Training

A two-day training workshop in the use of the Snap Software was offered to the department. In February 2004, two certified instructors from Mercator Corporation arrived to conduct the training. In preparation for the training session, the instructors were given a copy of the HES paper questionnaire which they used to structure the workshop training and prepare relevant content to the survey. The instructors developed consolidated 'fast-track' course booklets that supplemented the detailed operational manual for the software.  These booklets contained instructions for the basic operation of the software as well as sample questionnaires that were used throughout the course of training.

A total of four staff members participated in the Snap two-day training workshop. Their background experience was broad and covered use of SPSS (Statistical Package for the Social Sciences), CSPro (Census and Survey Processing), IMPS (Integrated Microcomputer Processing), and questionnaire design.

The content of the training session covered the following areas:-

·       an overview of the software toolbars

·       questionnaire design concepts

·       question styles

·       questionnaire editing

·       uploading questionnaires to the handheld

·       data retrieval

·       transfer of data to and from other programs

·       tabulation of data

·       creation of charts

The instructors were competent in the delivery of the training material. The software was generally user friendly and in many areas comparable to software previously used by the staff.

4          QUESTIONNAIRE DESIGN

4.1              Digitization of the Printed Questionnaire

Two officers were assigned to digitize the printed questionnaire using the handheld computer. The digitization process was fairly easy as the printed questionnaire was the model to follow.  Each section, heading and question on the questionnaire was transcribed onto the screen in the same format. The Snap software automatically created a record layout (data dictionary) as each question was added. Snap also allowed for text formatting such as font, font size and bold-facing. It was effective in creating and saving question styles that could be applied to other questions. As a result, there was no need to re-format each question individually.

Although the transcribing process was easy, it was slow and tedious primarily due to the length of the questionnaire. Consequently, an alternative method was sought to expedite the process. Since Snap software was compatible with the SPSS software, a record layout that included all survey questions and variables was developed using the SPSS software and then imported into Snap.

Using the SPSS record layout, the Snap software automatically produced basic question screens ready for formatting. As such, any saved question styles was easily applied at this point. This method proved more efficient as it was faster to input the data into SPSS and then import into Snap. One task entailed creating the record layout in SPSS. Another task involved importing the data into snap and performing the necessary formatting and page breaks.

4.2              Skip Instructions

After the questions and text were formatted, the required ranges and skip instructions were incorporated into the questionnaire. Range limitations prevented interviewers from entering out-of-range responses into the questionnaire. For example, an individual's age recorded as 178 years would be an out-of-range response.  Skip instructions were important as they navigated the interviewer through the interview and prevented redundant questions from being asked. For example, allowing questions on fertility to be asked only of female respondents.

In using a paper questionnaire, there is always the possibility that the interviewer will ignore or not correctly follow the skip patterns. However, using the CAPI method prevents this. Once a response is keyed in the PDA, the interviewer is automatically transferred to the next applicable question or section of the questionnaire.  This built in feature minimizes interviewer error significantly. The Snap software offers a facility to program skip instructions very easily using common language without the need for complicated syntax.  

4.3              Challenges Faced in Digitizing the Questionnaire

In attempting to digitize the printed survey questionnaire, there were specific aspects of the interviewing process that presented challenges for the potential interviewer. These were:

·       the use of a roster-style grid to collect information, and

·       the inability to conduct partial interviews using the handheld computer

The questionnaire was designed to interview more than one person per household. The paper questionnaire used a roster-style grid to collect information from each individual.  It was possible to create grid-style questions on the handheld, but it did entail some concerns. 

The size of the screen had to be considered when using a grid. Though it was possible for the screen to scroll across as data was entered, this process made the data entry more difficult for the interviewer.

Administering the survey questionnaire using a grid on the PDA meant that all eligible household members needed to be present during the interviewer's first visit.  Once data for a household had been submitted to the computer it was impossible to retrieve and edit the data. As a result, the interviewer could not submit new data for any member of the household that was absent unless the data was held in check until the next visit. But this meant that the interviewer could not commence interviewing another household, until the first was completed. This constraint was the root of the second challenge.

Due to the length of the HES questionnaire, it was not mandatory for the interviewers to fully complete it in one visit. Therefore, to make efficient use of the time in the field the interviewers were encouraged to visit as many households as possible. This posed a problem using the CAPI method which was quite contrary to the PAPI method where the interviewer had the option of closing the questionnaire booklet and moving to the next household using a different questionnaire. 

4.4              Solutions to Digitization Challenges

The following viable solutions were developed to deal with the ensuing challenges.  First, the seventeen (17) sections in the questionnaire were created as individual files. This allowed the interviewer to open and complete any section of the questionnaire for distinct households on different field visits. For example, the interviewer was able to complete Sections 1, 2, and 3 on the first household visit then return on another visit to complete Sections 4, 5, and 6. The resulting restriction to this solution was that any section that the interviewer had started during the interview had to be fully completed before leaving the household. The interviewer was then able to interview another household while in the field.

The individual sections of the questionnaire were designed to permit individuals who were not present during the initial interview visit to be interviewed during later household visits. To facilitate this, fields for the Sample I.D. and Person Number were inserted at the beginning of each section. The interviewer was required to enter a nine-digit number, unique to each household, at the start of each section. The household composition section, which was completed on paper, assigned Person Numbers to each household member. Using a drop down box, the interviewer was able to select the number of the household member being interviewed. The combination of these two fields created a unique identifier for each individual. The key to this solution was the development of a unique identifier, which was equally important when retrieving the data.

The proposed solution underwent extensive in-house testing. In doing this, it was observed that if implemented, the interviewers could not track and review any previous completed sections once the data had been submitted. This prompted the development of a survey completion form which was designed such that the interviewer could check response boxes signifying completion of each section.

The form was designed as a grid with the questionnaire sections outlined across the columns and the individual person numbers listed by row. The interviewer would place a check in the appropriate grid box section by section as the household member was being interviewed. The interviewer was then able to monitor their completion progress by both questionnaire section and individual person number. The form proved to be very effective. Hence, it was also used by those interviewers using the PAPI method.

4.5       In-House Testing

After all sections of the questionnaire had been completely digitized, numerous tests were conducted using different mock situations. The objective was to test the skip patterns and the flow of the questions in each section. Once the errors were corrected, a final test of the questions was applied to ensure that the program could run smoothly from Sections 1 through 17. The entire digitization of the printed questionnaire which includes the testing of the questionnaire flow took roughly one month to complete.

5          TRAINING OF INTERVIEWERS

5.1              General Training       

A five-day training session was conducted for all temporary field staff. Roughly 100 persons were trained in understanding the purpose of the expenditure survey and how to effectively administer the survey questionnaire. The training also included interviewing techniques and the completion of the survey administration forms.  A test was administered at the end of the training session. Those persons obtaining a 75% pass or higher on the test were selected as potential field interviewers.

5.2              Interviewer Selection (CAPI)

The five (5) interviewers that were needed to administer the survey questionnaire using the handheld computers were then picked from the pool of successful persons. However, in addition to being computer literate, they had to meet at least one of the following additional criteria.

                                                                                 i.            Previous interviewing experience

                                                                               ii.            Experience in the use of PDAs

Of the five interviewers selected only three had previous experience as a survey interviewer. The remaining two had a high level of computer literacy. This mix of experience however facilitated the possibility of assessing whether previous survey or PDA experience impacted on the level of difficulty in administering the questionnaire using the hand held. 

5.3              CAPI Training

A one-night training session was scheduled to instruct the five interviewers how to administer the survey questionnaire using the PDA. The interviewers had already participated in the PAPI training.  Therefore, the theoretical foundation about the conduct of the survey, the survey instrument and the administrative documents had been laid. As such, the CAPI training focused primarily on working through the digital questionnaire using the PDA in the most effective and efficient way.

After the one-night training, interviewers were instructed to perform mock interviews using the PDA's with family members and friends. This exercise acted as a pre-test to familiarize the interviewers with the CAPI method, in particular the software, formats, layout and general flow of administering the digitized questionnaire. Having done this they could conduct interviews with greater confidence and efficiency in the field. Interviewers were encouraged to contact the office during the pre-test to ask questions, seek clarification on procedures and report any problems that may arise.

6          FIELD INTERVIEWING

6.1       Distribution of Survey Supplies & Assignments

The Household Expenditure Survey was launched on May 7^th, 2004. Each CAPI interviewer was equipped with a tote bag containing the handheld computer, an attachable keyboard, required stationary and a charger cord. The interviewers were also given a listing of household addresses and survey packets, each containing one printed questionnaire, two weekly diaries and survey administration forms for completion. Each interviewer was expected to complete 10 households. Their assignment was to use the PAPI method to conduct two household interviews and the CAPI method for the remaining eight households. This process enabled the interviewer to compare the two data collection methods and provide the office with the appropriate feedback. A total of 50 households was deemed an optimal sample size for the conduct of the CAPI method pilot study.

6.2              CAPI Survey Process

When conducting CAPI surveys, the interviewers were responsible for completing only Section 1 - Household Composition, using the printed questionnaire. The remaining sections of the questionnaire were to be administered using the handheld computer.  It was thought that the roster design of Section 1 in the questionnaire would present difficulty in collecting the data using the PDA:

The advantages of using the printed questionnaire to collect the information in Section are as follows:

                                                         i.            person records for each household were kept together;

                                                       ii.            information collected could be used as a cross check in the data processing phase;

                                                      iii.            interviewers had a written record of the household members who had to be interviewed, and,

                                                     iv.            the printed questionnaire also acted as a supplement if the interviewer ran into difficulties with the PDA or needed to record notes.

The CAPI interviewers followed the same procedures as the PAPI interviewers for the collection of expenditure data using the weekly diaries. One diary was left with the households during the first week to record their expenditures followed by a mid-week call-up as a reminder. The second diary was left the following week.

6.3       Interviewer Contact

While in the field, the five CAPI interviewers were instructed to report directly to office staff who had designed the CAPI questionnaire. A detailed account of the CAPI method and experiences in the field were to be monitored and recorded. Unfortunately, contact with the CAPI interviewers was extremely challenging throughout the entire survey period. This was similarly experienced by supervisors assigned to interviewers using the PAPI method.

At the end of the survey period a total of twenty contacts had been made but with only four of the CAPI interviewers. One interviewer was absolutely impossible to contact amid numerous telephone calls and left messages. It was later discovered that the interviewer had left the Island unexpectedly without informing staff at the office.

6.4       Interviewer Feedback

(a)           Interviewers were distraught with the task of entering the nine-digit household identification number for each section once completed. They reported the process as being very tedious. This concern was anticipated and did not come as a total surprise to office staff. Although the task was tedious it was not difficult to carry out.

In response to this issue, it was proposed that the nine-digit field be replaced with two simple fields - a three-digit field for the census district number and a two-digit field for the household number. The household number field would be programmed as a website style drop-down box with the numbers 1 - 12 as options representing the total number of assigned households on the address listing. The combination of these fields would create a unique identifier for each household. This method was thought to be less tedious than transcribing a nine-digit number for each questionnaire section.  However, it was later decided not to implement these changes while the interviewers were in the field.   This would create problems in data processing and would result in delays for the interviewers. Firstly, it would have resulted in different I.D. field formats, making data processing difficult.  Furthermore, the new I.D. fields would have to be changed, re-installed onto the PDA and then tested.    Hence, the idea was documented for possible implementation during the conduct of future surveys.

(b)           Interviewers found it difficult interviewing households with multiple respondents, that is, those households with five or more persons 16 years or older who had the ability to spend money on their own behalf. Although the PDA was able to handle the magnitude of the household size, the process was seen as challenging and time consuming to the CAPI interviewers. The PAPI method allowed an interviewer to survey more than one person at a time by simply moving across the columns for each item of expenditure. Although during the training, interviewers were instructed to interview one person at a time and not simultaneously, this process seemed very practical in the field for large households particularly when all members were present. However, this strategy was impossible to apply using the handheld computer. The program had been designed to receive one person's data at a time.

The 2000 Census indicated that the average household size was 2.47 persons. Therefore, to alleviate interview burden while in the field, the interviewers were given the option of surveying larger households, i.e., those with four (4) or more respondents, using the PAPI method.  Smaller households, those with three or less respondents would be surveyed using the CAPI method. Interviewers were already equipped with the printed questionnaires since they were required to interview two households using the PAPI method. It was noted that further development was needed in this area to ensure that this process is less taxing during the conduct of future surveys.

(c)           During the training, interviewers were instructed to add expenditures made by persons under 16 years to that of the Household Reference Person. The 2000 Census of Population & Housing revealed that roughly 370 persons under 16 years were employed during the reference period. As such, expenditures by these household members were deemed significant enough to capture during the HES. Using the PAPI method, these expenditures were written in the spaces provided on the questionnaire. Summations however, were performed after the interviewer had left the household. In administering the CAPI method, the summation of expenditures for those persons less than 16 years had to be made immediately so that the expenditure for the household reference person was keyed into the handheld computer accordingly. The interviewers found this additional step time consuming as a calculator was needed in some instances to sum the expenditures.

Office staff viewed this matter as unique to a household expenditure survey. Furthermore, this issue was distinct only to households with persons under 16 years of age and applicable to a limited number of sections in the questionnaire. Consequently, no solution was offered and the problem shelved for future research if needed.

(d)           After six weeks in the field, two of the CAPI interviewers contacted the office to report that they were experiencing problems finding the software program on the PDA and that interviews could not be conducted. After office staff examined the PDA's and investigated its use by the interviewers, it was discovered that both interviewers had left the computers for a period of time without re-charging, therefore causing the batteries to die. The problem was then solved by charging the PDA's and uploading the Snap software again. Unfortunately, the survey data that had been collected and stored in the PDA's was lost.

This immediately became a concern to office staff. The PDA was already equipped with an internal memory card for backing up data. Therefore, the remaining interviewers were reminded to charge the computers regularly and back up the survey data after each interview.  There were no reports of power loss while conducting interviews. However, it was noted that the PDA battery would discharge automatically if the PDA was not used for a long period of time, even if the unit was turned off. This experience prompted the need for more frequent reporting to the office by the CAPI interviewers. It was also thought that for future use, the transmission of survey data may be feasible through email if secure enough.

(e)           On a brighter note, one interviewer reported that the calendar program, which came standard with the PDA computer, was effective in recording appointments for interviews. The program had alarms to remind the interviewer of up-coming survey appointments. This was an excellent feature that assisted in keeping the interviewer organized.

After surpassing the initial challenges with using the CAPI method, the interviewers reported that as they conducting more surveys using the PDA the process became much easier. However, they reported that CAPI became tedious when the household size exceeded three persons.  One interviewer informed office staff that after getting used to the CAPI method it was preferred over the PAPI method.  Based on an analysis of survey returns, there were not substantial differences in completion times between the data collection methods.  A single-person household took up to an hour to interview, while a household of three or more spenders took at least two to three hours.

6.5  Interviewer Performance

In total, 31 households were visited and completely surveyed by the CAPI interviewers, of which 19 were completed using the CAPI method. However, of the 19 CAPI completions, survey data for 10 households was lost due to the loss of power for the two PDA's. Consequently, data was obtained from only 9 CAPI administered surveys. Although this led to some disappointment by the PDA office team, the experience was invaluable and provided greater insight on the importance of implementing strict monitoring measures during the conduct of a pilot study.

In summary, of the 50 households that were assigned to the CAPI interviewers, the following summarises the field work performance:-

                                                         i.            Three (3) interviewers remained in the field until the conclusion of the survey period. A total of 28 households were visited.

                                                       ii.            One (1) interviewer completed three (3) households and later informed the office of their inability to continue the work.

                                                      iii.            One interviewer had to leave the Island unexpectedly for the majority of the survey period and reported no completions.

7          DATA PROCESSING

7.1              Data Retrieval

Data obtained from each completed questionnaire were saved on the handheld computer in the form of an e-mail message. These messages were retrieved from the PDA computer using a synchronisation process when the handheld was placed into a desktop cradle. The Windows operating system automatically performed the synchronisation.

During this process, data from each questionnaire was sent to the Inbox of Microsoft Outlook on the Desktop PC.  The Snap software, when prompted, scanned the Inbox for any replies from the current survey questionnaire. Messages containing replies were imported and converted into raw data, while messages not recognised were left as unread.

It was also possible for interviewers to submit their completed surveys via the web using e-mail without having to bring the PDA's into the office.  This option was not to used as it meant setting up user accounts for each interviewer which presented security issues.

The process of importing the household expenditure data from the PDA's to the Desktop PC occurred smoothly and it literally took three clicks of the mouse. This process was clearly an advantage to using CAPI method as manual data entry or scanning of survey data can be extremely time-consuming.

7.2       Appending the Data

As individuals from the same household were interviewed using different set-up files, it was necessary to append the individual person to obtain total household expenditure. In preparation for this process, the survey data was exported from the Snap software into SPSS. 

To append the data a 'break variable' had to be used. This 'break variable' is a unique combination of values which defines a group and generates one case in a new aggregated file. Using the Sample I.D. and Person Number as break variables, each individual person in a household was appended to the record of their respective household reference person. This process was performed for each of the 17 Sections in the questionnaire. The final result led to one record being produced for each household but comprising all household members.

7.3       Summation of Individual Expenditures

All the expenditure data collected from individual household members needed to be aggregated so that the analysis of the results is done at the household level. The PAPI method required the interviewer to manually sum the expenditures for each household member and record the amount in a total column. This however, was not necessary using the CAPI method as these calculations could be performed using the desktop computer. Using SPSS, the calculations were performed and the summations placed into new field variables representing total household expenditure. 

7.4       Data Editing/Coding

(a)        The nine completed survey questionnaires were edited for correctness and consistency. The editing process did not return many errors. There was less than 2% missing values. The minimum number of survey returns however may not be sufficient to completely assess the extent of error using the CAPI method. Hence this presents limitations to measuring whether data quality gains were obtained as a result of using CAPI.

(b)        All skip instructions were followed correctly and all data collected were within the prescribed ranges. As stated previously there were minimum errors observed. It must be noted, however, that this degree of quality was also seen in the PAPI surveys conducted by these interviewers.

(c)        There was one case of duplication in which two of the same individual numbers for a particular household were recorded. However, the interviewer had made a note of the possible error and provided the correct person number to be assigned to the household member. The possibility of this error stemmed from the decision to administer the questionnaire sections individually. Further research in this area should produce means of eliminating this occurrence of duplication.

(d)        The "Write-in" fields for occupation and industry were coded on the data file using the Bermuda Classification for Occupation (BCO) and the Bermuda Classification of All Economic Activity (BCEA).

8                    PROJECT COST

8.1              Project Cost

The following gives a breakdown of the costs associated with this pilot study. The expenses are separated by the cost of software, hardware, training and the remuneration of interviewers. All expenditures are given in US dollars and exclude any sales tax.

Software Costs

Hardware Costs

Software Training

Interviewer Remuneration

The interviewers were paid $150 for the one-week training session only if they attended for more than three days and $100 for each survey questionnaire completed in the field. This level of pay was necessary to attract temporary personnel to participate.

9                    CONCLUSION

9.1              Summary

This was a first time experience for testing the effectiveness of using Computer Assisted Personal Interviewing as a method for data collection.  The pilot study provided first hand knowledge about the advantages and disadvantages of the PDA technology, which will be valuable for consideration when conducting future surveys. 

It can be concluded that the CAPI method is a viable alternative for collecting survey data. However, caution must be exercised when deciding on a method of data capture for various surveys. Decisions should be made based on the length and complexity of the survey questionnaire and the type of questions asked.    

The complexity of the Household Expenditure Survey made for a challenging exercise, which in-house staff and field workers faced with innovation and determination. Upon reflection, it was quite evident that strict survey controls were needed for the fieldwork operations, such as weekly reporting to office staff, constant follow-up and increased observation of the CAPI interviewers in the field.

However, the following outlines some of the benefits that can be assumed by using the CAPI method:-

·       Less interviewer fatigue: Once interviewers become accustomed to CAPI, less physical and mental effort can be realized compared with paper questionnaires.  Skip instructions and ranges are built into the program and the PDA provides greater security for data.

·       Improved data quality.   The limited number of surveys collected during the pilot study made it impossible to verify the claim.  However, due to the nature of CAPI, substantial gains in data quality can still be expected.   The PDA can be programmed to make on-the-spot edit checks allowing errors to be detected earlier.

·       Reduction of manual editing: CAPI virtually eliminated the need for the manual editing phase. Most edits were performed using the PC.  

·       High-speed delivery of data: Large amounts of survey information can be converted into raw data in a fraction of the time it takes to scan data or carry out manual data entry.

·        Reduced Costs:  Although not experienced in the pilot study, savings can be realized as a result of reduction in printing cost for questionnaires and hiring of temporary personnel. 

9.2              Future Initiatives for Use of PDA's

·         Monthly survey of food prices for the Consumer Price Index;

·         Quarterly survey of statistics for Resident Purchases Overseas; and,

·         Use for data collection of sub-populations in the 2010 Census.

The Bermuda Department of Statistics is optimistic in using CAPI as an efficient method of data collection.  As experience with the technology grows, its benefit hopefully can be exploited over a wide range of projects.

Contact Information

For further information or answers to any questions that may arise from this paper, you may contact:

Desmond L. Trott

Census and Survey Officer

Bermuda Government

Department of Statistics

dltrott@gov.bm

or

Andrew M. Simpson

Trainee Statistician

Bermuda Government

Department of Statistics

amsimpson@gov.bm