Web-based Data Collection and Assessment in Student Affairs*

Perhaps the most exciting development in recent years in the assessment field has been the advent of Internet-based surveying. While there are many ways of collecting data electronically, perhaps the most user friendly is when prospective respondents are contacted by email, informed about the purposes and parameters of the study, and then asked to link to a website to respond to the survey. Respondents can click on appropriate responses for each item, and then return the survey electronically. Depending upon the sophistication of the data collection software, individual responses can be assembled into a database and analyzed, providing the investigator with information ready to be further analyzed and interpreted.

We believe this method of collecting and analyzing data has the potential to revolutionize the way in which we conduct assessments in student affairs and higher education, but not without some considerable caveats.

Advantages of Web-based Data Collection

1. Both quantitative and qualitative information can be gathered. Respondents can be asked to answer questions that are worded in any standard way, regardless of the response scales. Similarly, open-ended questions that allow the respondent to form and submit their own answers can also be incorporated into the instrument design.
      
2. Data can be collected in a user-friendly manner. Instead of the drudgery of completing and returning a mailed questionnaire, web-based instruments demand far less respondent time and effort, and unlike telephone surveys, can be completed at a time and place convenient to the respondent.
     
3. Return rate may be greater and more timely. There is some evidence (Marine, 2000) that when multiple modes of data collection are compared, Web-based survey responses rates are consistently higher than mailed or telephone surveys.
     
4. Respondent time to complete the survey is reduced. "Point and click" takes less time than responding verbally to a telephone survey or in writing on a mailed questionnaire, if the amount of time to complete the survey does not exceed seven or eight minutes.
     
5. Data collection time is reduced. Mailed surveys sometimes take months to complete the data collection process; web-based surveys can take as little as three weeks.
    
6. Anonymity can be maintained. One of the major objections to web-based surveys is that many people believe that anonymity of responses cannot be guaranteed. On the contrary, when placing an instrument on the web, results can be collected without any identifying information attached.
    
7. Respondent pool can be increased. With a web-based survey, researchers can cross the boundaries of time and distance to reach target audiences, particularly if the audience is widely dispersed geographically.
    
8. Data can be more efficiently managed. Unlike other forms of data collection, web-based data can be recorded and analyzed electronically and automatically, saving time and money, and eliminating data recording errors.
    
9. Duplicate responses can be identified. Another concern of critics of web-based data collection is the fear that a rogue computer user might submit multiple responses to the same survey, thus compromising the integrity of the study. On the contrary, software programs can process responses in ways that identify if the same person submits more than one survey.
    
10. Instruments can be piloted more easily. Piloting an instrument can be done through e-mail or web-based approaches. Respondents are asked to fill out the survey and make comments on its validity and clarity, and do so within a very short period of time.
     
11. Instruments can be retooled to accommodate changes. Compared to other data collection methods, surveys can be easily and quickly modified, saving time, money, and inconvenience.
     
12. Certain costs may be lower. There may be some significant cost savings using web-based approaches, compared to other data collection methods, depending upon the availability of computer equipment, software, and expertise.
     
13. Greater control over responses can be more easily achieved. Web-based data collection has greater potential for solving the problem of how to deal with questions that are answered improperly. HTML coding allows for investigators to ask for and receive specific responses without deviation. Further, a survey may be designed that does not allow respondents to skip questions by not allowing those respondents to continue until a field is filled in.

The bottom line is that, overall, web-based data collection offers a lot of advantages compared to other data collection methods, but there are some downsides to be considered.

Disadvantages of Web-Based Data Collection

1. Not all respondents have access to the web. While most institutions provide computer access to the web, there may be some that do not. There may also be some institutions where some have access and others do not.
    
2. No all respondents have the necessary computer literacy skills. While most of today's students are "computer literate" some are not, lacking basic keyboarding or other skills necessary to respond to web-based instruments.
      
3. Web-based data collection requires different time and expertise. Some steps are added to the data collection process that are not required for other data collection procedures, such as finding web space, authoring web pages, and linking the responses to a data base or text file, all of which required computer time and expertise.
    
4. Distrust of anonymity assurances may be a problem. While as pointed out earlier, web-based surveys can be structured to ensure anonymity of responses, respondents may not believe it, and may not respond. It must be noted, however, that this same problem exists with other data collection methods.
     
5. Rate of return may be inconsistent. While as noted above, return rates are generally higher than mailed or telephone surveys, it has been our experience that depending on the survey and the procedures used to collect data, return rates may vary from as little as 15% to as high as 80%.
     
6. Certain costs may be more. Web-bases surveys may incur costs not normally associated with more traditional data collection methods. They include computer processing time, wages for the HTML author, software and computer application costs, hardware purchases, cost for web space, and electronic security and storage fees. As point out earlier, these costs will vary depending upon the type of support and assistance provided by a particular institution.
     
7. Hardware, software, and server malfunctions may occur. The realities of web-based data collection procedures are that hard drives crash, software glitches and failures occur, floppy discs can fail, and other problems may happen. Human errors in programming, storing data, and lack of expertise can also cause problems.

So what's the definitive answer on web-based data collection? It spite of its many disadvantages, we believe it is the wave of the future, and that ten years from now, mailed and telephone surveys will be as archaic as buggy whips and eight track tape players. What do you think?

Reference

     Marine, R. J. (2000). Evolution of survey modes during the 1990s. University Park, PA: Unpublished manuscript.

*This article is based primarily on "Web-Based Data Collection," by Thomas I. Wortman and M. Lee Upcraft, a chapter in a book entitled Assessment Practice in Student Affairs: An Applications Manual, authored and edited by John H. Schuh, M. Lee Upcraft and Associates. San Francisco: Jossey-Bass (In Press).