Join our mailing list!
©2007
StudentAffairs.com
All Rights Reserved
|
Web-based Data Collection and Assessment in
Student Affairs*
Perhaps the most exciting development in recent years in
the assessment field has been the advent of Internet-based
surveying. While there are many ways of collecting data
electronically, perhaps the most user friendly is when
prospective respondents are contacted by email, informed
about the purposes and parameters of the study, and then
asked to link to a website to respond to the survey.
Respondents can click on appropriate responses for each
item, and then return the survey electronically. Depending
upon the sophistication of the data collection software,
individual responses can be assembled into a database and
analyzed, providing the investigator with information ready
to be further analyzed and interpreted.
We believe this method of collecting and analyzing data
has the potential to revolutionize the way in which we
conduct assessments in student affairs and higher education,
but not without some considerable caveats.
Advantages of Web-based Data Collection
- Both quantitative and qualitative information
can be gathered. Respondents can be asked to
answer questions that are worded in any standard way,
regardless of the response scales. Similarly, open-ended
questions that allow the respondent to form and submit
their own answers can also be incorporated into the
instrument design.
- Data can be collected in a user-friendly
manner. Instead of the drudgery of completing and
returning a mailed questionnaire, web-based instruments
demand far less respondent time and effort, and unlike
telephone surveys, can be completed at a time and place
convenient to the respondent.
- Return rate may be greater and more
timely. There is some evidence (Marine,
2000) that when multiple modes of data collection are
compared, Web-based survey responses rates are
consistently higher than mailed or telephone surveys.
- Respondent time to complete the survey is
reduced. "Point and click" takes less time than
responding verbally to a telephone survey or in writing
on a mailed questionnaire, if the amount of time to
complete the survey does not exceed seven or eight
minutes.
- Data collection time is reduced. Mailed
surveys sometimes take months to complete the data
collection process; web-based surveys can take as little
as three weeks.
- Anonymity can be maintained. One of the
major objections to web-based surveys is that many people
believe that anonymity of responses cannot be guaranteed.
On the contrary, when placing an instrument on the web,
results can be collected without any identifying
information attached.
- Respondent pool can be
increased. With a web-based survey, researchers
can cross the boundaries of time and distance to reach
target audiences, particularly if the audience is widely
dispersed geographically.
- Data can be more efficiently managed.
Unlike other forms of data collection, web-based
data can be recorded and analyzed electronically and
automatically, saving time and money, and eliminating
data recording errors.
- Duplicate responses can be identified.
Another concern of critics of web-based data collection
is the fear that a rogue computer user might submit
multiple responses to the same survey, thus compromising
the integrity of the study. On the contrary, software
programs can process responses in ways that identify if
the same person submits more than one survey.
- Instruments can be piloted more easily.
Piloting an instrument can be done through e-mail
or web-based approaches. Respondents are asked to fill
out the survey and make comments on its validity and
clarity, and do so within a very short period of
time.
- Instruments can be retooled to accommodate
changes. Compared to other data collection
methods, surveys can be easily and quickly modified,
saving time, money, and inconvenience.
- Certain costs may be lower. There
may be some significant cost savings using
web-based approaches, compared to other data collection
methods, depending upon the availability of computer
equipment, software, and expertise.
- Greater control over responses can be more
easily achieved. Web-based data collection has
greater potential for solving the problem of how to deal
with questions that are answered improperly. HTML coding
allows for investigators to ask for and receive specific
responses without deviation. Further, a survey may be
designed that does not allow respondents to skip
questions by not allowing those respondents to continue
until a field is filled in.
The bottom line is that, overall, web-based data
collection offers a lot of advantages compared to other data
collection methods, but there are some downsides to be
considered.
Disadvantages of Web-Based Data Collection
- Not all respondents have access to the web.
While most institutions provide computer access
to the web, there may be some that do not. There may also
be some institutions where some have access and others do
not.
- No all respondents have the necessary computer
literacy skills. While most of today's students
are "computer literate" some are not, lacking basic
keyboarding or other skills necessary to respond to
web-based instruments.
- Web-based data collection requires different
time and expertise. Some steps are added to the
data collection process that are not required for other
data collection procedures, such as finding web space,
authoring web pages, and linking the responses to a data
base or text file, all of which required computer time
and expertise.
- Distrust of anonymity assurances may be a
problem. While as pointed out earlier, web-based
surveys can be structured to ensure anonymity of
responses, respondents may not believe it, and may not
respond. It must be noted, however, that this same
problem exists with other data collection methods.
- Rate of return may be inconsistent.
While as noted above, return rates are generally
higher than mailed or telephone surveys, it has been our
experience that depending on the survey and the
procedures used to collect data, return rates may vary
from as little as 15% to as high as 80%.
- Certain costs may be more. Web-bases
surveys may incur costs not normally associated with more
traditional data collection methods. They include
computer processing time, wages for the HTML author,
software and computer application costs, hardware
purchases, cost for web space, and electronic security
and storage fees. As point out earlier, these costs will
vary depending upon the type of support and assistance
provided by a particular institution.
- Hardware, software, and server malfunctions may
occur. The realities of web-based data collection
procedures are that hard drives crash, software glitches
and failures occur, floppy discs can fail, and other
problems may happen. Human errors in programming, storing
data, and lack of expertise can also cause problems.
So what's the definitive answer on web-based data
collection? It spite of its many disadvantages, we believe
it is the wave of the future, and that ten years from now,
mailed and telephone surveys will be as archaic as buggy
whips and eight track tape players. What do you think?
Reference
Marine,
R. J. (2000). Evolution of survey modes during the
1990s. University Park, PA: Unpublished manuscript.
*This article is based primarily on
"Web-Based Data Collection," by Thomas I. Wortman and M. Lee
Upcraft, a chapter in a book entitled Assessment Practice
in Student Affairs: An Applications Manual, authored and
edited by John H. Schuh, M. Lee Upcraft and Associates. San
Francisco: Jossey-Bass
(In Press).
Submit a
Letter to the Editor |
Printer
Friendly Version
|
