Models for Evaluating Student Affairs Web Sites

A good web site is a delight and a poor web site is worse than no web site at all. But, what makes a good web site? This is an evaluation question, and an answerable one. A literature is emerging to address these kinds of evaluation questions (Center for Instructional Technology, Academic Technology and Networks, 2000; Cornell University Libraries, 2001;and Schrock, 1998). While there are many "best practice" statements and models of good and bad web sites (Flanders, 2001), these are generally values driven and developed in the absence of data.

Information Technology (IT) creates asynchronous computer mediated interactions between people. Computer mediated interaction does not contain valuable visual communication cues, in contrast to face-to-face interaction. The loss of these cues can lead to misunderstandings on many levels. The lack of feedback mechanisms, inherent in face-to-face interaction, can severely limit communication. Student affairs professionals can never be sure that students are reading our web pages in the way that we intended, and students can never be sure if we are responding to their needs. Student Affairs is essentially student focused and interpersonal, and our web sites should reflect our core values.

Understanding what makes a good web site is difficult; especially understanding what makes a good web site within a specific campuses institutional and organizational context. Introducing the reader to multiple strategies for enhancing web site quality is the heart of this material. All of the evaluation questions are focused on elements that can be changed to better meet student need.

Using multiple strategies for evaluating your web site will result in a stronger evaluation, and consequently a better web site. While each evaluation strategy can be used alone, the strategies presented here were not designed to stand alone, but to reflect different evaluation questions. Because each campus will have a unique web site, the strategies presented here contain general questions that form the beginning of an evaluation plan. Campus specific questions should be developed and explored.

The strategies presented here are introduced as models. Much as a model car is a miniature representation of a real car, lacking size, detail and the ability to drive, these models are miniatures of full-scale evaluation strategies. To use a full scale evaluation, some assembly is required.

Basic Values For Student Affairs Web Sites

Our work in student affairs is values driven. In evaluating a site, determine the basic values that should be reflected and determine if these are reflected in the site. The following list of basic values are proposed as a useful start in developing a list of values to be used in evaluating student affairs web sites:

• Inclusion: All students, student groups and student organizations must be included in material presented in the site.
• Representation: All departments and divisions in student affairs, and allied departments must be represented.
• Active: Content must contain active information such as current events, schedules, names, dates, and times.
• Informative: Content must contain regulatory information such as codes of student conduct and the student government constitution.
• Interactive: The web site must provide mechanisms for student interaction.
• Currency: All information must be current and none out-of-date.
• Convenience: The site must be designed for easy use.
• Community Building: The web site must link appropriately to the campus, the community and the world.
• Developmental: The web site must promote student development goals.
• Learning focused: The web site must promote appropriate learning goals.
• Clarity: The web site must have a written set of goals and measurable outcomes.
• Accountability: The site must contain evaluation mechanisms such as feed back sections and counters.
• Accessibility: The web site must meet accessibility guidelines.

Evaluating A Web Site As Campus Ecology

Environmental theory and practice is a strong tool to examine the features of your web site (Banning, Davies & Quick, 2000; Wallace, 2000). Asking what, how, who, when and where questions are important in evaluating a web site. Web pages can succumb to the same analytical techniques as the campus environment.

The questions below reflect a human aggregate and a perceptual model of campus. There are undoubtedly many additional questions that can and should be asked on different campuses.

• Does your web site primarily reflect data, ideas, people or things?
• Does this orientation reflect the diversity and reality of your campus climate?
• Does your web site primarily reflect Realistic, Investigative, Artistic, Social, Enterprising or Conventional people and situations?
• Does this reflect the diversity and reality of your campus climate?
• Does your web site reflect the diverse career and degree aspirations of your students?
• Is there a 'human touch' on each page, such as a quote, drawing, photograph or personal statement?
• Are the types of people on your campus, such as gender, age, ethnicity, social class, faculty, student, support staff, administration, art majors, engineers, etc. reflected visually and in text on the web site?
• Is their material on the web site focused on the different types of people, such as ethnic students, commuter students, older students, etc. on your campus?
• What is the ratio of people to content on the web site?
• What is the ration of active information to reference information on your pages. Active information is staff names, phone numbers and events, which need to be updated regularly. Reference information is the code of student conduct and other regulatory information which is not subject to frequent change.
• What is the ratio of academic material to social material on your web site?

As an evaluation exercise to learn how students perceive your web site, have five students from a different campus look at all of your pages and write a brief description of your campus based on your web pages.

Web Sites as Student Development Tools

Student development and learning provide wonderful means to examine features of your web site. These questions should be answered as a degree, not as yes-no questions, and specific web features should be associated with student development and learning goals.

• What models of student development drive your web site?
• To what extent does the web site promote student development?
• Using Chickering (1993) as a developmental model, the following questions might be applied to a web site:
  • How is the development of purpose, such as careers, volunteerism, reflected in the web site?
  • How is the development of identity reflected in the web site?
  • To what extent does the web site promote student autonomy?
• To what extent does the web site promote student community development?

Evaluating Web Sites as Learning Tools

Concepts and practices of student learning are effective in evaluating a student affairs web site. Learning is the acquisition of knowledge and skills as the result of experience. The web site can be seen as an experience and as providing information about experiences on campus.

• What models of student learning drive your web site?
• Are there articulated learning goals for the web site?
• If a taxonomy of learning, such as Bloom (1956) is part of the Student Affairs Mission, how are these presented in the web site?
• To what extent does the web site promote student learning?
• What knowledge base is presented or linked in the web site, such as campus events, regulations, contact information, social support or academic support?
• What knowledge is assumed in order to use the web site, such as department names and organizational chart?
• What skills are presented or linked in the web site, such as intramural sports or social skills programs?
• What skills are assumed in order to use the web site, such as accessibility?
• What values are reflected in your web site?
• Which of Bloom's taxonomy items are covered in your web site?
• How do we track student access of web pages and how students use the site?

Evaluating Web Site Design and Features

Web sites can be seen as a projection of the institution's values and structure. Most web sites in student affairs use the organizational structure as the web structure. Unfortunately, most students use the organization in different ways. Organizational structure is designed for certain managerial goals; web site structures should be designed with other goals in mind.

• What are the goals for your web site?
• Who is the target audience for your web site?
• Does your web site architecture reflect your organizational chart or does your web site architecture reflect how students access information? 
• Is material available within three clicks?
• Can specific material be found within 30 seconds?
• Are there counters on each page?
• Is individual access tracked?
• If information is captured about student use of your web site, is there a privacy statement?

Student Evaluation of Student Affairs Web Pages

One method of evaluating a web site is to have a novice user try to find specific information on the site. To do this have a student who does not know your web site try to find specific items starting from the campus home page using only links. If these cannot be found, then the student should use the campus search feature to find them. Watch as the student tries to find the information, and pay attention to how the student uses the site. If you need to answer any questions from the student, the search is a failure, since all information and directions for getting it should be on the web site. 

The student should try to find active information that is subject to regular changes, such as the intramural sports teams and schedule, the name and phone number for the Director of the career center, a photo of the CSAO, and a list of student organizations. The student should also try to find regulatory information that is not subject to regular changes, such as the Student Government Constitution, the campus Code of Student Conduct, and the Student Affairs Mission Statement. Links are an important part of any web site, and the student should try to find links from the student affairs pages to affiliated student affairs sites, professional organizations, and to specific campus academic activities.

Evaluating Web Page Design and Content for Accessibility

Accessibility is a campus requirement and a core student affairs value. Building accessibility and web page accessibility should be of equal concern. There is extensive literature and sets of recommendations for web page accessibility. The Web Accessibility Initiative (WAI) at the World Wide Web Consortium (W3C) provides excellent and thorough information and accessibility guidelines, which are summarized below. A visit to their web site is required reading when evaluating web pages.  

Web Accessibility Initiative (WAI) Guidelines that Web Content Developers Must Satisfy. In General Provide a text equivalent for every non-text element. Ensure that all information conveyed with color is also available without color, for example from context or markup. Clearly identify changes in the natural language of a document's text and any text equivalents (e.g., captions). Organize documents so they may be read without style sheets. Ensure that equivalents for dynamic content are updated when the dynamic content changes. Until user agents allow users to control flickering, avoid causing the screen to flicker. Use the clearest and simplest language appropriate for a site's content. And if you use images and image maps Provide redundant text links for each active region of a server-side image map. Provide client-side image maps instead of server-side image maps except where the regions cannot be defined with an available geometric shape. And if you use tables For data tables, identify row and column headers. For data tables that have two or more logical levels of row or column headers, use markup to associate data cells and header cells. And if you use frames Title each frame to facilitate frame identification and navigation. And if you use applets and scripts Ensure that pages are usable when scripts, applets, or other programmatic objects are turned off or not supported. If this is not possible, provide equivalent information on an alternative accessible page. And if you use multimedia Until user agents can automatically read aloud the text equivalent of a visual track, provide an auditory description of the important information of the visual track of a multimedia presentation. For any time-based multimedia presentation (e.g., a movie or animation), synchronize equivalent alternatives (e.g., captions or auditory descriptions of the visual track) with the presentation. And if all else fails If, after best efforts, you cannot create an accessible page, provide a link to an alternative page that uses W3C technologies, is accessible, has equivalent information (or functionality), and is updated as often as the inaccessible (original) page.

Checklist for Evaluating a Student Affairs Web Site

Web sites can be designed with many things in mind such as stakeholders, expense, color scheme, and many other web features. Four main areas, web site navigation and design, web site technical details, web site aesthetics and web site content will serve as a beginning point for the evaluation process for student affairs web pages. While complicated web site evaluation schemes exist, simple is always a good starting point.

Presented in table form below is a checklist that includes details for each of the four areas and which can serve as a starting point for evaluating a web site.  The point assignments are arbitrary, but serve as a vehicle to summarize the data collection process.

Navigation and Design
	Points
	(4)	(3)	(2)	(1)	(0)
Material should be found within three clicks from the home page and take no more than 30 seconds to load, even on a slow modem.	3 clicks	4 clicks	5 clicks	6 clicks	7 clicks
All pages should list a most recent update date, and have been updated within the past 12 months.	All		Most		Some
There should be an appropriate and consistent tool bar/navigation set on pages.	All		Most		Some
It should be very easy is it to find the Student Affairs pages from the school home page. *Main link - not a pull down menu item.	Home page link*	Second page link	Menu item link		No link
Material should be organized or indexed by function (department).	All depts.		Most depts.		Some depts.
Material should be organized or indexed by population being served - Information for students, faculty and staff.	All		2 of 3		Stdts. only
Sum of points in each column
Navigation and Design Points
Technical Details
	Points
	(4)	(3)	(2)	(1)	(0)
There should be a search function for the pages or site.	Yes				No
There should be a feedback section for pages or site.	Yes				No
There should be page counters on all pages.	Yes				No
Pages should meet accessibility standards.	Yes		Most		No
Pages should reside on a fast server.	Fast		Med.		Slow
Pages should load quickly at 28.8 baud.	Fast		Med.		Slow
All links should be active.	All				Most
Sum of points in each column
Technical Details Points
Aesthetic Appropriateness
	Points
	(4)	(3)	(2)	(1)	(0)
There should be a consistent 'look and feel' within the site.	All		Many		Some
There should be school and division identification throughout the site.	All		Many		Some
The pages should be appropriately attractive.	All		Many		Some
Sum of points in each column
Aesthetic Appropriateness Points
Content
	Points
	(4)	(3)	(2)	(1)	(0)
Information should be provided for all types of current, prospective and alumni students, faculty and staff.	All		3 of 5		1 of 5
Care should be taken to not overemphasize any student sub-population.	All		Many		Some
Student affairs functional areas should be covered broadly.	All		Many		Some
Student affairs functional areas should be covered in depth.	Deep		Some		None
Academic material should be well integrated into the student affairs pages.	Fully		Some		None
Material must be current and accurate.	All		Many		Some
Department or division home page should list contact names, numbers and E-mail addresses for important people, as well as hours of operation.	All		Many		Some
Department or Division Mission is available.	Yes				No
Sum of points in each column
Content Points
Total Evaluation Score

References

Banning, J. H., Davies, T. G., & Quick, D. G., (2000) The campus web visit, Student Affairs Online,1(Fall). Available at http://www.studentaffairs.com/ejournal/Fall_2000/art3.html

Bloom, B. (1956). Taxonomy of educational objectives. New York, David McKay Company, Inc.

Center for Instructional Technology, Academic Technology and Networks(2000). Evaluating web sites for educational uses: Bibliography and checklist, University of North Carolina, Chapel Hill, NC. Available at: http://www.unc.edu/cit/guides/irg-49.html  

Chickering, A, & Rieser, L. (1993) Education and Identity (2nd Ed.) San Francisco: Jossey-Bass

Cornell University Libraries (2001) Evaluating Web Sites: Criteria and Tools, Available at: http://www.library.cornell.edu/okuref/research/webeval.html 

Flanders, V. (2001) Web Pages That Suck.com. Available at http://www.webpagesthatsuck.com/ 

Schrock, K., (1998) Evaluation of World Wide Web sites: An annotated bibliography, ERIC EDO-IR-98-02 http://ericir.syr.edu/ithome/digests/edoir9802.html

Wallace, H. (2000). Campus ecology theory and websites: One example of applying traditional student affairs theory to technology, Student Affairs Online, 1(Fall). Available at http://www.studentaffairs.com/ejournal/Fall_2000/art4.html