Chapter Five: Recommendations


Since there were a number of issues identified as challenges during development of the workshop, we decided to look at the assessment data in light of these challenges in order to make informed recommendations. In some cases, the data might indicate a need for changes in the workshop itself; in others, it might point to a need for further development of the working environment so that it would better support Web-based teaching and learning.

Time Allotted for Workshop

Throughout our planning, the greatest challenge was choosing the topics and the learning activities so that we packed the most crucial information and skill development into the fewest number of days. We were well aware that our planned eight days, even when broken up by a weekend, would probably be seen as too much time spent in a workshop by at least some participants.

Hence, it was not surprising to have some participants mention either that the workshop itself was too long or that certain aspects of the workshop were not an effective use of time. However, two things need to be noted lest we focus too hard on things that could be improved.

  1. The majority of the respondents, in all cases, thought that time was used effectively throughout the workshop.
  2. In areas where one or two respondents thought less time could be spent on an activity (independent work time, guest speakers, and learning about copyright law) an equal number of respondents indicated that they wanted more time spent on these activities or that they were a valuable part of the workshop.

Regarding the workshop duration and allocation of time to specific activities, we have an expected range of responses. The majority of respondents indicating satisfaction with the amount of time and its distribution among activities. Therefore, although we had one or two vocal participants who said that the workshop was too long, the majority of particpants did not indicate that this was true for them.

Therefore, we do not recommend that the duration of the workshop be shortened. If particular workshop modules are not found to be useful, they might not be offered in the future, but other modules might be added to take their places. Based on feedback during the debriefing session, several software packages (such as eGradebook and WebCT) might be added to the workshop if time could be found for them. Modules should not be cut simply to reduce the amount of total workshop time.

In a similar way, there were one or two respondents at several stages of workshop assessment who indicated that there was too much work time. The majority, however, found the amount of independent work time to be adequate, and there were occasional comments indicating that more work time would be beneficial. In the final reports, the item most frequently cited (7 times out of 11 reports) as a valuable part of the workshop was the independent, supported work time. Clearly, this item cannot be removed from the workshop design and should not be significantly shortened.

Faculty Development, Workload and Compensation

We have been fortunate to have a history of support for faculty development on which to build this new workshop. Since 1999, there have been one or two intensive "camp" workshops nearly every year, with shorter workshops offered frequently during the school year and over academic breaks. In designing the Web Course Delivery Workshop, we built and expanded upon this experience, aiming to reap the benefits of institutionalized faculty development presented by Sorg and Darling (2000). For the full list of desired benefits, see the Challenges section of this paper.

In looking at the data from all of the assessments, we find evidence that, for at least some participants, the workshop delivered the following benefits:

Other benefits of a faculty development program, such as developing an on-going faculty learning community, were not realized as a result of this workshop. No single instructional offering can meet all the needs of the faculty members as they move into this new teaching venue. This particular workshop needs to be part of an overall faculty development plan that incorporates a variety of activities.

One of the few comments about faculty development, workload and compensation issues came out in the mid-session survey. Two respondents mentioned that they were encouraged by the administration's apparent support for online instruction, but they were concerned that workload and compensation issues were not yet resolved.

While it might seem reasonable to avoid offering faculty development workshops until these questions are answered, this approach brings with it a risk that the administration will not develop answers to these questions in a vacuum. That is, until there is identifiable faculty committment in the form of existing courses, the administration may not consider the formation of policy to cover the new paradigm a priority. It can become a chicken and egg question, and one must start the process somewhere.

As we see from the offering of this workshop at the very beginning of the policy-formation process, pioneering faculty members were able to design and offer online, tech-enhanced, and hybrid courses. In fact, the first known hybrid course on our campus was designed during this workshop. We strongly recommend therefore that this workshop needs to be offered even if there are not clear-cut policies on faculty development, workload, and compensation. The existing policies, such as they are, should be part of the workshop itself, so that faculty members understand what they are getting into. However, they should also be aware that they, by their very act of creating online courses, will probably shape the policies.

It should also be mentioned that a small grant was awarded to the participants of this workshop for the purpose of purchasing equipment or software necessary to the completion of the project. One workshop participant mentioned during the debriefing session that this was a helpful aspect of the workshop. There were no other comments about the grant in the assessments for this workshop, although we have noted in other "camp" workshops where grants are given for equipment purchases that this is undeniably a draw factor.

Given that reliable, high-speed computer equipment is essential to conducting an online course, this stipend should perhaps be increased to assure that faculty members teaching online have the tools they need to be efficient. For our seven-day intensive Tech Camp, the stipend is $3000, which must be spent on a laptop computer plus required software and peripherals. The intent is to provide professors with technology to improve their classroom teaching. In online learning, computer technology is not merely a way to improve teaching, it is a tool without which teaching will simply not occur.

Preconceptions about Time Commitment when Teaching Online

As is noted in the previous section, by the mid-session assessment, some faculty respondents were concerned that teaching online could require more time than teaching the equivalent course face-to-face. This was not a surprise. At this point of the workshop, the participants had only seen online teaching from the student's standpoint in which, for many online instructional models, students are expected to participate in courses by writing frequently. Our faculty learners were understandably concerned about the amount of reading they would be required to do as instructors.

At the end of week one, participants had not begun to think about ways in which efficiencies of practice and course design can be used to make the time commitment in online teaching more manageable. That aspect of the workshop would be introduced in the following week. During this second half of the workshop, they would also hear from a number of current online instructors and learn something of how the experienced instructors manage their time.

The instruction during the second week apparently had an impact on the time commitment concerns because they were not mentioned in the final survey, the debriefing, or the final reports. While we cannot be certain from our limited data, why the time commitment of teaching online did not draw comment after the second week, we recommend continuing with the use of experienced online instructors as presenters, especially during the second week of the workshop when the focus is on introducing instructional tools and methods. This gives workshop participants solid answers to questions about the time commitment from people who are successfully teaching in the online environment.

Modeling of Teaching Methods

The choice of synchronous or asynchronous delivery method throughout the first week of the workshop was generally considered effective by the workshop participants. The majority (87.5%) considered our use of face-to-face time effective. One respondent considered the use of the time effective but said that the pace was too slow. Similarly, the asynchronous time was considered generally effective by the majority of respondents. In actuality, the difficulty faced was not with the effective choice between synchronous or asynchronous methods, but with having one or two advanced users in a workshop designed for novices.

By the end of the workshop, all respondents concluded that the instructors used appropriate instructional methods and that they had paced the workshops well. Hence, we do not recommend any changes to the methods indicated for each module. If other modules are created to meet specific needs of an institution, consider using the same philosphy when chosing instructional methodology: look at the learning objectives for the module and choose the method that will best attain that objective. If unsure whether that method lends itself better to a sychronous or asynchronous delivery format, look at the two lists given in the appropriate section of Challenges for guidance.

Use of Technology Tools in Workshop Pedagogically Appropriate

In designing this workshop, our intention was to make the workshop itself an example of the hybrid model and so not only presented modules that provided technical instruction but also presented instruction using the tools whenever appropriate for the desired learning objective. Since we were aiming to model solid pedagogical methods, we looked to the assessment data to see if our respondents had voiced any opinions.

Generally, the data indicate that workshop participants did not perceive inappropriate use of technology in the workshop and valued that we actually used online learning tools in the workshop rather than just talk about how they might be used in a classroom. However, we cannot draw too strong a conclusion here because we did not ask the question directly and only look at the other data and comments to get a sense of the participants' perceptions in this area.

One comment, however, does stand out. In the mid-session survey, one respondent mentioned that there was "too much focus on time in front of the computers." This comment was actually one that we welcomed seeing in the written assessment since we had heard similar comments in other online workshops taken by faculty members.

Recall that the first week of the workshop focused on putting the workshop attendees in the role of students in a hybrid class. This role, in order to be accurate, necessitated using the computers as an integral part of the participants' learning. While they certainly are aware that online learning will involve a computer, many of the faculty have never been in such a role and really do not grasp what it means to be a student in an online course. Nor do they often have a firm understanding of how much of their time will be spent in front of a computer in order to teach. This can be an eye-opening experience as they realize how much socialization they may give up or need to find in different modes when they enter this new mode of teaching. Therefore, we were pleased to see this awareness surface in not only this respondent's comment but also in informal conversations during the workshop itself.

As a result, we do not recommend substantial changes to the philosophy of involving workshop participants as online learners nor do we advocate lessening the amount of time they spend in computer-based tasks during the workshop. If they are not comfortable with computer-mediated learning in this environment, it is better to get this out in the open for discussion and reflection during the workshop. It may be possible that certain instructors would be better suited to continuing in a face-to-face learning situation, in which case they should be aided in discovering that fact early in the design of an online course. Alternatively, some instructors may be perfectly able to make the adjustment to online teaching so long as they are aware of the changes they will encounter. Putting them in an environment where they may become uncomfortable does them a service in the long run.

Copyright Issues

Of all the modules in the curriculum, the ones that were the most challenging to write were those on the educational use of copyrighted materials. This is a complex topic, made more so by the various and expanding environments in which learning occurs. Creating learning modules that will not take substantial amounts of time but will result in a significant learning experience can be a challenge. This curriculum includes three separate but related modules on the use of copyrighted material for educational purposes, which might seem extreme. So, we looked at the assesment data in particular to answer the question of whether or not this was too much time devoted to one topic.

In general, the survey results were favorable. The majority of respondents in the final survey agreed that the content of the workshop was appropriate to their background. All respondents agreed that the content was appropriately balanced between general and specific topics. All respondents also agreed that the workshop covered topics that they needed to know.

In the comments section of the final survey, one respondent did write that too much time (over the course of three days) was spent on copyright. However, two other respondents cited that copyright was one of the valuable aspects of the workshop.

On the balance, we recommend that copyright be presented in depth in a manner similar to our other modules. Reading, discussion, and case studies, while they take time, engage the learner in a way that a simple presentation does not.

One possible change would be to address the question of use of copyrighted material from another angle. While writing the Challenges section, we speculated that some of the confusion regarding use of copyrighted material may stem from the different roles that faculty members hold in relation to copyrighted material as authors, owners, and users. This analysis might be used to develop another or alternative case study to help faculty members create and use copyrighted material in their work.

Students' Suitability for Online Learning

There is little data from the workshop assessments upon which to base recommendations. This is an unfortunate gap in our assessment process since the question of student readiness for online education was raised by workshop participants during discussions. Also, there was some theorizing that online courses might be more appropriate for graduate students or upperclassmates than for freshmen. If this remains a design consideration, we should include one or more questions in the assessments to ascertain faculty perceptions regarding student readiness for online learning.

The one area of comment regarding student suitability or readiness for online learning that we see in the assessment data showed appreciation for the tools we had in place for self-assessment. The browser test was specifically mentioned, and a need for more tools like this was indicated.

Since the offering of this workshop, more student readiness assessments and training for online learning are slowly being developed. The University of Minnesota School of Nursing, for example, has developed an online orientation course that must be successfully completed before other online courses for the program may be taken. Enforcing such a prerequisite, however, may involve a coordinated effort among various entitites. Requiring that students take readiness assessments or online orientations may involve not only the instructor and program coordinators, but also an institution's registration system. This is the point at which a grassroots approach may need to evolve and involve systemic change and support from an institution's administration.

Teaching Pedagogy May Pose a Political Issue for Information Technology

Based on research into effective faculty technical skill development, this curriculum integrated the teaching of technical tools with pedagogical instruction. This was a departure from normal practice at our institution where separation between the two activities is generally maintained.

Based on the assessment results, the experiment was a success. One item in the final workshop survey addressed the mix of technical and pedagogical instruction. For that item, 89% indicated agreement, and only 11% indicated disagreement with the survey statement that the workshop "[h]ad a good mix of technical and pedagogy instruction". Because of the way the item was worded, we do not know which should have been more heavily favored by the respondents who indicated disagreement, and there were no free-form comments that addressed this issue. However, it is clear that the majority of the respondents thought the mix was adequate.

Success in this area may have depended upon a number of factors. As is mentioned in subsequent sections, the curriculum is designed to give a voice to current online instructors and other key departments. Making use of their expertise in development and presentation of some modules may have given the perception that educators were conducting the pedagogical instruction portion of the workshop.

In addition, most instruction on pedagogy conducted by the Information Technology Systems and Services staff was done in a constructivist mode. Hence, we were not telling faculty members how to teach. Instead, we were facilitating their learning and making use of their existing expertise and experience. This is a far less threatening stance and probably contributed substantially to the success of this new model of integrating technical and pedagogical instruction. Care should be taken when conducting this workshop to make use of these two approaches, which are specified in the instructors' notes for each module where appropriate.

Giving Voice to Leading-Edge Teachers

As mentioned in the previous section, incorporating presentations by current online instructors was probably one of the factors that made the workshop a success. From the standpoint of the learners, this was a valuable part of the workshop. Throughout the assessment data, the inclusion of presentations by people who have designed and offered courses online was cited as a valuable aspect of the workshop by between 25% and 77% of the respondents.

Still, some respondents, during the debriefing session, expressed the opinion that there were too many guest presenters, and that some example courses were overwhelmingly extensive and intimidating to novice online instructors. In part, this opinion may have stemmed from how the guest presenters were scheduled.

When designing the workshop, we had tentatively organized modules so that presentations of similar instructional modules would be grouped together and that there would be a progression from the streamlined implementations to the more complex. This tentative schedule had to be adjusted in order to accommodate the availability of the guest speakers.As a result , we conducted the workshop with a less-than-ideal schedule, distributing sessions over several modules.

This was a risk discussed during the design of this particular workshop's schedule. We decided to modify the ideal schedule to accommodate the schedules of the speakers in order to place greater priority on including the experience of the pioneering instructors. Since this workshop was first conducted at the beginning of our institution's incorporation of online learning, we chose to acknowledge and learn from the work already done by those who had first embraced this mode of teaching. This was a very select group of people, and there were few from whom to draw guest speakers. In the future, we will have an increasingly large pool of experienced online instructors from which to draw guest presenters, and the ideal schedule may be approached.

For institutions that also are beginning to adopt online learning as a mainstream instructional mode, we recommend that the balance of inclusion vs. efficiency be tipped toward including the most influential pioneering online educators even if it means breaking up your ideal workshop schedule to accommodate their schedules. As more experienced instructors become available, arrange the workshop so that related concepts remain ideally grouped for the benefit of the learners.

Participants Need to Experience Both Student and Teacher Perspective

We could not find any support in the literature for this approach in which we introduced learners to some of the software programs used at our institution to conduct online teaching. This method grew out of our experience in mentoring faculty members one-on-one, where it has worked very well. However, we were not sure how this would work in a group setting. Fortunately, we have assessment data now upon which to base recommendations for the future.

Overall, this approach was well received by the workshop participants. It was mentioned as one of the valuable aspects of the workshop in the final survey, the debriefing, and the final participant reports. Also, in the final workshop survey, all respondents agreed that the instructors used appropriate instructional methods and that the workshop exercises alerted them to how students would use the instructional tools. Therefore, this approach gave the participants exposure to the student view of online instruction without making them defensive or excessively uncomfortable.

One comment made during the mid-session assessment indicated some possible discomfort with our approach. A respondent noted that he or she thought there was "too much focus on time in front of the computers" as part of the first week of the workshop. Rather than causing us to reconsider how the first week's activities play out, however, we think this response is a healthy and valuable one. As discussed above, if a faculty member is not comfortable with being an online student in this short simulation, then he or she might reconsider teaching a course entirely or predominantly online. As with any course, the most valuable lessons are not always the ones that are planned.

Based on our experiences and the assessment data, we strongly encourage adoption of this approach, particularly in situations where prospective online instructors have never actually been students in an online course.

What Support Exists on Campus for the Hybrid Model?

In the "Challenges" section, we noted that, at the time this workshop was developed, no one was teaching a course that fit into the hybrid model with regular, weekly class sessions and regular, weekly online activities. Members of some learning communities met face-to-face at the beginning of a term with the rest of the term's activities taking place online, but these could only hint at how a blended course would work with both types of activities taking place each week. As such, we could not give definitive answers to questions that might arise about the administration's support of such a course, nor could we invite a seasoned instructor to speak about how a hybrid course would work.

In light of such challenges, we might have chosen not to include this model in our workshop, however, we took the risk with the hope that some pioneering instructor would pick up the challenge. At our institution, that has usually been the way new methods of instruction, especially using technology, have worked their way into the mainstream. We counted on this trend to introduce the hybrid model at our institution.

In the assessment data, we see appreciation for coverage of the hybrid instructional model in theory and the administration's indication of support for courses incorporating this type of course design. In the mid-session survey, one respondent in particular indicated switching an existing course to a hybrid model because of a greater understanding of how online and in-class activities could complement each other. Also, in the final survey the hybrid model and support for it was mentioned as a helpful aspect.

When we looked at the final projects and reports, we were pleasantly surprised to find that not just one but four instructors had taken up the challenge and have developed and conducted hybrid courses. As our experience shows, it is possible to lay the groundwork in a workshop for a new type of course and have learners take that theory and make it real, even if they have no one else's experience to build upon. Provided that there is truly support from an institution's administration, we recommend presenting options to learners and letting them build courses using whatever model is most approprate for each course, even if the model has never been implemented at that institution before.

Representing a Full Range of Teaching Options Without Overwhelming Participants

Did the range of teaching options presented overwhelm the participants? That had been a major consideration as we designed the workshop since we wanted to give enough information but realized that we could not cover all possible online course design models in the time available.

On this point, the assessment data are mixed. Throughout the various assessments, one or two respondents mentioned the value of discussions and presentations on the hybrid, learning community, and INI (individualized instruction) models. On the other hand, some participants were overwhelmed by the number of presentations involved in representing the range of possibilities. In fact, a respondent reacted to the various sessions as an attempt to "sell" certain online instruction models.

If this were the only data addressing this question, we might conduct this workshop again with similar presentations coordinated with group learning activities more effectively and lessen the impression that we were advocating for particular models. However, it is interesting to note that, in the mid-session evaluation, only one respondent changed his or her planned instructional model because of a greater understanding of pedagogical models. The other three changed theirs because they learned of the limitations of technology they had assumed they would use.

This brings up the related question: Were the presentations on the various instructional models essential to this workshop? Since we did not ask learners directly about the value of these presentations nor their impact on the form of their projects, we are not certain that they were, in fact, required. Given the slight impact of the instructional model presentation at mid-session, this is a design aspect that should be reviewed for future offerings of the workshop.

In this area, it is difficult to make a firm recommendation. It is possible, especially as more courses are offered online, that workshop participants will enter with sufficient knowledge of the range of teaching options to make a valid choice before the workshop begins. As an alternative for those unfamiliar with the full range of possibilities, a pre-course could be offered. However, it would be difficult to conduct this workshop if participants hold different definitions and understandings of how the various models worked. While lessening the amount of coverage of options would help minimize the taxing length of the workshop, care would need to be taken to come to common definitions early in the workshop in order to prevent confusion.

Positivist Versus Constructivist

There is no data from the workshop assessments upon which to base recommendations. As we explain in the "Challenges" section on "Positivist Versus Constructivist: Our Approach," we do not view this as a contest between good and evil, but a continuum between two philosophies where an appropriate method of teaching exists for a given learning outcome. To arrive at a suitable method for a desired learning outcome, these questions are asked:

  1. When is it appropriate to base learning on observable data?
  2. When is it appropriate to base learning on meanings developed by the learner?
  3. How do these two approaches blend together to inform knowledge and action?

Examining any method used in teaching is an ongoing process, and may change over time. Drawing battle lines over the superiority of one philosophy over another ignores the continuum between the two philosophies. We see the value in moving along this continuum as the learning situation dictates.

Copyright 2005 Barbara Z. Johnson and Bruce D. Reeves