Introduction
The 2016 Instructional Technology Survey was released on March 28th, 2016, and was considered “closed” on April 8th, 2016. The survey was distributed to roughly 390 faculty members at St Olaf College via the stolaf-faculty email alias. 92 responses were received, for a gross response rate of 23.6%.
Survey Instrument & Methods
The survey instrument consisted of 120 questions in GravityForms on the St Olaf IT WordPress site, organized into 7 sections: Demographic Data, Priorities for IT and iTech, Digital Scholarship Center, Classroom Technology, relationship with technology, Moodle, and finally how faculty use technology with their students. The Moodle section was written by Carly Born and Paula Lackie of Carleton College’s Information Technology Services, and represents the satisfaction assessment half of their own survey that they distributed to the the faculty-staff community at Carleton College. The Moodle survey instrument was approved by Carleton IRB, and the St Olaf IR board respected the Carleton approval after reviewing the survey instrument.
Survey questions generally followed Likert scales organized by topic, followed by an open-response question at the end of each topic. There were a total of 11 qualitative-response questions, two of which concerned classrooms and could not easily be fit into a standard format, and 9 open-response questions. Charts were created for the entire College (All-College), and for each academic division, except for Interdisciplinary and General Studies (IGS). Only 5 individuals identified IGS as their primary division affiliation. In all, 61 bar charts were created from the survey responses, plotting percent of respondents for each Likert-scale response item. The final question group in the survey instrument, mostly concerned with student-owned devices in the classroom only had 11 total responses College-wide. A chart was created to display all-College responses, but not for the individual academic divisions, since the response rate for any one division was too low.
Discussion
Section One: Demographics
Out of the 92 responses received, the majority were from tenured and tenure-track faculty (73.9%). Within the Assistant Professor category, there are likely long-term non-tenure track faculty as well, but they are impossible to identify within the responses. Within academic divisions, the majority of the responses came from the Humanities (31.5%) and the lowest from Interdisciplinary and General Studies (5.4%), but it is likely the full-time faculty are affiliated with other divisions as well. The second-highest response rate came from Natural Sciences and Mathematics (28.3%)
Please note: click on a chart to embiggen it for easier viewing. Use the arrows on the left and right of each chart to navigate between charts for the entire College and each academic division.
Section Two: IT/iTech Priorities
Questions in this section covered how respondents approach learning new technology, what technologies the community are interested in, and where Instructional Technology should best be spending its time and efforts. Across the entire College, there is considerable interest in creating media (34% very interested, 45% slightly interested), in web publishing (31% very interested, 46% slightly interested), and in data visualization (33% very interested, 45% slightly interested). Following those interests, there remains significant interest in GIS and mapping (26% very interested, 29% slightly interested), and integration of mobile devices in teaching (22% very interested, 45% slightly interested).
For learning new technology, the most popular method College-wide is in-person consultations with IT staff, followed by hands-on workshops, and just diving into the technology. However, responses varied across academic divisions, driven largely by the Humanities division, where 66% of respondents were very interested in In-person consultations, and 48% very interested in Hands-on workshops. By contrast, faculty in the Natural Sciences and Mathematics division were less interested in In-person consultation (42%) and Hands-on workshops (23%, though 66% of science faculty viewed workshops as somewhat effective). Online documentation was viewed as at least somewhat effective by the vast majority of faculty. Interestingly, hand-on workshops were viewed as more effective than topic-driven workshop series by a considerable margin (43% very effective vs. 20% very effective, though both workshop types were viewed similarly as somewhat effective). While many viewed Lynda.com video tutorials as somewhat effective (42%), only 10% of respondents viewed Lynda.com as very effective, and 26% of faculty couldn’t speak to the efficacy of Lynda.com and responded with Don’t know. Support for Lynda.com is greatest within the Fine Arts division (19% Very effective, 38% Somewhat effective) but even there 38% of faculty Don’t know about its effectiveness. Strong support for Lynda.com is weakest within Natural Sciences and Mathematics with only 4% of respondents listing Lynda.com as a Very effective, but 42% listing them as Somewhat effective. However, these data show the faculty perception of Lynda.com; they say nothing about how the Lynda.com service is viewed by the student community, nor anything about the adoption of Lynda.com in St Olaf courses.
Overall, the College community felt that Instructional Technology’s highest priorities should be Active Learning classrooms (53%), faculty development opportunities (46%), Moodle (43%), and faculty project support (36%). Less important were topics like lecture capture (3%), video conferencing support (12%), and Open Lab (23%). Interestingly, a full 38% of the responding faculty felt that a Stargate was a high priority for the fourth floor of Rolvaag Library, though majority felt that a Stargate was not important (29%), or that they didn’t know what a Stargate was (33%). Yet this remains an exciting possibility to add to iTech’s services.
Section Three: DiSCO and DiSCO Services
This section covered the likelihood that services already offered within the DiSCO would be used by the faculty community, and future plans for iTech and DiSCO services. For DiSCO services, by far the most likely service to be used is consultation with DiSCO and iTech staff (35% Very likely). Interestingly, this response is higher than the Open Lab response in the previous section (23% vs 29%). It appears that the Open Lab concept, which held prior to the IT department’s move to the 4th floor of RML and the creation of the DiSCO, has morphed somewhat. 14% felt that they would be Very Likely to use the sound booth rooms, and 32% felt that they would be Likely to use the rooms. Response rates were driven largely by Fine Arts (38% Very likely to use Sound Booths). These rooms are in high demand regardless. The video studio has similar response values; 15% Very likely, and 41% Likely to use the space.
On the other hand, the majority of faculty responded that they would be Very unlikely (30%) or Unlikely (37%) to use the 3D printer, with only 5% of faculty responding that they would be Very likely (5%) to use it. Similarly, 45% of faculty responded that they would be Unlikely to use small group work space in the DiSCO, and 16% would be Very unlikely to use them. Yet anecdotally, we see faculty members working in the DiSCO without students on a regular basis– often to work with the computers with specialized software. While only 11% of respondents felt that they would be Very likely to use an open computer lab in the DiSCO, 32% felt that they would be Likely to do so.
For future plans, the faculty responses were generally equivocal with little clarity about the direction they’d like the DiSCO to take. However, somewhat interestingly, only 4% of faculty felt that DiSCO staff being available after hours was Very important, while 33% felt it was Somewhat important. 21% felt that after-hours staffing was Slightly important, and 33% felt it was Not important. Only 10% didn’t know or had no opinion on this issue. A full 33% of the responding faculty felt that a Holodeck was a Very important addition to the DiSCO in the future, however– something we will get right on as technology permits.
Other future plans, such as a potential makerspace, huddle spaces, video conference room, and presentation practice have unclear responses split relatively evenly over the possible choices. Workshops and tutorials, on the other hand, do show significant support. 23% of respondents felt workshops were Very important, with 42% felt workshops were Somewhat important. Only 9% felt workshops in the DiSCO were Not important.
Overall, while oftentimes a majority of faculty respond that they will be unlikely to use a particular DiSCO service, even a small minority of faculty assignments that require audio or video components is more than enough to saturate existing DiSCO capacity. While many future improvements don’t enjoy unequivocal support from the faculty, workshops do. It also seems clear that the faculty view the DiSCO as the primary service point for Instructional Technology, regardless of any Open Lab opportunities iTech may support.
Section Four: Teaching Spaces and Classroom Technology
This section of the survey contained questions about specific classroom spaces that are not included in these survey responses because they were impossible to chart. However, they did go directly to the Presentation Technology Manager and the Instructional Technology team for assessment and action. The other major question in the section concerned how often specific classroom technologies are used by the faculty. For this question, it’s instructive to notice the Never responses: faculty Never use VHS tapes (85%), or a Region 2 or 3 DVD (72%). However, 12% of Humanities respondents use Region 2 or 3 DVDs weekly, and 58% use them at least monthly, so they remain important for some pedagogical applications.
Likewise, 63% of respondents Never use clickers, and 43% never use wireless laptop connections. But sometimes a Never response appears to be for lack of availability– 19% of respondents said they’d use a laptop wireless connection if it were available in the room, where 13% said they use a wireless laptop connection daily, and 15% said the same for Clickers, where 11% use them daily. Similarly, 28% of respondents make daily use of smart displays on classroom podiums, 34% never use them, but 16% would use them if available. And even when not in daily use, many respondents use resources weekly or monthly as well, but generally not in situations where Never responses are in the majority.
We anticipate use of wireless laptop connections and smart displays to increase as availability of the resource increases within classrooms.
Section Five: You and Technology
Questions in this category related to respondents’ comfort level with technology and the kinds of technology they use. College-wide, the majority of respondents were Neutral (24%) to Extremely comfortable (23%) with technology. The majority of respondents college-wide were in between the two (46%). Comfort level was also stable across reported age ranges of the faculty– respondents in the 60+ age category were just as likely to be comfortable with technology as their colleagues in the 41-60 range, though those faculty members in the 21-40 category reported being the most comfortable with technology. What is clear from the survey response data is that age is not strongly correlated with discomfort with technology within the St Olaf College faculty. Different age groups may use technology differently and adopt different practices, and comfort levels do vary between the groups, but among respondents no faculty age group or academic division is notable for discomfort with technology.
Indeed, only 3% of respondents reported that they were not comfortable with technology, centered largely within the Humanities. Within that division, 7% reported being not comfortable with technology, and 17% reported being extremely comfortable with technology. Within Natural Sciences and Mathematics, however, 31% of their respondents are extremely comfortable with technology, and 54% say they’re between Neutral and extremely comfortable. No respondents in the Natural Sciences and Mathematics division reported being not comfortable with technology. Fine Arts respondents, too, had the vast majority of respondents as being neutral or more comfortable with technology, and a plurality (37.5%) are extremely comfortable with technology, similar to Social Sciences, where only 11.8% expressed discomfort with technology, the same number as being extremely comfortable. Only 6% of Fine Arts division respondents were not comfortable with technology, and no respondents from the Social Science division reported being not comfortable with technology.
Technology adoption is widespread across the College. Of the 92 respondents College-wide, 65% are using Google Drive daily, 58% are using Google Docs daily. And that 58% is a firm number; the lowest use rate is 56%. College-wide, 65% of faculty are using Google Drive, and 58% are using Google Docs. 70% are using Moodle daily, either for their courses or for community projects. 46% are using social media like Facebook and Twitter daily as well. For Adobe products, only 5% College-wide are using Creative Suite daily (and 23% monthly), while 39% are using Adobe Acrobat daily (and 30% weekly). 51% of responding faculty use some sort of tablet or e-reader on a daily basis.
On the whole, the St Olaf College faculty are largely quite comfortable with technology, make extensive use of it in their daily lives, and are working to incorporate technology into their teaching.
Section Six: Moodle
The Moodle section was comprised of 36 questions in four groups. The Moodle questions that were included in this year’s survey instrument were from Carleton College’s Moodle survey. St Olaf IT collaborated with Carleton by including their Moodle-satisfaction questions. This was by far the largest section of the 2016 survey instrument.
Broadly speaking, users are satisfied with Moodle administration activities, or they simply don’t use a given feature. There is, however, one glaring exception: the gradebook. Only 3% of responding faculty were very satisfied with setting up the Moodle gradebook to calculate grades. 15% were generally satisfied, but 18% and 20%, respectively, were generally unsatisfied or very dissatisfied. 44% don’t use the gradebook feature.
The 2015-16 Moodle server was the first St Olaf Moodle server to use Natural grade weighting– except that there were major bugs with Natural weighting at the time, and the best judgement of the Moodle team was to disable Natural weighting for the 2015-16 academic year. Additionally, the Moodle team implemented a custom, fully-responsive theme to look better on mobile devices for the 2015-16 server. While the theme by most accounts did look better, it also overrode part of the gradebook (that froze the left column with student’s names), causing much confusion and widespread adoption of the new Moodle single-view for gradebook items. The Instructional Technology team is very mindful of this issue. The new Moodle server for 2016-17 has a functional Natural weighting facility, a stock theme fully supports all gradebook features, and entirely re-written documentation for St Olaf’s Moodle, including for gradebook. We hope that the gradebook experience for all faculty has been improved with the changes.
For Moodle’s communication features, again, faculty are broadly satisfied with Moodle, or simply don’t use the features. Assignment features in Moodle are similar as well– but the majority of Moodle users simply don’t use the features. It is not clear if it’s the case that faculty know about and choose not to use a given feature, or simply don’t use the feature because they didn’t know it existed. For the Other interactions group of Moodle questions, “I don’t use the feature” was by far the majority response for all the prompts, though, again, respondents were largely satisfied with Moodle features.
From the tenor of the responses about Moodle it seems clear that, with the exception of the gradebook, that faculty are generally satisfied with Moodle– to the extent they use it. And while they’re broadly satisfied, they don’t love Moodle either. Comments like the below were relatively common:
- “Moodle is cumbersome in general. I don’t consider it to be an efficient, user-friendly tool. It is a jack-of-all-trades, in the sense of doing many things but none of them particularly well. Nevertheless, I use it often and the students are comfortable with it. I wish it were quicker and easier for me to work with!”
- “Moodle cannot be user modified in ways I would like to use it. It’s OK.”
- “I think Moodle has improved a great deal over the years! Go Moodle!”
- ‘Again, since I find the Moodle gradebook so difficult to use I tend to grade and communicate feedback to students via email.”
- “The items above that I’ve marked “do not use” are done in ways not using Moodle – generally because Moodle is not efficient or effective doing those things”
- “I am extremely pleased students can track their own grades, and that assignments and my responses are all in one place. So much better than the old days.”
While many of Moodle’s successes and failures stem directly from Moodle Core itself, since St Olaf IT does not significantly modify Moodle, Instructional Technology and the Moodle team are very aware of the faculty’s usability concerns. We are working to reduce significant user-interface changes year-to-year by adopting a standard stock theme that is included with Moodle, and we’re created complete new documentation for Moodle at St Olaf– especially for Gradebook. Find it at wp.stolaf.edu/it/moodle-documentation.
Section Seven: Faculty Perception of Student IT Needs
This section has questions revolving around how faculty perceive students’ IT needs are being met. The faculty are in agreement that students expect some element of technology in their courses (83%), that they have the technological skills to succeed in their courses (77%), and that they have access to the technology they need to complete their course activities (87%). 84% of respondents agree that students have adequate network access outside of class, and 79% of respondents agree that students have adequate network access in class. 53% of respondents agree that their students don’t know what a rotary telephone is– though 3% strongly disagree with that.
While there were some neutral responses to all of these questions (no more than 13%), disagreement was relatively minor. The only strong disagreement from respondents lay in rotary telephones, and disagreement on any one question did not exceed 7%.
Responses did vary by division, however: fully 96% of faculty in Natural Sciences and Mathematics felt that students expect some element of technology in their courses, whereas in the Social Sciences only 67% agreed or strongly agreed, 20% were neutral, and 13% disagreed. In the Humanities, 75% of faculty agreed about student technological expectations, 10% were neutral, and 7% disagreed. In the Fine Arts, only 6% were neutral and 88% agreed or strongly agreed, with 6% saying they didn’t know.
Lastly, the survey instrument had a series of questions about making use of student-owned devices in the classroom. These questions were not completed by most respondents– only 11 out of the total of 92. As such, there were not enough responses from each division to justify creating charts for them. Such charts would not be compelling or offer much insight. Still, while use of student-owned devices have broad acceptance in classrooms, 18% of those who responded do not allow laptops or tablets in their classrooms, and 45% do not allow smartphones. 9% represented laptops and tablets as anathema, and 27% for smartphones. Anathema was inserted somewhat as a joke for the last group of questions in the survey, but did give the opportunity to give some extra voice to not using technology in the classroom. Given the wording of the question, it’s unclear if this discussion of student-owned technology in classroom is within the course overall, or during exams when the Honor Code is in force– it was intended to mean pedagogical use of student-owned devices.
We’re pleased that, broadly speaking, the services that St Olaf IT offers enjoy such a positive perception from the faculty and we will continue to work hard to improve our technological resources and infrastructure and the community’s access to them.
Conclusions
One of the major takeaways from the responses is that more than anything else, the faculty want access to Instructional Technology staff for faculty development and project support. Instructional Technology should also place resources into Active Learning classroom development and continued support of and improvement to Moodle. Within DiSCO, there is less clarity in the responses, but workshops emerge as a clear winner in what faculty would like to see in the future in the space.
In the classrooms, some technologies would probably experience more widespread use if they were more available, particularly wireless laptop connections. Largely, however, responses from faculty about the technology they use in the classroom was characterized by what they’re not using– they’re not using VHS in the classroom any longer, but DVDs do get occasional (monthly) use. However, the use of Region 2 or 3 DVDs is not widely used throughout the College, but is more heavily used by the Humanities division. The tenor of the responses seems to suggest the faculty would like more ways to connect their laptops to classroom podiums and would make use of more classroom technology if it was available in a given classroom. If you’d like to see a particular technology in a given room, email classrooms@stolaf.edu and the Presentation Technology Manager can work with you to meet your needs.
Based on the survey responses overall, faculty at St Olaf College are comfortable with technology, use it in their teaching, and make extensive use of it in the daily lives. Google products are now in widespread use, and faculty are making use of social media and other technologies as well.
Moodle is probably the single most important service that Instructional Technology directly supports. Moodle may not be loved by the faculty, but, with the possible exception of the gradebook, it is also is not hated either. Indeed faculty generally don’t use many of Moodle’s features, either because they choose not to or because they simply don’t know that Moodle can support a particular task. As we’ve transitioned to new documentation and a new server model in the last year, we’re monitoring reaction to those changes carefully– iTech wants everyone’s Moodle experience to be a good one.
Finally, based on the responses we’ve received, it appears the the faculty agree that St Olaf student’s technology needs are being met by St Olaf IT, both in and out of class. While we’re pleased by this result, we are always eager to support the resources we currently have and expand our capabilities. If there’s something you’d like to see IT support, please don’t hesitate to let us know.
Thank you very much for your help in completing the 2016 Instructional Technology Survey. We really appreciate your feedback.