It’s no secret that the demand for online courses is on the rise, affecting the overall educational landscape. At the same time, it is also affecting the technology in the academic world. The large number of online courses is creating a learning experience that is open, accessible and participative. It’s helping fortify online students’ learning experiences.

When it comes to online assessments and exams, the technologies for invigilating distance students are relatively recent. One option that began in 2008 is called Live Online Proctoring (LOP).

While CUOL courses transpire in live classrooms, the online sections are in virtual space. Hence, every semester we find CUOL distance students having difficulty scheduling their exams because of remote locations or the absence of legitimate proctors. While we cannot gauge the exact number requiring online proctoring, every year at least 1,000 CUOL distance students may require this arrangement.

In fall 2013, CUOL cautiously piloted LOP and introduced the service to a few distance students, who have since completed 14 online exams under the proctors’ watchful eye. In short, the process was simple and the results were favourable; the exams were conducted online via cuLearn (ĐÓ°ÉÔ­´´â€™s LMS) and held at a specific time (concurrent with the local exam schedules), with proctors monitoring students live via webcam. Generally speaking, there was one proctor for every four students; each proctor simultaneously monitors an array of computer screens equipped with multimedia (i.e., sound, camera, etc.). Overall, the online proctoring service is similar to traditional proctoring service, except that live proctors are monitoring the test-takers from a distance.

Findings

How does one effectively safeguard the integrity of an online examination? The validity and the quality of online proctoring is every instructor’s concern. We’ve investigated different venues and validated LOP because of its equally effective method of proctoring students; it complies with our requirements and conforms to the university’s examination policies.

This model helps students by deterring cheating as they are monitored and recorded throughout the entire examination period. While it is difficult to know how many actually get away with cheating using this model, the question really is how many are caught. According to the National College Testing Association (NCTA), cheating is widespread and academic institutions must find ways to overcome these challenges. The keynote speaker in this conference confirmed that 68 per cent of students admitted to cheating (Detection and Prevention of Test Fraud: The State of the Art, NCTA Conference, Aug 3, 2012). It is our hope that LOP discourages cheaters.

Benefits of LOP

1. Replicates conventional invigilation: Students are acquainted with traditional proctoring. This model is meant to do for online learners what proctors do in traditional exam rooms.
2. Records the entirety of the invigilation: Proctoring is recorded instead of merely glancing at the student in the exam room, and the professor can review any incidents on video (by request).
3. Flexibility: Proctoring is open 24/7; it accommodates time differences and challenging schedules, thus helping to complete exams on time.
4. Last-minute proctoring arrangements: Although this is not encouraged, the company that we dealt with had accommodated one of our students 24 hours after our initial request for proctoring. That is faster than sending the exam via the courier service. The whole process from start to finish was completed within 48 hours.
5. Reasonable service charges: A typical proctor fee is $25 for a two-hour exam and $15 for each additional hour.
6. Immediate return of exams and grading: Prof. Ron Saunders (LAWS 2301 and 2302) commented favourably on the fast turnaround time for completing exams.
   

   “Ron Saunders has also ventured into online proctoring—a tool that has allowed him to get exams back from distance students instantaneously instead of having to wait for them to be mailed in from around the world. “It makes a huge difference,” he adds. Immediate exams mean much faster results for students, no matter where they took the test—which is especially advantageous for the popular Criminal Justice Systems.” CUOL newsletter: December 2013 issue.

7. Students are receptive to LOP: Below are quotes from two distance students who were proctored online. “Going into the exam with the online proctor, I won’t lie—I was very nervous,” says Alyssa Malette, a distance student studying in Berlin, Germany. “But the overall process was amazing. It was like the online proctor wasn’t even there, and was totally relaxed and comfortable. My proctor was super friendly, so that made the process a lot smoother.” “The online proctor process was neat,” says Jackie Kingsbury, who was a bit concerned prior to the exam because of her unfamiliarity with webcams. “[The proctor] Kim ‘took over’ my computer while I sat in my kitchen and I was able to see her moving the mouse and checking to make sure that I had all the technological requirements for the exam. I was able to see Kim and chat with her via webcam—I’m old and have never done this before—which was nice because I was able to put a face to the person who was supervising my exam.” CUOL Newsletter: December 2013 issue
8. Better proctor-student ratio: The 1:4 ratio is far better than 1:50 (conventional proctoring). With 1:4, proctors can closely monitor and correct suspicious behaviours as they happen.
9. Rigorous authentication process: Remote proctoring authenticates online students to make sure that enrolled students—not fraudulent individuals—are the ones sitting for exams. I’ve seen how proctors screen test-takers during the authentication process. They ask students to bring their IDs close to the camera and record them. This is more advantageous than the conventional approach because the video recording provides additional verification (their faces and IDs are recorded).
10. Proctors deal with the technical aspects of online exams.
11. Any questions/corrections are dealt with immediately. For instance, live proctors (although virtually at a distance) observe and can speak to students if need be.
12. This method is also suitable for online learners because they are already familiar with the online environment.

Problems We Encountered

1. If two or more internet users in a household use wireless, it is recommended for the test-taker to use a hard-wired internet connection. Two students had intermittent internet connections during their exams possibly due to their network (wireless or Wi-Fi). Here are a couple of transcripts from the proctoring company.
   

   Issue: Customer lost connection to the exam was offline for approx. 5 minutes, I tried calling the contact number for the student but it was the number to customer care for the school. The student called me after I was off the phone with customer service for ĐÓ°ÉÔ­´´ and informed me that she lost her internet connection and had to reboot her system. If rebooting the customer connected with me again and we were able to resume the exam.

   As I mentioned in my first blog post, the question arose of how the proctor addresses a power outage. Similarly, how does the proctor address an internet interruption? What is the proper protocol for handling this situation? Here is one such example.

   Issue: Got disconnected from Bomgar once during set up and Bomgar and MegeMeeting once during the exam. Was able to reconnect automatically in about a minute both times. Switched Bomgar and MegaMeeting to low bandwidth and continued with the set up/exam. Bandwidth was about 10 Mbps download when I ran the speed test, he was on wireless.

2. Using LOP means an increased workload i.e., digitizing the exams in cuLearn (ĐÓ°ÉÔ­´´â€™s LMS), setup time with the proctoring company, including registering students for the first time. We’ve taken on these extra tasks to get to know the processes, what’s involved, etc. Also, contracts, payments, forms, and other hidden tasks (calls from international students off-hours, technical aspects of cuLearn, etc.) have taken considerable time and effort in addition to dealing with the current workload.
3. Instructions have to be clear and precise. As this process is unfamiliar to a first-time user, we’ve spent considerable time with distance students and instructors, making sure that all parties involved understand the process.
4. Currently CUOL’s LOP is not scalable. It can only proctor live up to 40-50 students at the same time. For a large group of more than one hundred, the proctoring will have to be done over a period of two days.
5. One should be wary of selecting the right online proctoring institution. LOP is only as good as the quality of the proctors that drive it.

The Bottom Line

While academics continue to work hard transitioning to a digital world, more and more students are turning to online education, and they are continually seeking learning experiences that are accessible and participative. With this, the need for online proctoring will continue to intensify.

The key component for credible testing is the security – ensuring academic integrity and helping curb cheating. This was foremost in our minds when we launched LOP. Online proctoring is only as good as the quality of the proctors – effective live proctoring is still reliant on the individual proctors. With this in mind, we sought the right company with well-trained proctors, preserving the integrity of our online exams.

Based on our pilot project, LOP did deliver and preserve the integrity of our examinations. Of course, the issue of cheating will not be eradicated completely, however, it offered a viable solution for protecting this critical element of online proctoring. Therefore, I would say this service is a viable solution for proctoring our distance students, and is an essential component of online learning.

The advent of online learning in higher education is a key part of the 21st century learning experience. It is a period of change for many academic institutions, since access to education is now more competitive, open and global. The rise of MOOCs and the appeal of online courses offered [by the ranking academic institutions] to the global community are evident all over the realm of academia. In particular, the trend in which online learning seems to be heading impels the growth of various academic technology tools designed to assist in the learning and administration of the ‘global’ students.

Not surprisingly, a considerable proportion of students nowadays are more inclined toward taking online courses because of the expediency and the convenience that they offer. Many of them see this as effective solutions to meeting the demands of their personal, family and work commitments. This is where the need arises to integrate suitable technology that supports online learning and mitigates the ever-growing demands for online courses. Consequently, many institutions are trying to catch up to be in the forefront on this paradigm shift.

In my line of work, we deal with online students, and while there are good percentages that are on campus, many of them are distance learners. Proctoring their exams, particularly those that reside abroad or in remote areas of the country, has been challenging. Hence, maintaining the integrity of exams becomes even more essential and a primary concern for all three constituents: the university, the instructors and the administrators.

I’ll be talking more about all these three constituents later, but for now, my focus is on a technology that we are piloting this fall term. It is called Live Online Proctoring (LOP). This is a step forward to the current practice that we do for our ‘global’ students, the proctored secure-pdf exams.

Live Online Proctoring started in 2008, and since then almost 500 institutions in the U.S. and worldwide have implemented it. As far as I know, the post-secondary institutions in Canada in different stages of engaging with LOP include the University of Toronto, Memorial University, Royal Roads University and Wilfrid Laurier University, as well as a number of colleges. The feedback that I have received provided positive reviews and testimonials of the service.

We’ll be following suit with five different courses piloting this innovative proctoring service. International students will be proctored by a company based in Toronto and Atlanta. Remote-proctoring services rely on the same technology that has made it possible for people to earn college degrees without having to report to a campus.

How it works

Using his or her own computer and webcam, the student can take exams at home, at work, or anywhere they have internet access. LOP allows exam takers to complete their exam anywhere while still ensuring the integrity of the exam for the institution.

1. The student will connect with a live proctor from one of the online proctoring centres in Toronto via web cam. The proctor will help the student through the exam processes—they are there to help if technical difficulties arise.
2. The student then will connect his or her computer screen to the proctor. This allows the proctor to see the student’s screen and enables the proctor to assist with the setup before the start of the exam.
3. The proctor will ask the student to show two photo IDs, preferably one ĐÓ°ÉÔ­´´ ID. The proctor will then take a photo of student. Further authentication is necessary and the proctor will ask the student to answer a few questions. These questions are generated from the information we provide.

When the online exam begins, the proctor (ratio 1:4) watches the students throughout the entire duration of the exam and records any movements, sounds, keystrokes, etc. The proctor also has control of the student’s computer and can stop the exam if he or she notices any irregularities. An example from a Chronicle of Higher Education article on the subject mentioned tracking irregular eye movement— “one student attached a sticky note below the webcam, but a proctor caught him glancing up at the note and made him hold a mirror up to the screen. Busted.”

Understandably, there are concerns that LOP is insecure, that the integrity of the exam is compromised, and the authentication process is not trustworthy, factual or reliable. To dispel these concerns, I have further investigated and have considered these facts:

Pros

• A ratio of 1:4 as opposed to 1:50
• It is recorded instead of merely glancing at the student in the exam room
• The professor can review any incidents on video
• The proctor has control over the student’s computer. This is better than individual distance proctor
• Online exams are created through cuLearn, our own portal
• Proctoring of the proctors. Proctors are also monitored
• LOP seems more stringent than anything that takes place in a large lecture hall on test day. It takes the time and goes to these lengths to validate a test taker – not true in a room of 100+ students

Cons

• It’s invasive
• The proctors’ qualifications (I have requested documentation on the qualifications of their proctors)
• Authentication – a student could fool the proctor with fake ID
• There is the possibility that a proctor might try to collude with a test-taker to cheat, or jot down the content of a particular exam with the intention of selling it to future students taking the same course
• Proctors who work from home are liable to get distracted

Would this technology meet our criteria for proper invigilation? Reports from other institutions are positive, but we’ll have to be on the lookout. Appropriate invigilation requires finding the right balance among the constituents. The needs of the students, the [accessibility] requirements of legitimate proctors, and the university’s prescribed invigilation policies have to be met.

First, we need to look at the needs of the students. When the shipping cost of an exam exceeded the course fee, we were prompted to find better ways to invigilate our distance students. As a step forward, we have implemented proctored secure PDF exams [password protected]. This currently meets our standard for proctoring our ‘global’ students. We know the exam is closely guarded. At present, this is the best way to our knowledge to preserving the integrity of the exam for distance students.

Second, the concerns of our faculty members must be in the forefront. I will give a specific detailed example. At the CUOL’s AGM on Aug. 29, we proposed a pilot project on Live Online Proctoring (LOP). There were three questions that were asked, specifically:

1. How does the proctor address a power outage during an online exam? What is the proper protocol for handling this situation?
2. What’s the procedure if a student asks for a washroom break during an online exam?
3. If you allow a washroom break (which you really must, given how our bodies work), how do you guard against cheating?

Getting the answers to these questions provides some assurance to our faculty members (all answers are quoted from the exam proctor provider, except for one, as indicated).

1. A power outage for the student less than 5 minutes long is automatically reconnected to their proctor when they get their internet back. Beyond 5 minutes they will need to call and reschedule.In a power outage for the proctor, the student is automatically reassigned to another available proctor, and in most cases the student doesn’t even realize this. Once the original proctor is back up, the student is transferred back.
2. Have or conduct a 2-part exam; after completing the first part, have a washroom break and then resume by releasing the second part of the exam (suggested by an internal staff). You have to make the call on Bio-breaks. We enter a “yes” or “no” in our scheduler as to allowing it or not. It is specific to the exam, so if you have some profs who allow it and some that don’t, we can accommodate.
3. We are aligned on the policy for cheating. We send the professor an exception notification via email that we suspect a student is cheating. The professor can contact us for a copy of the video and make that determination themselves. This way the exam is completed and the decision is left in the hands of the professor after they review the video files. We do a room scan when they return. You will have to make the decision to allow or not allow breaks though. Also, if your testing center has a policy today on bio breaks, we should try to use that.

And finally, we have to be heedful at the set of rules and regulations the university has given us. Whatever solutions we come up with, they must conform to the university’s requirements and adhere to the policies that are put in place.

Because the infrastructure for LOP has greatly improved, the old biases against online proctoring are diminishing and the companies that offer remote-proctoring services are now facing acceptance from many universities. Many however still feel that proctoring should be entrusted to legitimate proctors (physically present) to preserve the integrity of the exams and should not be entrusted to online proctors who are often thousands of miles from the students. This is a legitimate concern. So, we’ll install a series of protocols aimed at making LOP as airtight as possible.

Early on in my blog I mentioned the importance of using appropriate technology in academics is paramount. For years now, since 2008, LOP has been extensively used by universities and colleges in the US and worldwide. Certainly, five or six years ago, there was a big debate about whether online proctoring was ready for academic institutions. We can continue to debate this, but today I want to share with you another indication that in terms of the future, and with the advent of MOOCs, that I think we will see LOP infrastructure unfold. And then many of our doubts and uncertainties will disappear. For now I am equally mindful that finding solutions to these concerns is important, as is ensuring that we always keep the important set of values we observed when we proctored our distance students.

Concerns, myths and uncertainties aside, we’re thrilled to be incorporating Live Online Proctoring into our CUOL curriculum this fall. For more information on the service or if you have any questions, please email me at nestor.querido@carleton.ca.