Improving Student Success by Being Automatically Personal

Abstract – This paper describes the development and use of “automatically-personal e-mail” routines allowing one to send interpretive e-mails to one’s class based on clicking a command in an Excel grade book. The macros are included in a template file which are available under the Attribution-NonCommercial-ShareAlike 4.0 International Creative Commons License. Nudges, in the form of light-touch directed-feedback have been shown to be effective in engaging students, but they are often time consuming for faculty. The author has found that he can send detailed performance updates to students automatically through macros in a well defined Excel Gradebook. This increases student engagement as they see it as a way of demonstrating caring. Using an Engineering Materials Course as an example, the author demonstrates the steps necessary to send 6 such nudges throughout an 11 week term. Sample commented coding and examples of the messages sent to students are provided as examples. Students are very satisfied with the auto-emails as shown. The paper closes with an analysis meant to help others implement this system in the minimal amount of time.

I. NEED

A. What is a Nudge

A “nudge,” is an intervention that encourages, but does not mandate, a certain behavior [1]. One form is “Light-touch, targeted feedback” sent to students via email can improve their perceptions of and performance in a class [2]. A professor sent personalized supportive e-mail to students to students who failed the first exam and found a small investment of time, signaled students that she cared and believes boosted recipients’ performance in the course [1]. Nudges are really demonstrations of caring which result in student engagement. Nudges are the answer to the question raised by another author who asks how do we engage the significant fraction of students who struggle and are often forgotten [3]. Reaching out can be complicated and time-consuming for the professor; the person most effective at nudging students to ask for help. In the author’s opinion all students should be nudged frequently throughout the term. This requires responding to another author’s question: What if engagement wasn’t complicated and didn’t take that much time?2. Automatically personal e-mails can be effective light-touch targeted-feedback nudges which engage students.

Student engagement in response to nudges builds resiliency. Currently, higher education institutions are facing a crisis with declining enrollment and student attrition. … Resiliency is one factor which will impact a student’s decision to stay or go [4]. Dictionary.com’s definition of resiliency is: ability to recover readily from adversity defined as an unfortunate event. The unfortunate event can be a lower than expected grade or expectation thereof. We can help students recover, by helping students be resilient, through increased engagement initiated by light-touch targeted-feedback nudges. These nudges designed to help students respond to difficulties in the classroom is most effective when specific feedback has been provided to the student by the professor so that key issues can be pinpointed and addressed [4]. I have been automatically personal with students for years. I began (early 90’s) with handwritten notes on each of 3 tests, moved to fill-in-the blanks on print, then to VBA which allowed for six nudges per term. Each message is feedback and makes it easy for the student or others helping the student pinpoint issues.

B. How this Nudge Works

In one example a professor started sending personalized emails (in which the body of each email was the same, but they were addressed to each student individually) to students who received a failing score on the first of three exams—approximately 10 percent of a class of 200. The email, 1) pointed out that the student didn’t do as well as expected on the exam, however, it was still early in the semester, and that changing habits now could turn their grade around. 2) asked the student whether they knew why they hadn’t performed well. 3) gave some suggestions and 4) finished by reiterating the fact that together we could improve their standing in the class [3]. Committing to doing the same is committing to a complicated time-consuming process. The automatically personal e-mails sent out to the entire class included (1), (3) and (4) and were not time-consuming. The Excel gradebook communicated with Outlook to nudge students through e-mail.

C. Results of a Nudge

The only negative feedback from students in the articles cited came when a message was sent out in error. “I was wondering why I was sent this message,” [a welcome-style email] one student wrote to a participating art instructor. “I believe that I have been coming to class regularly as well as taking notes actively. Thank you for your time.”(Kurlaender and Carrell)2. Linking the feedback to an Excel Workbook could have solved this problem. Also it would have been a chance to praise the student. Initial praise is as important as reaching out to those who are struggling.

Student response to nudges were usually positive. “No professor has ever cared about me.” was a sentence in the reply sent by a struggling student. This shows that by identifying struggling students and sending them personalized emails encouraging action and providing support, educators can make a significant difference to the success of their learners. Sent 20 e-mails and gave 5 responses from students. All thanked her. 2 specifically cited appreciation for caring compared to other professors. 3 promised to reflect and contact her 1 of these 3 asked for a response3. A number of professors said they were surprised by their students’ gratitude at their gestures. For example “I attend every class, go to the review sessions, and have turned in the extra credit so I am defiantly [sic] trying to do well but I am still struggling, I will come to office hours and try to meet up with our TA as well. Let me know if there is anything else I can do. Thank you!” (Source: Kurlaender and Carrell) [2]. Another professor said, more than half of the 20 students she emailed wrote back expressing their appreciation for her message and taking responsibility for their grades1. This is why early nudges are important.

Some faculty were neutral regarding nudges.

Kurlaender and her co-author on the project, Scott Carrell, professor of economics and co-faculty director of the California Education Laboratory at Davis, wanted to see what would happen if professors reached out to students individually via email just a few times a term, with the goal of promoting their sense of self-efficacy and help-seeking behavior. Over all, they found strong evidence that this “”light-touch, targeted feedback”” can positively affect student perceptions about a course and instructor. After the first email “nudge” about homework, students in the Davis study increased the time they spent on homework. After a second nudge about performance on their first course exam, students scored significantly higher on a second exam, compared to students in a control group who did not receive the emails. exception: There was a significant positive effect on overall course performance for students with fewer than 30 previous college credits. There was no significant positive effect on course withdrawals [2].

I did 3 nudges similar to their 2 and noted similar increases in course performance (HWs and Tests). In some terms I noticed an increase in withdraws when students were informed once 26.5% of their grade was established.

Some faculty responded negatively regarding the use of nudges. There were a series of negative comments regarding this article [2].

1. This sounds like a ton of work for something described as “”light-touch””.
2. All this to produce “”no significant positive effect on overall course performance and I was disappointed to see the tepid results the study got on class performance.
3. No significant positive effect on course withdrawals,
4. Having faculty members provide some simple, individualized feedback to students as an intervention.” Done! I like to call them exam scores or quiz scores – it is individualized with notations about student errors and misconceptions and is provided early in the semester so that students might alter their trajectory.
5. Doing these nudges one at a time is likely impractical. For example, one faculty member commented they would need to send 564 emails per term based on a 4-4 schedule, an average of 47 students per class, and 3 emails per semester. I’d end up sending dozens of different varieties of emails to those students. The first, “”welcome”” email would require me to look at attendance for the first couple of weeks. Then I’d send either generic “”welcome”” or a “”welcome, but why aren’t I seeing you regularly”” email. That would be pretty easy. But, the second and third would require a complete analysis of attendance/exam/quiz/homework performance to be able to send the appropriate email. I could write an Excel process to automate this analysis and, if research can demonstrate a clear, substantial, significant benefit to a “”light touch”” then maybe I’d spend the time doing it. But not until

There is no doubt that well graded and corrected tests are a form of individualized feedback, however just providing such feedback without improvement suggestions should not be expected to help students alter their trajectory. It is clear that this type of feedback is most needed in large courses, and therefore having the instructor sit next to a computer with a bunch of grades and attendance records deciding which students get a bcc is probably not worthwhile.

Other faculty had positive reactions regarding nudges. One person said: This is an excellent idea. Without an email from a professor a student would have no way of knowing that they’re doing badly on assignments. One respondent thought it was sarcasm and another claimed that because faculty post grades to an LMS and return work with a grade to students they know how they are doing in the course [2]. I hope this is not sarcasm because there is no reason to leave the grade interpretation up to the student. From my experience LMSs do not provide interpretation, they simply supply a set of numbers.

Nudging each student throughout the term will not be practical unless automated. I did not find anything in the literature regarding automatic light-touch targeted-feedback nudges which engage students.

II. APPROACH

I created an automated system whereby once grades are entered in an Excel Workbook all students can be e-mailed at the touch of a button. I found the code shown in Text Box 1 through a Google search. This common code can be copied from macro to macro as one creates more automatically personal messages. The main research question to be answered is: what is
needed to make this method successful?

Text Box 1: Generic Code to Send e-mail from Excel

A. ID Points in Term

It is most likely both impractical and not beneficial to send an interpretive message after every grade has been recorded. My goal is to be most effective at improving student learning without generating what I describe as stock market watching pressure. Therefore the nudge points need to be based on the course structure. Consider the following course which meets on Mondays and Thursdays each week. There were 8 HW’s 10% (1-6 1% ea, 7 and 8 2% each) due Monday and 8 Tests 60% (7.5% each) given on the following Thursday. I chose to give 6 nudges.

1&2) Because HW is key to successful completion of the course, I felt it important to give interpretive feedback after HW 1 and HW 2.

3) After the students have had 3 HW assignments and 3 tests I send out the MidTerm 1 update. At this point in time I can relate HW and Test performance as well as ensuring that students are given an indication of their improvement by the time official midterm grades are posted.

4&5) Rather than merely posting an official midterm grade I send out an interpretive message the Midterm 2 update. Both the MT2 and pre-final updates include a detailed description of what students can accomplish.

6) The final notice is sent to students prior to the final grade being posted. I thank them for being part of the course, give them information so they can improve in future courses, and offer my help in the future. Also, if there is a mistake I can correct it before posting final grades and having to go through a grade change.

B. HW 1 Samples and Student Responses

Thank you for turning in an excellent HW Assignment. I appreciate all your hard work. Turning in assignments such as this will help ensure your success in this course. Thanks again and let me know how I can help you continue to excel

Often a thank you for caring.

Thank you for turning in a good HW Assignment. I appreciate all your hard work. The difference between a Good HW and an Excellent HW is usually a minor misunderstanding of the criteria. Please review them and contact me if you have any questions. Thanks again, let me know how I can help you continue to excel, and I look forward to correcting your second HW assignment. Remember Turning in assignments such as this will help ensure your success in this course.

I wanted to seriously thank you for going over the work in detail (notes, this bit of mail, etc), especially when I turned it in under an unacceptable format. I really hope that I am able to improve my test scores so I am not so worried about my grade and so I can really take the class for all that I can. I really enjoy working in my shop and everything that we have touched on so far is and will be directly applicable to my job. Also, thanks for taking the time to listen to me. I forget how much it can help to sit down with someone and go over questions and concerns. Although it might not have seemed like much, but it kind of gave me the chance to step back and look at my situation. Don’t be surprised if I show up early for lab more often, it seemed to help me this time and I will take whatever chance I can to improve myself. Have a good evening and see you tomorrow.

I noticed that the first HW you turned in was marginal. This concerns me because completing HW assignments is necessary to succeed in this course. Please review the first HW assignment and the grading criteria. Contact me if you have any questions, I realize that it might be getting used to the requirements. I want to help you succeed in this course. Thanks again and I look forward to grading your second HW assignment.

Thank you for your concern. I didn’t realize that when you asked for the memorandum that you wanted the whole lab report. I fixed that after the first homework. We have looked at your notes regarding our first homework assignment. Our team was wondering if there would be any way for us to redo the first homework assignment and receive partial credit back. If this is not possible, could we have you review homework assignment two to check for correct formatting? Thank you for the heads-up about the homework. I will certainly use your feedback in editing this week’s homework/lab report. Was there anything outside of the criteria sheet and commented homework that was of particular concern?

After reviewing the grades for the first HW assignment I noticed that the assignment you submitted was unacceptable. I am very concerned about this. Succeeding in this course requires that you submit acceptable assignments. Please let me know how I can help you succeed in this course. I look forward to: hearing from you, working with you and correcting your second HW assignment.

Because this is a group homework assignment, and all future assignments will also be done in this group we would all like to meet with you at the same time. Could we possibly meet with you after class this afternoon?

I noticed that you did not turn in the first HW assignment. I am very concerned about this because submitting acceptable assignments is key to your success in this course. Please contact me so that we can work together to ensure you complete the course successfully. I look forward to hearing from you, working with you and grading your second HW assignment.

Can we talk after class today and figure out what time works best to sit down and discuss your course.

C. HW 2 Sample

I felt it very important to follow up the first HW message with a second comparing the two. This allows me to congratulate students for progress made, nip possible decreases in performance in the bud, praise students with consistently high performance, and warn those whose HW performance is consistently less than acceptable. There are 11 possible messages based upon the combination of the 2 HW grades. These are based on 1 of 5 sublines, 1 of 11 gradelines and 1 of 5 interplines. The details of the gradeline(1- 11) and interpline(1-5) are in the code. A sample of code used for the HW 2 update is shown in Text Box 2.

Text Box 2: Case-Select Structure Sample for HW 2 Update

An example message (see Text Box 1) is below. emailmessage = salutation + Chr(10) + gradestate + ” ” + gradeline + ” ” + interpline + Chr(10) + Chr(10) + sigline

The sigline is standard. The salutation is based on the student’s name as shown in the code in Text Box 3. The gradestate, gradeline, and interpline are determined for each student from the data in the Excel Workbook. Chr(10) is needed for a hard return and “ ” is required for spaces

This is a sample message.
Dear WhoEver:

Text Box 4: Sample MidTerm 1 Message

Your first two homework grades are, for HW1: Unacceptable and for HW2: Marginal. Your performance on the HW has improved, thank you for the increased effort. However, it is very important that you turn in acceptable (Good or Excellent) HW’s Otherwise you will have difficulty successfully completing this course. Please contact me so I can help you continue to improve and thus succeed. I look forward to working with you.

D. MidTerm 1

MT 1 is sent out after I have 3 tests and 3 HW to relate. This also allows me to give students an indication of their improvement by the official midterm notification. An example message is shown I Text Box 4.

In order to determine the HW Message one must count the number of acceptable, barely acceptable, unacceptable, and skipped HW assignments as shown in Figure 1 (Excel).

Figure 1: Determining HW Status After 3 HW’s and 3 Tests for MidTerm 1 Update

E. MidTerm2 and Advance

Text Box 5 is a sample message sent at the same time the official midterm grade is sent to the registrar.

It is possible that a line of the Advance (pre-final email) might include the following. To earn a D in this course your score on the final exam will need to be at least -15. of 300 points. Students like knowing that they are guaranteed a grade higher than D or in some cases C.

Text Box 5: MidTerm 2 Message

F. Notice

A sample final notice message follows.

I wanted to let you know that your score on the final examination was 174, which corresponds to a D. As a result your course grade is: C+. You showed major improvement throughout the term, but your performance significantly declined on the final examination. Please review what you did during the term so that you can repeat it in future courses. Please assess how you prepared for the final examination, so that you can learn to use final exams to your advantage. Thank you for being part of  this  class.  If  I  can  help  you  implement the suggestions I made, or in any other way in the future, please let me know.

III. NET BENEFITS

A. Tipping Points

One must schedule time at tipping points. In the courses I teach all HW was submitted, graded, and corrected electronically. Thus HW can be returned outside of class time. Further HW was not graded based on correctness but on how easy it was for me to correct. Just before the HW is due I post narrative solutions. Consider the course shown earlier which has class-sessions on MTh and labs TW. HW is due each Monday. Thus if HW 1 is due on 2^nd Monday, it must be returned by 2^nd Friday.

Each HW takes approximately 10 minutes to grade and more importantly correct. If HW is assigned to 3 person teams and there are 48 students in a given class section this can be done in just under 3 hours. Note once the grades are entered in an Excel Workbook e-mails to all 48 students are sent out in less than a minute. My experience is that after sending out these e-mails one will get between 16 and 32 e- mails from students. as shown in Table (Messages v Responses) reading and responding will take between 1 and 2 hours. These 1 or 2 hours are rewarding as the 2-3 hrs time spent correcting not simply grading HW. This is not a sarcastic comment, I found the level of dialog to be intellectually rewarding. Scheduling time at tipping points is easy to do using a calendaring system. I suggest doing it at the beginning of the term. For example 2^nd Friday Priority Item – Grade HW 1, send out AutoPersonal Update 1/6, and work with students.

B. Pedagogical Survey

Students appreciate the automatically personal e-mails. I developed a pedagogical survey which was based on an ascending survey. Students were asked the following question.

To what extent did you find the automatically personal updates [HW 1, HW 2, Test 3 (MT 1), Test 5(MT 2) and Advance Grade] valuable?

• The content of the automatically personal updates were useless at best. They may have hampered my ability to successfully complete the course. I do not even know if I will read the automatically personal final grade notification. (Unacceptable)
• The content of the automatically personal updates were somewhat useful. However, they had essentially no impact on my being able to successfully complete the course. The automatically personal final grade notification will be nice to receive. (Marginal)
• The content of the automatically personal updates were useful. I appreciated them, and they had some impact on my being able to successfully complete the course. The automatically personal final grade notification will be nice to receive. (Acceptable)
• The content of the automatically personal updates were useful. I appreciated them. They positively impacted my being able to successfully complete the course. I look forward to the automatically personal final grade notification. (Good)
• The content of the automatically personal updates were useful. I appreciated them. They had a major impact in my being able to successfully complete the course. I look forward to the automatically personal final grade notification. (Outstanding)

Figure 2: Student Satisfaction with Auto-Personal e-mails as a Pedagogical Technique. GB = Good or Better (good or outstanding), LTA = Less than Acceptable (unacceptable or marginal)

The student responses are shown in Figure 2. The criterion for each overall rating is indicated on the graph. The rating of acceptable for W15 and U16 were very close to good. In one case the %LTA was just above 25% and in the other the average was 2.6 instead of 2.7. The low ratings in S14 and F15 were do to very heavy teaching loads and hospitalizations which delayed grading. Students like the auto-personal e-mails, they expect them and are justifiably upset when they are not delivered. They cannot be delivered if one falls behind in grading.

C. Analysis

• In about half of the terms mentioned in this paper there was an increase in W’s. This usually occurred after the third update. Students chose to give up even though 74.5% of their grade was undecided. I have every reason to believe that the % of W’s would decrease if the early performance updates were eliminated. Obviously, that is not an acceptable fix and should call attention to the use of DFW% as an inaccurate performance metric. In the future these e-mails will include what students need to do to achieve certain letter grades.
• There is a chance that students when learning that very little effort is required for a D settle. This course was taken as the last sole course by a significant number of students. A student with a GPA of 3.052 or higher is guaranteed a 3.0 average if they pass the one course. 3.565 guarantees a 3.5 average.(Based on 40 courses and this being last remaining).
• The coding can be done in Word as it is based on a lot of repetition.
• The set up of the Excel Workbook is based on a lot of copy/paste.
• The time spent interacting with students usually electronically after these updates are sent is extremely rewarding. Scheduling such times in advance, around when students are expected to respond is key as is scheduling time to grade when the checkpoint is due.
• Once the macros are written, selecting a menu item after recording grades is not a lot of work and every student gets an automatically personal e-mail.
• It is hoped that the macros, and workbook are a template that will minimize the development time for others.

D. Conclusions

• It is possible to effectively nudge students frequently throughout the term if such a system is automatically connected to Outlook via Excel.
• It is hoped that others will adapt this system for use in their courses to improve student engagement through nudges and report their findings. Topics for future research include but are not limited to the content of nudges, the tipping points selected, and the frequency of nudges.
• Those not using MS Office will have difficulty implementing this system. While the code has been made as self-explanatory as possible some familiarity with Excel functions and VBA programming is needed to adapt this system.
• A more in depth study correlating specific student comments to specific messages was not possible due to IRB concerns. The same is true regarding performance improvement as the result of these nudges.

A movie showing customization is available on markapalmertechonline.com.

The following web page has the code for all macros in Word and a sample Excel Workbook. These are available under the Attribution-NonCommercial-ShareAlike 4.0 International Creative Commons License

REFERENCES

[1] Zoë Cohen, Becky Supiano: How One Email From You Could Help Students Succeed; The Chronicle of Higher Education – Teaching Newsletter, 8/9/2018

[2] Colleen Flaherty: My Professor Cares; Inside Higher Ed, 1/14/2019

[3] Zoë Cohen: Small Changes, Large Rewards: How Individualized Emails Increase Classroom Performance, 7/17/2018

[4] Susan Ohrlabo: 3 Ways to Assess and Build Student Resiliency; Academic Impressions, 12/15/2014