PhD Qualifying Exam Issues

[sambaprof]

Our Department offers PhD degrees in different tracks in PhD ranging from Computer Science to Computer Engineering to Electrical Engineering.
The department conducts PhD qualifying exams at the beginning of the semester and the results are given 2 weeks after the exam. The results for the Fall 2022 qualifying exam was given last week.

I am relatively new to the University and this is the first time the PhD students that I am advising/supervising are taking the qualifying exam.

Also I  have no say on the exam design or delivery.  'Computer Science' area PhD qualifying exam is handled by the chair of 'graduate program committee for Computer Science' and the 'Computer Engineering/Electrical Engineering' area PhD qualifying exam is handled by the 'chair of graduate program committee for Computer Engineering/Electrical Engineering'.

I am in Computer Science area and couple of my PhD students got failing grades in the Qualifying exam subjects. One student failed in all three subjects and another student failed in two out of three subjects.


It appears that a total of 8 students (including 2 students I am advising/supervising)  appeared for qualifying exam in Computer Science area this Fall 2022 and none of the students passed in all three subject. Whereas all three students who appeared in Electrical/Computer Engineering this Fall 2022 has passed.

These qualifying exams are supposed to be testing the student's knowledge and skills at the undergraduate level for any given subject.
The student's mention that they prepared hard and yet failed compared to the students in Electrical Engineering and Computer Engineering  area.

Following are the list of issues as mentioned by the students.

1. The exam contained 8 questions (in subject 1) with three different sub-questions for each question, 18 questions with some sub-questions
    (for subject 2), and 7 questions( for subject 3))  which they say is quite impractical to complete in 6 hours. (Less Time)
2. Students are wondering why no student passed in all 3 courses? (Difficulty level not according to syllabus that the students were told to prepare).
    The student says he can demonstrate it with questions on paper and preparation books offered on the undergraduate syllabus mentioned in the
    department website.
3. Student says he has passed the "subject 1" graduate course  with "A" grade and "subject 2"  graduate course with 'B+' grade.  However, he wonders
    how he could fail the undergrad course level course for qualifying exam purposes.
4. Students were not yet provided the opportunity for the  review of the responses paper, so that they can understand  what was the issue in their exam
    and how can they correct it in the future.
5. The students say that they agree could  not completing the "Subject 3" but they would have passed that exam if they would have got enough time.
6. The students say they can prove their knowledge by taking an oral examination like some other borderline candidates got the option and wondering if
    they can get that  option...
7. The students claim that they were given challenging questions at the graduate level course level. However, they were told by the 'chair of graduate
    program committee for Computer Engineering/Electrical Engineering'  who was in charge of delivering the exam, the exam will be according to the .
    undergraduate syllabus from the department website.
8. The students were wondering how come the "Computer Engineering/Electrical Engineering" got time from 9:30am-5:00pm with the "Subject 1"
    from 9.30 to 11.30 am, "Subject 2" from "12.30 pm to 2.30 pm" and "Subject 3" from "3.00 pm to 5.00 pm", as published accordingly with the timings
    published in the Department website. Whereas "Computer Science" area got straight timings from 10 am to 4 pm  for all three subjects.
9. The Department provided lunch to the students. Since the 'Computer Engineering/Electrical Engineering' area gave specific lunch break, those students
     enjoyed their lunch break.  The 'Computer Science' students were also provided the lunch but the lunch break was not provided (as the students had
     only 6 hours to complete the exam and if they had to eat they will  lose time to complete the exam.). So the students I am advising did not took the
     lunch and yet they did not had time to complete the exam.
10. Students have only two tries. If they could not pass they have to leave the program.
11. It looks like the Fairness of these exams are also questionable. For example, for a student in the 'Electrical Engineering/Computer Engineering' area,
     it looks like one of the subject's exam is set by the student's advisor itself and my students claim that the other student prepared only for last 10 days
     (less than 30% effort of my students) and yet passed.

I would like to graduate quality students myself but the issues raised by the students would also need to be heard and addressed. I believe the students are doing reasonably well. In fact one of the student's hard work contributed lot to an NSF proposal with the preliminary results etc., which helped me secure about $600K from a recent  NSF award as Sole PI.

Our university is an R2 university. The university and the department is not highly ranked. So, it is a real challenge to recruit graduate students.

I am investing lot of time to recruit, advise and work with the students. I am also worried about the morale of the existing students and recruiting new students. I  also need to graduate at least one or two PhDs successfully , if I want to apply for the "Full Professor".

While 'Electrical Engineering and Computer Engineering'  area graduate program committee chair is advising/supervising PhD students,  'Computer Science'  area graduate program committee chair is NOT advising/supervising any PhD students. I am wondering if this also contributes to this issue.

Having said this, I am also not sure if this is an issue with student preparation. I am also worried not to get into any sort of conflict with the chair of 'graduate program committee for Computer Science' since he handles the department admission decision as well. More importantly it looks like these students will have only one more attempt to clear the qualifying exam.

The PhD program is at College level and so I am planning to meet with the PhD program director, who is from a different department. I am actually going up for tenure this year and I just submitted the dossier last month.   

Please advise how I should handle this.

[Caracal]

Our Department offers PhD degrees in different tracks in PhD ranging from Computer Science to Computer Engineering to Electrical Engineering.
The department conducts PhD qualifying exams at the beginning of the semester and the results are given 2 weeks after the exam. The results for the Fall 2022 qualifying exam was given last week.

I am relatively new to the University and this is the first time the PhD students that I am advising/supervising are taking the qualifying exam.

Also I  have no say on the exam design or delivery.  'Computer Science' area PhD qualifying exam is handled by the chair of 'graduate program committee for Computer Science' and the 'Computer Engineering/Electrical Engineering' area PhD qualifying exam is handled by the 'chair of graduate program committee for Computer Engineering/Electrical Engineering'.

I am in Computer Science area and couple of my PhD students got failing grades in the Qualifying exam subjects. One student failed in all three subjects and another student failed in two out of three subjects.


It appears that a total of 8 students (including 2 students I am advising/supervising)  appeared for qualifying exam in Computer Science area this Fall 2022 and none of the students passed in all three subject. Whereas all three students who appeared in Electrical/Computer Engineering this Fall 2022 has passed.

These qualifying exams are supposed to be testing the student's knowledge and skills at the undergraduate level for any given subject.
The student's mention that they prepared hard and yet failed compared to the students in Electrical Engineering and Computer Engineering  area.

Following are the list of issues as mentioned by the students.

1. The exam contained 8 questions (in subject 1) with three different sub-questions for each question, 18 questions with some sub-questions
    (for subject 2), and 7 questions( for subject 3))  which they say is quite impractical to complete in 6 hours. (Less Time)
2. Students are wondering why no student passed in all 3 courses? (Difficulty level not according to syllabus that the students were told to prepare).
    The student says he can demonstrate it with questions on paper and preparation books offered on the undergraduate syllabus mentioned in the
    department website.
3. Student says he has passed the "subject 1" graduate course  with "A" grade and "subject 2"  graduate course with 'B+' grade.  However, he wonders
    how he could fail the undergrad course level course for qualifying exam purposes.
4. Students were not yet provided the opportunity for the  review of the responses paper, so that they can understand  what was the issue in their exam
    and how can they correct it in the future.
5. The students say that they agree could  not completing the "Subject 3" but they would have passed that exam if they would have got enough time.
6. The students say they can prove their knowledge by taking an oral examination like some other borderline candidates got the option and wondering if
    they can get that  option...
7. The students claim that they were given challenging questions at the graduate level course level. However, they were told by the 'chair of graduate
    program committee for Computer Engineering/Electrical Engineering'  who was in charge of delivering the exam, the exam will be according to the .
    undergraduate syllabus from the department website.
8. The students were wondering how come the "Computer Engineering/Electrical Engineering" got time from 9:30am-5:00pm with the "Subject 1"
    from 9.30 to 11.30 am, "Subject 2" from "12.30 pm to 2.30 pm" and "Subject 3" from "3.00 pm to 5.00 pm", as published accordingly with the timings
    published in the Department website. Whereas "Computer Science" area got straight timings from 10 am to 4 pm  for all three subjects.
9. The Department provided lunch to the students. Since the 'Computer Engineering/Electrical Engineering' area gave specific lunch break, those students
     enjoyed their lunch break.  The 'Computer Science' students were also provided the lunch but the lunch break was not provided (as the students had
     only 6 hours to complete the exam and if they had to eat they will  lose time to complete the exam.). So the students I am advising did not took the
     lunch and yet they did not had time to complete the exam.
10. Students have only two tries. If they could not pass they have to leave the program.
11. It looks like the Fairness of these exams are also questionable. For example, for a student in the 'Electrical Engineering/Computer Engineering' area,
     it looks like one of the subject's exam is set by the student's advisor itself and my students claim that the other student prepared only for last 10 days
     (less than 30% effort of my students) and yet passed.

I would like to graduate quality students myself but the issues raised by the students would also need to be heard and addressed. I believe the students are doing reasonably well. In fact one of the student's hard work contributed lot to an NSF proposal with the preliminary results etc., which helped me secure about $600K from a recent  NSF award as Sole PI.

Our university is an R2 university. The university and the department is not highly ranked. So, it is a real challenge to recruit graduate students.

I am investing lot of time to recruit, advise and work with the students. I am also worried about the morale of the existing students and recruiting new students. I  also need to graduate at least one or two PhDs successfully , if I want to apply for the "Full Professor".

While 'Electrical Engineering and Computer Engineering'  area graduate program committee chair is advising/supervising PhD students,  'Computer Science'  area graduate program committee chair is NOT advising/supervising any PhD students. I am wondering if this also contributes to this issue.

Having said this, I am also not sure if this is an issue with student preparation. I am also worried not to get into any sort of conflict with the chair of 'graduate program committee for Computer Science' since he handles the department admission decision as well. More importantly it looks like these students will have only one more attempt to clear the qualifying exam.

The PhD program is at College level and so I am planning to meet with the PhD program director, who is from a different department. I am actually going up for tenure this year and I just submitted the dossier last month.   

Please advise how I should handle this.

I think I would first want to get a handle on how typical that 0 for 8 passing number on all the exams is (and probably you would want to break it down by each individual subject.) "Students are upset because they failed the exam" is not likely to convince anyone, but if that's a really high failure rate, you'd like to think a director of grad studies would want want to take a look. It shouldn't be that hard to see if the exam was dramatically different in difficulty and format from previous years.

[Parasaurolophus]

+1 to Caracal: that looks like a surprisingly high failure rate, so you'll want a sense of how typical that is.  Also, have you yourself had a chance to review the exam questions? Do they seem appropriately pitched to you?


My PhD program had some serious problems with its logic requirement, including students with no prior experience being thrown into an advanced logic course with no additional support or time, previous answers circulating freely, and some advisors just telling their students to cheat. I was the only student in my year to pass (all the others... were caught cheating!), and that kicked off a huge departmental fight, which went as high up as the Dean. It was a clusterfuck. Recommendations were tabled, ignored by the relevant instructors, and as a result the entire class two years below me rebelled and outright refused to take the class or test out of requirement.

It was an ugly, ugly fight. The students ultimately prevailed, and the requirement now makes a lot more sense than it did. But you don't want to kick something like that off.

[fizzycist]

Can you be more specific about what you want advice on here? The students aren't being kicked out, they get another chance at this exam, correct? So are you hoping to be involved in the exam writing/grading process to make sure it is fair? Or are you just trying to get your students past the hurdle regardless of whether they pass the exam or not?

I once fought for a student who kept failing our equivalent of the exams you describe. I can't decide if I regret it or not. It was a long and bumpy road with the student. My life would have been easier if I'd just let him go, but I'm not one to optimize for ease in life haha. I find it oddly rewarding to help (drag?) the super difficult students across the finish line. But it is a huge amount of thankless work, not something I would advise others to do.

[Caracal]

Can you be more specific about what you want advice on here? The students aren't being kicked out, they get another chance at this exam, correct? So are you hoping to be involved in the exam writing/grading process to make sure it is fair? Or are you just trying to get your students past the hurdle regardless of whether they pass the exam or not?

I once fought for a student who kept failing our equivalent of the exams you describe. I can't decide if I regret it or not. It was a long and bumpy road with the student. My life would have been easier if I'd just let him go, but I'm not one to optimize for ease in life haha. I find it oddly rewarding to help (drag?) the super difficult students across the finish line. But it is a huge amount of thankless work, not something I would advise others to do.

A lot is going to depend on whether these exams are designed to winnow people out of the program. Students spent huge amounts of time studying for exams in my grad program, but it was quite rare for anyone to fail them. If someone wasn't going to be able to continue in the program, they were usually told that way before they took the exams. If that's the norm in your program, I would expect that the next exam will be much easier, even without your intervention.

The more dangerous situation is if its normal for a decent number of students to fail and be kicked out of the program, and everyone just assumes this is just an uncommonly crummy bunch when there the exam was faulty.

[arcturus]

From this and your other recent post (https://thefora.org/index.php?topic=3103.0) it sounds like you are new to this university and relatively inexperienced in advising PhD students in this department (even though you state that you are applying for tenure/promotion this year). In those circumstances, I recommend being careful to not "rock the boat" before learning more about the past history of qualifying exams from a trusted colleague. For example, the "no lunch break" aspect might have been implemented *at the request of current/previous students* insofar as it shortens the total length of time they are under pressure. Furthermore, by having it as a single time block, the students may have more flexibility regarding how they use their time on the different sections of the exam, giving them an advantage to complete the easy parts quickly and spend more time on sections they found difficult. Thus, the differences in how the exam is administered may not be an actual disadvantage for the students, even if they are pointing to these differences as an excuse for their performance.

Action items:

(1) Ask a trusted colleague about the history of qualifying exams in the department: their structure (why 3 sections?), the typical pass rate, the apparent differences in how the exams are administered (lunch break, etc), the content, etc.

(2) Look at the current and previous exams (I assume students have access to past exams to use as practice, so you should be able to see them as well). If you think they are at too high of a level, then you can work with the faculty committee that writes them to improve this aspect.

(3) Ask a trusted colleague whether it is typical for students on the borderline to be able to fulfill the requirement through an oral examination. If so, who administers the oral exam? Does the oral exam constitute the "second attempt", or will students still have that option if they do not pass the oral?

(4) If you do not think that the current qualifying exam structure is beneficial to the education of your students, you can start to work toward changing the exam using the committee structures in place in your department. This is a long term process (probably years), so it is beneficial to test the waters first before committing yourself to the concept of pushing for whole-scale change.

Finally, use your political capital wisely. An advisor should be looking out for the best interests of their students, but sometimes students are best served by failing out of the program. I have witnessed many examples of faculty pushing to get their students past some hurdle, like poor performance on a qualifying exam, only to later realize that these students really are not capable of completing the degree. So, yes, you should investigate what has happened to your students here, but be prepared to discover that your best action is to let the regular department policies/procedures dictate the next steps.

[sambaprof]

Digged deeper on the status of the qualifying exam by talking with other advisors and got the breakdown for the following students who took qualifying exam in Computer Science area

Student 1 -- First Attempt -- Failed All Three subjects
Student 2 -- First Attempt -- Passed One subject and Failed Two subjects 
Student 3 -- First Attempt -- Passed One Subject, Failed One Subject, Borderline in One subject -- has to take Oral exam in 2 weeks
Student 4 -- First Attempt -- Passed All Three Subjects
Student 5 -- First Attempt -- Passed in First subject, Failed in second subject , Borderline in the third subject -- has to take Oral exam in 2 weeks
Student 6 -- First attempt -- Passed One Subject and Failed Two Subjects
Student 7 -- Second Attempt -- Passed One subject (and Passed Other Two in the earlier attempts)
Student 8 -- Second attempt -- One subject Borderline -- has to take oral exam in 2 weeks; Passed other two subjects in earlier attempts

[traductio]

My PhD program had some serious problems with its logic requirement, including students with no prior experience being thrown into an advanced logic course with no additional support or time, previous answers circulating freely, and some advisors just telling their students to cheat. I was the only student in my year to pass (all the others... were caught cheating!), and that kicked off a huge departmental fight, which went as high up as the Dean. It was a clusterfuck. Recommendations were tabled, ignored by the relevant instructors, and as a result the entire class two years below me rebelled and outright refused to take the class or test out of requirement.

It was an ugly, ugly fight. The students ultimately prevailed, and the requirement now makes a lot more sense than it did. But you don't want to kick something like that off.

The PhD exams we have in my current department are ill-considered, but holy crap -- nothing like that. If I'm not mistaken, you went to a well respected school, too, right? What a mess.

[Parasaurolophus]

The PhD exams we have in my current department are ill-considered, but holy crap -- nothing like that. If I'm not mistaken, you went to a well respected school, too, right? What a mess.

Yeah... and the details are a whole lot uglier than I was able to convey above. It's a fun story over a beer, though!

[sambaprof]

The students are requesting,  if they can change to "Computer Engineering" area from "Computer Science" area for their next qualifying exam attempt in January 2023. Motivation behind this is the possibility that the qualifying exams are set and administered by a different faculty. Also their work is interdisciplinary in nature. So it seems reasonable to me as well. Moreover, the subjects in the "Computer Engineering" area are closely related to Computer Science.

So, I am considering to check with the  Department and Graduate committee chair to change their qualifying exam area  from "Computer Science" to "Computer Engineering".

I have also got an appointment with the "Graduate Program Committee Computer Science Science area" chair this coming week to talk to him and I am hoping to get the insight on what is going on in regards to the qualifying exam process and the students' performance. 

I really hate to lose the student who passed one of the three subjects compared to the student, who failed all three subjects.

Having said that I am OK to continue to advise as long as both the students can can pass the qualifying exam on their own. If they cannot pass in the next attempt in January 2023 and possibly one additional attempt in August 2023 (assuming  if the students' get the option for "Computer Engineering" area and the
Program Committee OKs it), I am OK to just let them drop from PhD program.

Please advise if this is the right thing to do.

[research_prof]

First of all, why does your program have so old school exams? Computer science and engineering programs nowadays have qualifying exams that are research focused and typically students form a committee and present their research. The passing rate of such exams is almost 100% as long as students are doing well in terms of research and their advisor is happy. Maybe you should ask your graduate program chair to reconsider the format of the exam which was current 20 years ago.

[Caracal]

First of all, why does your program have so old school exams? Computer science and engineering programs nowadays have qualifying exams that are research focused and typically students form a committee and present their research. The passing rate of such exams is almost 100% as long as students are doing well in terms of research and their advisor is happy. Maybe you should ask your graduate program chair to reconsider the format of the exam which was current 20 years ago.

Just curious, how normal is it for programs to use qualifying exams as a way to weed out students? In my program, everyone who took the exams passed them.

[research_prof]

First of all, why does your program have so old school exams? Computer science and engineering programs nowadays have qualifying exams that are research focused and typically students form a committee and present their research. The passing rate of such exams is almost 100% as long as students are doing well in terms of research and their advisor is happy. Maybe you should ask your graduate program chair to reconsider the format of the exam which was current 20 years ago.

Just curious, how normal is it for programs to use qualifying exams as a way to weed out students? In my program, everyone who took the exams passed them.

Very good question. Typically, higher ranked R1s in my field use qualifying exam as a check box. As long as your advisor is happy, you pass the exam. Lower ranked R1s and R2s use the exam as the means to weed out students. At my current institution, unfortunately, these exams reach the limits of some sort of punishment, where the people that put together the exam questions are often motivated by things not related to academics but for example whether they have ongoing beef with the student's advisor. I have fought so hard to change the format of the exams, but talking to faculty that do not understand and do not conduct any research, I have only found deaf ears.

[Parasaurolophus]

First of all, why does your program have so old school exams? Computer science and engineering programs nowadays have qualifying exams that are research focused and typically students form a committee and present their research. The passing rate of such exams is almost 100% as long as students are doing well in terms of research and their advisor is happy. Maybe you should ask your graduate program chair to reconsider the format of the exam which was current 20 years ago.

Just curious, how normal is it for programs to use qualifying exams as a way to weed out students? In my program, everyone who took the exams passed them.

In my field, you have content quals, one or more language exams, and a logic exam. Lots of students founder at one or more of these hurdles.

[Dismal]

There should be lunch equity. Not the most important issue, but similar policies should apply to all.

Can students switch tracks easy without taking different courses?