The Atlantic: The "Dead" Syllabus

[marshwiggle]

It never made sense to me that a syllabus wouldn't include the LOs: done right, they're clearly stated, objective, and measurable, and once established, it's easy to key every assignment to one or more of them.

This is key.

What if any steps are regularly taken to at least attempt to measure whether such outcomes are met, and what happens when it is reasonably determined that they have not been?

In my courses, I point them out to students at the beginning of term*, and then partway through (and sometimes late in the term) I give them a list of them and have them identify how many have been met completely, how many "so-so", and how many not-at-all. It's a good exercise for them and they see how much they have learned.

*(Of course, at the beginning of term most of the objectives won't even be comprehensible to them. In a kind of Dunning-Kruger sense, understanding what an objective means often establishes that they've achieved it.)

[pgher]

Uh-oh, someone on my campus has decided that they should start checking syllabi. I'm not sure what problem they're trying to solve. A lot of administrators here tend to solve a tiny problem with an onerous policy, rather than addressing the individual bad actors. Hopefully we can ride out the storm.

[MarathonRunner]

What if any steps are regularly taken to at least attempt to measure whether such outcomes are met, and what happens when it is reasonably determined that they have not been?

Learning outcomes have to be measurable, so measurement is incorporated into the learning outcomes. Professors know if a given outcome has been met for a given student. Learning outcomes can be measured in many different ways - this was something covered in my "teaching and learning in higher ed" course I took during my PhD. It's pretty basic stuff for effective teaching in my accredited field. Chairs and deans look at how many students have successfully achieved an outcome by course. If outcomes have not been met, students fail. If many students are not successfully achieving the learning outcomes, then courses may be revised or new courses are created. Accreditation teams also look at the measures and how many students have achieved the outcomes, and may provide suggestions for improvement, or even place a program on probation until things improve.

[kaysixteen]

I get that there are legit ways to ascertain whether any given learning outcome X has been met, but it would seem much harder to figure out WHY it has not been met...?

[quasihumanist]

First, I strongly disagree that all learning outcomes should be measurable.

We all have learning outcomes that we want students to achieve that are hard to measure.  If we just focus on the measurable ones, then we end up not achieving the ones that are harder to measure.  And then our students end up getting training, not an education.

Second, if most of your students are meeting most of your outcomes, you're not ambitious enough.

If you insist that most of your learning outcomes get met by most of your students, then you don't end up teaching better; you end up dumbing down your outcomes so you look more successful.

[jerseyjay]

Learning outcomes have to be measurable, so measurement is incorporated into the learning outcomes. Professors know if a given outcome has been met for a given student. Learning outcomes can be measured in many different ways - this was something covered in my "teaching and learning in higher ed" course I took during my PhD. It's pretty basic stuff for effective teaching in my accredited field. Chairs and deans look at how many students have successfully achieved an outcome by course. If outcomes have not been met, students fail. If many students are not successfully achieving the learning outcomes, then courses may be revised or new courses are created. Accreditation teams also look at the measures and how many students have achieved the outcomes, and may provide suggestions for improvement, or even place a program on probation until things improve.

Yes, in theory, but in practice I have never seen it work this way.

To the extent that learning outcomes are rolled into the normal assessment in the course, the instructor knows if the student has met them based on the grades. One history survey course that I taught as an adjunct at a local university had eight learning outcomes; three general education outcomes and five history outcomes. Most of them were vague enough that you would expect an introductory course to somehow touch on them ("Evaluate evidence and arguments critically or analytically"; "Analyze the significance of one or more major movements that have shaped the world's societies.") I tried to make the essay assignment and the essay exam reflective of those outcomes. But nobody ever asked me how well the students did on any or all of the learning outcomes.

At the department where I teach full-time, the chair (there have been several over the years) has never asked me how the students have done on the learning objectives.

I was for a while the assessment representative of our department both for the general education program and more broadly. For the general education program, we had all students in different level general education program submit their final essays to the committee, and then we randomly selected some and read them and discussed if they met the learning outcomes. Since these were from very different courses in very different departments, it was, in my mind, a complete waste of time. It was discontinued, first, with each professor being asked to estimate how many students met the learning outcomes, and then, after self-reporting turned out to be not that useful, with nothing.

On the departmental level, assessment means that we choose one of the AHA's "Tuning Project" goals each year, and then assess the senior theses based on these goals (e.g., use of primary sources). Then we discuss how well the students did this, and if they did not do well, we recommend that professors integrate, e.g., use of primary sources, in their courses next semester.

[lightning]

First, I strongly disagree that all learning outcomes should be measurable.

We all have learning outcomes that we want students to achieve that are hard to measure.  If we just focus on the measurable ones, then we end up not achieving the ones that are harder to measure.  And then our students end up getting training, not an education.

Second, if most of your students are meeting most of your outcomes, you're not ambitious enough.

If you insist that most of your learning outcomes get met by most of your students, then you don't end up teaching better; you end up dumbing down your outcomes so you look more successful.

^^^ THIS ^^^

Assessment was in entrenched in k-12 before it found its way into higher ed. We all saw first-hand, the ineffectiveness and detriment of Assessment on learning in k-12, in our incoming undergraduate college students: capable on paper--incapable in reality.

Now that Assessment has become entrenched in higher ed, is it any wonder that the employers and graduate schools are complaining about their incoming grad students/entry-level employees who are capable on paper but incapable in reality? Even worse, is it any wonder that there are many many graduates of programs who couldn't even get into a grad school or get a job, and they (and those close to them) are now questioning the value of higher education, even though all the boxes were checked on their SLOs in all of their classes?

I'm, of course, not going to go so far as saying that Assessment ruined higher ed, but it sure had a hand in it.

[MarathonRunner]

First, I strongly disagree that all learning outcomes should be measurable.

We all have learning outcomes that we want students to achieve that are hard to measure.  If we just focus on the measurable ones, then we end up not achieving the ones that are harder to measure.  And then our students end up getting training, not an education.

Second, if most of your students are meeting most of your outcomes, you're not ambitious enough.

If you insist that most of your learning outcomes get met by most of your students, then you don't end up teaching better; you end up dumbing down your outcomes so you look more successful.

We are an accredited professional program, accredited by a professional organization. In order for students to meet the entry level criteria to write the exam that allows them entry into our profession, they need to meet certain learning objectives that display they have mastered entry level competencies for our profession. If most students (and this is a highly selective program) couldn't satisfy the learning objectives, that indicate entry level competencies in our field, there would be something terribly wrong. Of course, not all students are going to succeed, even in a selective program.

Maybe it is different in the U.S. and/or in fields that aren't accredited by a professional organization that leads to a specific title that has restricted use (MD, DVM, RN, PEng, etc.) My field is very similar to those, and learning objectives are an important part of remaining accredited by the external professional accreditation body (think Engineers Canada, Committee on Accreditation of Canadian Medical Schools, etc., we are accredited by a similar body specific to our professional field).

Our learning outcomes consist of the following:

– Do what? (what cognitive or skill process will students demonstrate?)
– With what? (with what parts of the curriculum?)
– For what? (for what purpose? To accomplish what?)
- As evidenced by? (how do students demonstrate they have achieved this outcome? At what level?)

[Wahoo Redux]

First, I strongly disagree that all learning outcomes should be measurable.

We all have learning outcomes that we want students to achieve that are hard to measure.  If we just focus on the measurable ones, then we end up not achieving the ones that are harder to measure.  And then our students end up getting training, not an education.

Second, if most of your students are meeting most of your outcomes, you're not ambitious enough.

If you insist that most of your learning outcomes get met by most of your students, then you don't end up teaching better; you end up dumbing down your outcomes so you look more successful.

We are an accredited professional program, accredited by a professional organization. In order for students to meet the entry level criteria to write the exam that allows them entry into our profession, they need to meet certain learning objectives that display they have mastered entry level competencies for our profession. If most students (and this is a highly selective program) couldn't satisfy the learning objectives, that indicate entry level competencies in our field, there would be something terribly wrong. Of course, not all students are going to succeed, even in a selective program.

Maybe it is different in the U.S. and/or in fields that aren't accredited by a professional organization that leads to a specific title that has restricted use (MD, DVM, RN, PEng, etc.) My field is very similar to those, and learning objectives are an important part of remaining accredited by the external professional accreditation body (think Engineers Canada, Committee on Accreditation of Canadian Medical Schools, etc., we are accredited by a similar body specific to our professional field).

Our learning outcomes consist of the following:

– Do what? (what cognitive or skill process will students demonstrate?)
– With what? (with what parts of the curriculum?)
– For what? (for what purpose? To accomplish what?)
- As evidenced by? (how do students demonstrate they have achieved this outcome? At what level?)

A few English-y teacher types love learning outcomes.

Most of us, myself included, look at learning outcomes on a syllabus as more bureaucratic boilerplate. 

For a few cases we can identify skillsets such as grammar usage, functional library and database research, and fluency with MLA and APA citation styles, but most of what we teach are softer skills that are intellectual and hard to define.  We want students to learn to write better than they did before taking out classes, learn to critically read texts, analyze, research intelligently, and broaden their minds.  Sure, you can put these on a syllabus, but these have always been the goals and putting them on a page or classroom management software doesn't change a whit what we have to do. 

[marshwiggle]

First, I strongly disagree that all learning outcomes should be measurable.

We all have learning outcomes that we want students to achieve that are hard to measure.  If we just focus on the measurable ones, then we end up not achieving the ones that are harder to measure.  And then our students end up getting training, not an education.

Second, if most of your students are meeting most of your outcomes, you're not ambitious enough.

If you insist that most of your learning outcomes get met by most of your students, then you don't end up teaching better; you end up dumbing down your outcomes so you look more successful.

In my labs, my students have to write post-lab summaries of what they learned. Three things that are required for summary items is that they be

• New to this lab
• Useful for the future
• Specific; i.e. with enough detail that the summary itself is a useful document

So it's useless to say "I learned about Whatchamadoodles".
It's more useful to say "The Whatchamadoodle is made up of three parts; the flux capacitor, the McGuffin, and the foobar."

If there is nothing specific to learning outcomes from this course as distinct from other courses in the discipline, then why have all of the different courses?

[Caracal]

First, I strongly disagree that all learning outcomes should be measurable.

We all have learning outcomes that we want students to achieve that are hard to measure.  If we just focus on the measurable ones, then we end up not achieving the ones that are harder to measure.  And then our students end up getting training, not an education.

Second, if most of your students are meeting most of your outcomes, you're not ambitious enough.

If you insist that most of your learning outcomes get met by most of your students, then you don't end up teaching better; you end up dumbing down your outcomes so you look more successful.

We are an accredited professional program, accredited by a professional organization. In order for students to meet the entry level criteria to write the exam that allows them entry into our profession, they need to meet certain learning objectives that display they have mastered entry level competencies for our profession. If most students (and this is a highly selective program) couldn't satisfy the learning objectives, that indicate entry level competencies in our field, there would be something terribly wrong. Of course, not all students are going to succeed, even in a selective program.

Maybe it is different in the U.S. and/or in fields that aren't accredited by a professional organization that leads to a specific title that has restricted use (MD, DVM, RN, PEng, etc.) My field is very similar to those, and learning objectives are an important part of remaining accredited by the external professional accreditation body (think Engineers Canada, Committee on Accreditation of Canadian Medical Schools, etc., we are accredited by a similar body specific to our professional field).

Our learning outcomes consist of the following:

– Do what? (what cognitive or skill process will students demonstrate?)
– With what? (with what parts of the curriculum?)
– For what? (for what purpose? To accomplish what?)
- As evidenced by? (how do students demonstrate they have achieved this outcome? At what level?)

A few English-y teacher types love learning outcomes.

Most of us, myself included, look at learning outcomes on a syllabus as more bureaucratic boilerplate. 

For a few cases we can identify skillsets such as grammar usage, functional library and database research, and fluency with MLA and APA citation styles, but most of what we teach are softer skills that are intellectual and hard to define.  We want students to learn to write better than they did before taking out classes, learn to critically read texts, analyze, research intelligently, and broaden their minds.  Sure, you can put these on a syllabus, but these have always been the goals and putting them on a page or classroom management software doesn't change a whit what we have to do. 

Yeah, exactly. As far as I can tell, assessment is almost solely about pretending that you have set clear goals and that you are measuring them. The accrediting agencies don't have the slightest interest in whether any of it is valid. You could have an argument about whether trying to measure these things empirically really has any use, but that's beside the point when the process is designed to pretend you're doing these things instead of actually doing them.

[ciao_yall]

The whole "assessment of SLOs" came and went in my system when it was realized that students who pass the class also achieve the outcomes. Measuring them is kind of pointless. I would just have a target question on my final exam or project, then use that as the basis for my report on how many students achieved or did not achieve the specific assessment I was measuring.

Where I did learn to appreciate having standardized SLOs was that it did foster dialogue among other faculty with whom I was teaching the same, or similar classes.

It also helped when a faculty member decided they liked their "own" objectives better, and they really weren't related to the class the way it was designed or fit into the major. Think of a class on Applied Underwater Basketweaving, and they wanted the whole class to be all about how Basketweaving needed to be Inclusive. Noble goals, maybe, still, not the point of the class. Certainly could be included in the general class assignments and discussion, but at the end of the day the student needed to Apply. Basket. Weaving.

[marshwiggle]

The whole "assessment of SLOs" came and went in my system when it was realized that students who pass the class also achieve the outcomes. Measuring them is kind of pointless. I would just have a target question on my final exam or project, then use that as the basis for my report on how many students achieved or did not achieve the specific assessment I was measuring.

This is one of the smart ways to come up with outcomes. What things are you going to require them to know or do for assignments, exams, etc. Those are obviously the outcomes you're aiming at.

Where I did learn to appreciate having standardized SLOs was that it did foster dialogue among other faculty with whom I was teaching the same, or similar classes.

It also helped when a faculty member decided they liked their "own" objectives better, and they really weren't related to the class the way it was designed or fit into the major. Think of a class on Applied Underwater Basketweaving, and they wanted the whole class to be all about how Basketweaving needed to be Inclusive. Noble goals, maybe, still, not the point of the class. Certainly could be included in the general class assignments and discussion, but at the end of the day the student needed to Apply. Basket. Weaving.

Absolutely. Whenever courses have to fit into some sort of sequence, earlier courses will need to cover certain content, techniques, etc. that will be required by later courses.That identifies specific outcomes for the earlier courses.

[Ruralguy]

Agreed. If its a conversation among practitioners who share goals and also the classes are either hierarchical or at least share a common language (say, Thermodynamics and Mechanics--not related in a hierarchical way but related through understanding fundamental laws and mathematics and using that to solve problems), then assessment becomes not only straightforward, but potentially useful. If all of the courses are truly independent, then assessment can become a bunch of independent bs declarations.

[whistle_nutz]

The syllabus template where I teach was drafted, no shit, by a dean who has never taught a class. Course descriptions are typically one phrase long, not complete sentences.