Law School Grades are Bogus

By: David

I intend to keep 90% of my posts under 700 words. But because the university recently posted our grades, I thought I'd rehash a classic that I first posted on LinkedIn. 

Anyone who has spoken with me in the past couple of years about evaluating students knows that I’m not a fan of letter grades (A, B, C, D, and F). In fact, I think our society relies on them to our detriment. (If you need a primer, here's an article written from a different perspective from The Atlantic. A key quote: "[p]oints-based grading undermines learning and creativityrewards cheatingdamages students' peer relationships and trust in their teachersencourages students to avoid challenging work, and teaches students to value grades over knowledge.") But that's not the primary focus of this post.

One of the problems with law school is that it takes the letter grade perversion of evaluation a step further than K-12 schools, creating an even wider gap between grades and what they’re supposed to represent. To understand how and why, it’s important to explain how most law schools approach education, and then we’ll see how that compounds the complications of their approach to grading. I’m going to use Georgetown Law specifically because its approach is the one I'm most familiar with.

Georgetown has a large law school program, with over 500 full-time students coming in last year. We’re then split into Sections. For instance, I was in Section 4 along with over 100 other students. Aside from three classes where we were broken into smaller groups, everyone in each section takes all their classes together. That means all of Section 4 took torts, criminal justice, property, civil procedure, and constitutional law together. We were all in large lecture hall-type rooms where we all took the class with the same professor at the same time. The other, small group courses--legal reading and writing, and contracts—took place in smaller rooms with about 20-30 people. Our one elective--either a statutory interpretation-focused course or an international law-focused course—was also in a large lecture hall-type room, but included a mix of students from other Sections.

First, a caveat: What follows is a generalization of the 1L year (the first year of law school) and is not necessarily representative of all professors or law schools. It is, however, an accurate description from my perspective.

Class Instruction

The act of educating at law school goes like this: Throughout the semester you read your assignments, which almost exclusively consists of judge “opinions” (the written rationales by judges for why they reached their particular decision; also called "cases"). Assignments are between 20-60 pages long per class, depending on the professor, and reading ten pages of case law takes about an hour (including note taking).

You're never told ahead of time what to look for in any reading. Instead, you're to read everything, take notes based on what you thought was worth noting, and then show up to class and be prepared to answer whatever questions the professors throw at you. Now, while everyone notes the facts, reasoning, and “holding”—the judge’s final decision—of the cases, it’s a rare instance when all students seem to note all the same facts and reasoning for a class and those are also the facts and reasoning the professor asks about. Instead, it's not uncommon to fumble through the reading, get to class, be asked a question you didn’t know the exact answer to, and then either have to re-read multiple paragraphs in class while the professor waits, or hope against hope that the professor calls on another student who might know the answer.

As you can imagine, it’s a remarkably inefficient process for the first few weeks of law school because each professor likes to focus on different aspects of cases, and you just don’t know those aspects until you’ve sat through several classes. And then, even after weeks of classes, it’s an inefficient process.

Guided reading, where professors share with students stuff like what to focus on, tips for understanding the case, context, and so on would be nice, but it's rarely given. It is all too common to miss the forest for the trees as you try to capture every detail that may be asked of you (specific sentences, paragraphs, and what seemed (seems) like unimportant minutia) rather than trying to learn and understand the broader concepts and how those concepts relate to one another. This approach seems even sillier in retrospect because knowing stuff like the origins of admiralty law when discussing Torts is frankly unimportant both for 1L torts and for almost every legal job.

It’s also worth noting that there doesn’t seem to be any evidence that reading textbooks consisting exclusively of cases is the most effective or efficient way to learn how to think like a lawyer. Moreover, it's not proven to be a more effective method than other traditional forms of education, such as using textbooks with pictures, drawings, background information, diagrams that connect concepts, key points, etc. (There are notes by the authors of the textbooks after the cases, but the notes are often mostly a series of rhetorical questions that contribute little or nothing to learning, take up precious time to read, and are therefore often merely skimmed or entirely skipped by students.)

In contrast, the Institute of Educational Sciences has found six methods of teaching that have been empirically shown to improve learning:

  1. Pair graphics with words

  2. Link abstract concepts with concrete representations

  3. Pose probing questions

  4. Repeatedly alternate between problems with solutions provided and problems the students must solve

  5. Distribute practice by asking questions about topics multiple times over the course of several weeks

  6. Assess to boost retention by giving formative assessments (doesn't have to be for a grade)

Of the six, the only one all professors consistently adhere to is asking probing questions. Notably, none of the IES's research findings say “Read large blocks of text about topics for which you have little to no context.” Reading cases was something law schools started to do in the late 1800s and for whatever reason have stubbornly resisted changing. To their credit, law schools previously forced students to memorize laws. So, I mean, that’s progress, right?

Finals

So that’s the semester. You read for several hours a day and go to class. And that’s it. No quizzes or tests during the semester. No checks on learning to ensure you’re on the right track. No method whatsoever to ensure you’re actually learning anything or learning what the professor wants you to learn, aside from being able to answer the handful of questions you may be asked throughout the semester, and for which there is no penalty if you’re unable to offer a correct or preferred response. And then, BOOM. Finals.

Your entire grade for each class is based on a single final exam. Even your most rock star peer, who was capable of asking profound questions during class, and who was always ready to answer any question a professor may ask, is at the mercy of the final exam to determine the entirety of his/her academic fate. Success throughout the semester in the form of notable classroom participation means nothing to your GPA.

The exams, without exception, require writing about 10-20 pages in 3-24 hours (depending on the professor). The format of the exams is only revealed in vague terms beforehand, and the content you're to be tested on is unknown. You read literally hundreds of pages from a textbook over the course of the semester, and then do your best to memorize as much of it as humanly possible and organize it into neat notes. Oh, and there is no Reading Week or other extended additional time to prepare. You may finish classes on Friday and then take an exam for one of the classes the following Tuesday.

Because you don't know what the exam will be about, exam prep and the exam itself are crapshoots. Some courses, like torts, essentially only require applying the core tenants of the subject to a given situation (duty, breach, causation, injury, and damages). Other courses, like constitutional law, require knowing a much, much greater number of topics (e.g., the nation's founding, clauses, limits of executive authority, etc.) and each topic's sub-elements (e.g., for the founding: who founded, when, what was discussed and by whom, how did they vote, why; how did the Court interpret it, when, why, etc.).

To take an extreme example, a student could memorize everything there is to know about constitutional law from 1820 onward, but if there is just one question about the founding, and the student doesn't know the answer, then there is no chance the student will get an A because you simply can't afford the miss out on the points of a question.

However extreme, that same issue plays out on a much smaller scale throughout the exam. You may know all there is to know about 80% of the material you covered in class, but if just one question on the exam is about the 20% you didn't know and didn't know you didn't know, then you're screwed. This is especially true even when you have a faint idea of what the right answer is when the exam is, say, three hours long, because to get an A, you must spend every minute of that exam applying your knowledge in order to perform well. There is not enough time to pause and consult your textbook for more than a couple of minutes. Unlike in math, for instance, where you can work through a problem to figure it out, law exams are generally a "You either know it or you don't" situation, though your notes can sometimes assist. For example, you either know when consent applies to searches or you don't. You either know the three levels of executive power from Jackson's opinion or you don't.

As I've alluded to, this approach to exams is absurd for the following reasons, among others:

  1. Being able to write a coherent essay under these intense time constraints is not the best--or even a proven accurate--reflection of who will be a good lawyer (let alone a good law professor). Even if you know the topic well and could give a brilliant oral explanation, if you cannot quite organize your written thoughts in the allotted time, you will not get an A.

  2. The narrow nature of the exam (some have only two segments) means that knowing nothing but the answers to those few questions is all that matters. This is a bit crazy, because it takes a leap of logic to suggest that because a student happens to know the few areas of one course that happen to appear on the exam, it necessarily means the student likely knows all other areas of that course equally well. That's the premise of giving a letter grade from a single exam that is supposed to represent one's comprehension of the entire course's content, by the way. As humans, we each comprehend different topics to different extents, but the final exam doesn't account for this.

  3. If you happen to be ill on the day of the exam or experience a recent episode in your life that distracts you, or you have a massive headache or any number of a hundred other reasons, you will likely perform poorly, even if on any other day you would have excelled. And yet, you're stuck with that bad grade.

  4. There is no check on learning during the semester, so you may believe you understand certain concepts well--or at least well enough--and not study them as deeply as others. In any other academic program, this would be sensible because time spent on something you already know is time not spent on something you don't know. However, because we don't really know what we don't know in law school, we're left to just guess how to best spend our study time.

I feel confident I could drone on about all the reasons having a grade for a semester-long course based on exactly one exam is a terrible idea that is unreflective of students' actual abilities, but I'll spare you because there is yet another important step.

Grading

After turning in your exam, your professor will grade it. (It's important to note here that this isn't an attack on professors as people. They, like everyone, are creatures of their environments.) There are two general approaches to grading (which are often blurred):

  1. Apply a grading matrix with a certain number of points awarded for mentioning and applying certain legal arguments, and

  2. Stroking one's chin and applying one's wisdom in a somewhat-less-than-whimsical fashion (as explained below, the grading curve makes this approach to grading even more dubious than it sounds)

Both methods are deeply flawed. The matrix approach, while ostensibly objective, requires that the professor create the grading matrix prior to issuing the exam so they can remain objective (i.e., they don't add to or modify their approach to grading based on how students respond to the exam). However, as professors who use this approach admit, they are willing to award "bonus" points for arguments they deem good, but that didn't make it onto the original matrix. This may seem good for the students, but it means the matrix approach is no longer objective. Determining something that wasn't on the matrix is good enough to be awarded points is purely subjective. Additionally, if mentioning something can be worth a range--say, 1-4 points--it's an entirely subjective action to deem one person's mention of the concept as worth 2 points, while another student gets 3.

Furthermore, I'm not convinced that if you gave two professors the exact same matrix and copies of every student's exam that the professors would award every exam (or all but a very few) the exact same number of points. The inability to do so undermines this approach to grading by revealing the non-objective nature of matrix grading in practice.

Yet, it's still a better approach than the chin stroke method.

The chin stroke method largely relies on form, style, and professor preference. If you happen to write in a style they like (creative arguments v. well-trodden arguments, flowery v. pointed, use Oxford commas or not, cite cases they find most interesting or not, use a tone they find pleasing or not, complex sentences v. simple ones, etc.) then, all else being equal, you will get a higher grade. Think about that. Even if you use the exact same legal reasoning as a peer, you could receive an inferior grade based solely on a professor's preference for style. You are not then being graded on your knowledge of the content and how you apply it, but in how you present it, even if your presentation is grammatically correct and the language you use is clear.

Why can't the professor just award everyone a good grade if their legal arguments are good, regardless of presentation style? Because most law schools adhere to the ridiculous idea of a grading curve.

The curve means that even if everyone writes an above-average paper, not everyone will get an above-average grade. For instance, the recommended curve at Georgetown Law is:

A: 12%, A-: 19%, B+: 28%, B: 31%, and B- and Below: 5-10%.

That means that in a Section of 100 students (similar to Georgetown Section sizes), only 12 people will get an A, even if 3 students in one Section write exams better than all students who get an A in other Sections.

Now, it may be true that telling a solid A from the rest is relatively easy, but it strains reason to believe that the line between papers that get an A- rather than an A (if it's on the better side) or B+ (if it's on the weaker side) is so clear. This is especially true in the chin stroking approach. When 20 papers use the same legal argument, how do you determine which of those 20 will get the B+ while the others get an A-? The line is so blurry as to be meaningless. I'd be willing to wager a large sum of money that if two professors who co-taught a course were to separately award grades on exams, they would never award all the same grades to the same students. How could they when the distinction between grades is essentially arbitrary?

This is why some schools have done away with grades. The grades provide no meaningful information. What does an A- in property mean? Doe the student know what an easement is? Copyright law? Who has liability under various situations regarding lateral support? Of course, you can't tell. All the A- tells you is that the student supposedly did better on a single exam than 69% of her peers. For all you know, all the students did remarkably poorly compared to other Sections or law students at other schools, so being in the top 31% is no better than being the sharpest butter knife in the silverware drawer. Because it conveys precious little useful information of how that particular student compares to law students at other schools, it therefore does very little to inform recruiters which students are the best candidates for a job.

The reliance on GPA becomes even dicier when recruiters simply look at resumes--even for students from the same school. For example, at Georgetown there are six Sections. To be in the top 10% of each section you'd need a GPA of:

1: 3.76, 2: 3.71, 3: 3.71, 4: 3.67, 5: 3.66, and 6: 3.74

This means that if you're looking at a pile of resumes, you'd assume the student getting a 3.76 or 3.71 was better than the student from Section 5 who got a 3.66--but you'd be wrong. They both would be in the top 10% of their section. A 3.76 in Section 1 is not obviously better than the 3.66 in Section 5. The Section 5 professors just apparently grade harder. Our insistence on a grading curve, combined with the subjective grading, makes such recruitment follies unavoidable, much to the detriment to the students in Sections with lower GPAs. And some may declare that GPAs are actually rarely considered or play only a very small part in recruitment and hiring, but if that was the case, why do law firms specifically ask for them? Almost no applications separately ask for information like volunteer or internship experience in addition to cover letters and resumes as they do for GPAs.

The scary thing is that in order to make it past the first cut with recruiters from selective firms and businesses, you must have a good GPA. You GPA matters. Getting an A- instead of a B+ in a course can have very real consequences, because it can be the difference between being in the top third of your Section or not. Though, really, what's the difference between someone with a 3.60 and a 3.61? Or even a 3.65? Alas, GPAs are one of the most efficient ways for recruiters to separate students so they can focus on what they falsely presume must be the high achievers. There is a misplaced assumption that because GPAs are numbers, they are surely objective, based on quantifiable, valid, and reliable measurements. In fact, the grading system is so bankrupt of actual value, it robs deserving students of top employment opportunities.

Some schools instead award more general assessments, like pass/fail, or high pass/pass/low pass/fail (where I assume the high pass is for exceptional, objectively superior work, and the low pass is for people who can demonstrate knowledge of the material at a minimally acceptable level). This is a step in the right direction.

And, of course, we must ask: What is the point of grades? Are they supposed to predict success? What is success? A study from Harvard indicates grades are not predictive of who becomes a partner at a firm. It also shows that, on average, students who enter the public sector have higher GPAs than students who enter law firms, so GPAs aren't a good predictor for income (unsurprisingly, firms pay better than public sector jobs, on average) if that is how we want to measure success.

I submit that grades show which students professors believe answered their exam questions in the manner the professors prefer the questions to be answered and little else. They are not a good indicator of who studied the most, who understands the textbook the best, who will become a top litigator, who will make partner in a firm, or who is most likely to become a senator. In short, letter grades are poor indicators of almost everything, though employers inaccurately believe grades indicate a great deal.

Counterarguments and Conclusion

One argument I foresee is that the class rank and GPA are actually there to counteract the weaknesses of this system. That is, regardless of how grading occurs, a GPA and class rank shows how the student did over time relative to peers. It's a measure of consistency. Surely a student with a 3.60 GPA and who is in the top third is superior to students with lower GPAs and who are not in the top third, right? NO. You can't throw together a deeply deficient system and expect the output to suddenly be a valid measurement. That'd be like using only rusty junkyard parts to build an engine and suggesting it could compete in a NASCAR race. For the whole to be greater than the sum of its parts, the parts must still be good.

Suppose every professor awarded a student an A-. Great. But if that A- signifies so little, then it certainly can't be used to signify that student is any better than one who received all B+'s (and then you have to consider the student who gets all A-'s versus the one with half A's and half B+'s, or the student who does brilliantly their first semester but horrific their second semester, and all kinds of other combinations). Class rankings are built on GPAs. GPAs are built on course grades. Course grades are built on the curve. The curve is built on the exams. Exams are built on content covered in class. And as I've explained, that means every step of the system is too warped and ineffective to be meaningful.

Another argument may be that the teaching and assessment methods at law school prepare students to be lawyers. That it's a feature, not a bug, that assignments consist solely of reading pages of text, and that the final exams consist of short answers and long essays. I'd counter that there is no evidence law school teaching methods are the most effective for preparing lawyers and that the exams and their time constraints do not accurately reflect day-to-day reality for most lawyers. In fact, I'd argue the approaches taken at Georgetown and the majority of other law schools are often counterproductive and ineffective. Why? Because there are mounds of research on pedagogy and andragogy, and none of it suggests the methods developed over 100 years ago were optimal. It's time for something new, based on 21st-century research, rather than the 1800's whimsy.