Blogging about all sorts of things--governance in higher education, in businesses, and in law firms; bankruptcy ethics; popular culture & the law; Enron & other corporate fiascos; professional responsibility generally; movies; ballroom dancing; and anything else that gets my attention.
Tuesday, April 03, 2007
My bad--and thanks to "anonymous" and my dad for finding glitches in my USNWR spreadsheet
Maybe I'm still missing something, but I think your "bar passage ratio" is wrong. I think the figure you are using in that field is simply the overall passage rate of that paticular state's bar exam. New Mexico and Utah are at the top because 91/90 percent of everybody who takes those states' bar exams passes, regardless of school. For example, for Yale, 94.2% of them pass the NY bar (the primary bar they take); 74% is the overall success rate in the NY bar by everybody who takes it regardless of law school. I don't see what ratio you are talking about in listing Yale as "74% bar passage ratio." One could try to come up with a figure that measures how well/badly a particular lawschool does against the background rate in the state, but that isn't what these numbers do. Again, sorry if I'm missing something.
I agree that they are straight from the rankings, but (at least on the copy I have in front of me) US News just calls it (accurately) "jurisdiction's overall bar passage rate" while you are calling it (inaccurately, I think) "School's bar pass RATIO (ratio of the school's bar passage rate of the 2005 graduating class to that jurisdiction's overall state bar passage rate for first-time test takers in summer 2005 and winter 2006." In other words, your spreadsheets say you are "ranking" schools based on some sort of ratio that compares the school's performance to the rest of that state's bar takers -- but in fact your "ranking" just reports the overall passage rate for different jurisdictions.
Hi, again, Anon--USNWR defines the column "bar passage rate" in its methodology as:
The ratio of the school's bar passage rate of the 2005 graduating class to that jurisdiction's overall state bar passage rate for first-time test takers in summer 2005 and winter 2006.
So I just put in the parenthetical to link to USNWR's methodology.
The main US News rankings chart page that I am looking at doesn't provide the definition of "bar passage rate" that you quote, but I will take your word for it that they have said so. But still: Please just *look* at your "ratio" rankings. Don't they look odd? Both Utah schools are tied at 90. All three Okla schools are tied at 90. Every Minnesota and Missouri school is tied at 89. And then look at the bottom of the list. Every Calif school from Stanford to Chapman is tied in the basement at 62. Isn't it obvious from looking at this that what you are calling some sort of "ratio" of the school's performance to overall passage rate of that state's bar is *actually just reporting overall passage rate on that state's bar, regardless of school*! In other words, this is a meaningless ranking on its face.
If US News is giving the definition you say, for the numbers you say, then it is obviously wrong.
Some sort of comparative measure (e.g., Stanford grads pass the CA bar at a rate X points higher than the average CA bar taker, but Unaccredited Law Grads pass at a rate Y points lower) would be useful and easy to calculate. But your spreadsheet isn't it.
I agree with you that this factor isn't particularly useful, although I guess USNWR uses it to see how "difficult" the bar in the primary jurisdiction is. I do have a useful suggestion, though: Rob Morse at USNWR is very responsive to questions about the rankings, and you might try contacting him. Thanks!
I agree that it is somewhat useful to know a state bar exam's overall passage rate. That's why US News reports it. But it is not useful for you to use this figure to compile a "ranking" or "useful category." In what sense is it a meaningful ranking of law schools simply to list schools by state? (Why not just alphabetically, while we're at it?) US News doesn't purport to rank schools in such a way. It is even less useful to continue referring to this number as some sort of school-specific performance ratio when it has been pointed out that it is not (even if your error originally started from something US News somewhere said.) I didn't set out to write a string of messages, but just to provide a friendly correction that I imagined you would want to know about. I won't comment further.
7 comments:
Maybe I'm still missing something, but I think your "bar passage ratio" is wrong. I think the figure you are using in that field is simply the overall passage rate of that paticular state's bar exam. New Mexico and Utah are at the top because 91/90 percent of everybody who takes those states' bar exams passes, regardless of school. For example, for Yale, 94.2% of them pass the NY bar (the primary bar they take); 74% is the overall success rate in the NY bar by everybody who takes it regardless of law school. I don't see what ratio you are talking about in listing Yale as "74% bar passage ratio." One could try to come up with a figure that measures how well/badly a particular lawschool does against the background rate in the state, but that isn't what these numbers do. Again, sorry if I'm missing something.
Thanks again for weighing in--these numbers are straight from USNWR--I didn't do anything else to them.
I agree that they are straight from the rankings, but (at least on the copy I have in front of me) US News just calls it (accurately) "jurisdiction's overall bar passage rate" while you are calling it (inaccurately, I think) "School's bar pass RATIO (ratio of the school's bar passage rate of the 2005 graduating class to that jurisdiction's overall state bar passage rate for first-time test takers in summer 2005 and winter 2006."
In other words, your spreadsheets say you are "ranking" schools based on some sort of ratio that compares the school's performance to the rest of that state's bar takers -- but in fact your "ranking" just reports the overall passage rate for different jurisdictions.
Hi, again, Anon--USNWR defines the column "bar passage rate" in its methodology as:
The ratio of the school's bar passage rate of the 2005 graduating class to that jurisdiction's overall state bar passage rate for first-time test takers in summer 2005 and winter 2006.
So I just put in the parenthetical to link to USNWR's methodology.
The main US News rankings chart page that I am looking at doesn't provide the definition of "bar passage rate" that you quote, but I will take your word for it that they have said so.
But still:
Please just *look* at your "ratio" rankings. Don't they look odd? Both Utah schools are tied at 90. All three Okla schools are tied at 90. Every Minnesota and Missouri school is tied at 89. And then look at the bottom of the list. Every Calif school from Stanford to Chapman is tied in the basement at 62.
Isn't it obvious from looking at this that what you are calling some sort of "ratio" of the school's performance to overall passage rate of that state's bar is *actually just reporting overall passage rate on that state's bar, regardless of school*! In other words, this is a meaningless ranking on its face.
If US News is giving the definition you say, for the numbers you say, then it is obviously wrong.
Some sort of comparative measure (e.g., Stanford grads pass the CA bar at a rate X points higher than the average CA bar taker, but Unaccredited Law Grads pass at a rate Y points lower) would be useful and easy to calculate. But your spreadsheet isn't it.
I agree with you that this factor isn't particularly useful, although I guess USNWR uses it to see how "difficult" the bar in the primary jurisdiction is. I do have a useful suggestion, though: Rob Morse at USNWR is very responsive to questions about the rankings, and you might try contacting him. Thanks!
I agree that it is somewhat useful to know a state bar exam's overall passage rate. That's why US News reports it.
But it is not useful for you to use this figure to compile a "ranking" or "useful category." In what sense is it a meaningful ranking of law schools simply to list schools by state? (Why not just alphabetically, while we're at it?) US News doesn't purport to rank schools in such a way. It is even less useful to continue referring to this number as some sort of school-specific performance ratio when it has been pointed out that it is not (even if your error originally started from something US News somewhere said.)
I didn't set out to write a string of messages, but just to provide a friendly correction that I imagined you would want to know about. I won't comment further.
Post a Comment