This is a question about the philosophy of science.
How do you know clinical trials are better than observational methods? How do you know controlled experiments are better than post-hoc reasoning and logical deduction?
Standardized testing is just the application of the scientific method to education. If you don't believe in the scientific method then you shouldn't apply it here.
But if you are asking for an empirical method to compare the scientific method to "other ways of knowing", there isn't one.
Standardized testing is just the application of the scientific method to education.
That's not true in any way.
The purpose of the education system is to create a well-informed citizenry prepared for a productive life and for the duty of running a country.
The goal of standardized testing is to get some very rough metrics about superficial, easy-to-observe aspects of some basic parts of the apparatus.
The two are very tenuously related.
If we were applying the scientific method, we wouldn't be doing things like national, uncontrolled experiments that assume that basing major decisions on those rough metrics will result in improvements along the axes we actually care about. And then acting as if those metrics tell us everything about one aspect of the system, teachers, because that's where the political focus is.
Applying the scientific method would involve formulating hypotheses and then testing them through lab work and field trials. That's what we do with medicine, at least some of the time.
Heck, we could start by testing the hypothesis that getting everybody to focus lots of classroom time on standardized tests improves actual education.
> Heck, we could start by testing the hypothesis that getting everybody to focus lots of classroom time on standardized tests improves actual education.
Which is from 1992. I'm sure if I put some effort into it, I could find more, or at least demonstrate that standardized testing has not meaningfully changed in the last 20 years.
> Could you be specific? In particular, how do I measure whether a citizen is prepared for a productive life or not?
I like that you ignored the latter half.
You don't measure the citizen. You measure the rate at which the country is liberalizing. That is, the rate at which its core philosophical attitude is becoming more individualist, egalitarian, meliorist, and universalist. The purpose of public education is to shift us from the medieval era into the modern era. (And to be clear, being conservative is the opposite of being radical; it does not contradict being liberal. Many things that belong on the political Right are legitimately liberal but not Left.)
How many people are concerned with individual rights? How many people demonstrate equal moral worth and status to all individuals? How many people have a positive outlook on the future? How many people recognize universal qualities in all human beings? The rate at which these indicators are moving is positive, but that rate should be increasing in a country with a strong educational system; you'd ask the questions differently in a different era.
So you are proposing that the purpose of education is indoctrination into value system X, and propose that a successful system will indoctrinate more people?
Your goal is now vaguely stated. But how do I actually count the # of people concerned with individual rights/positive outlook/etc?
Yes, the purpose of all education is indoctrination to a set of ideas. Home schooling is also an exercise in indoctrination to a set of ideas. Critical thinking is an idea, egalitarianism is an idea, individuality is an idea, problem solving is an idea.
Education is a way of ensuring a more advanced and productive society through imparting certain knowledge and ideas to its children. The best metric for measuring the performance of an education system is rates of criminality. If a society has high crime rates then its education system is failing to produce productive members.
Education can be divorced from indoctrination. An educated person can display particular skills and knowledge, they don't need to believe that knowledge.
(See, e.g., Einstein displaying knowledge of quantum mechanics without actually believing it, or assorted creationists who actually understand evolution.)
I think your claim that criminality is the best measure of knowledge is interesting. Among other things, it suggests that the best way to increase knowledge might be to shut down the education system and implement a police state. (For the cost of the current education system we could increase the number of cops/courts/prisons by a factor of 6.)
> Among other things, it suggests that the best way to increase knowledge might be to shut down the education system and implement a police state.
Actually, an easier way to reduce the crime rate to zero is to repeal all categories of crime. Make it legal to steal and murder. Then all you have are existential crimes: "How could you let this happen!?" types: where crime is a metaphor rather than a fact.
> (For the cost of the current education system we could increase the number of cops/courts/prisons by a factor of 6.)
Crime rate is a decent metric, but I feel it's too indirect to be useful. If they do change, there are a lot of possibilities that could explain it: police aren't enforcing silly laws, that laws are reasonably legislated, police have been corrupted by bribery, and so on.
> So you are proposing that the purpose of education is indoctrination into value system X, and propose that a successful system will indoctrinate more people?
What did you think it was?
> Your goal is now vaguely stated. But how do I actually count the # of people concerned with individual rights/positive outlook/etc?
I think living a good life and doing something that benefits other people (aka earning money) is a good idea. If somebody disagrees, he can of course take his kids and go live under the bridge and open his school there. Let's see where we'd have more applicants - in a school that teaches kids how to be successful or in a school under a bridge that teaches them money is nothing, you can subsist on garbage and live in cardboard boxes forever. I think you can predict the result - and explaining this result would also explain which system is better, in opinion of your peers.
That's still indoctrination. I'm not sure why that's hard to understand. You can make all the assertions you like; you're still indoctrinating a particular value system.
OK, it's indoctrination. Teaching kids to use toilet, wipe their butts, wash their hands and not put a fork into an electrical outlet is an indoctrination too. So what?
Polling is another form of standardized testing (unless you propose asking different questions of each participant).
It seems you don't object to standardized testing at all, but merely to the goals of the current school system (conveying knowledge/skills rather than indoctrination).
> Polling is another form of standardized testing (unless you propose asking different questions of each participant).
No, it's not. Polling is an analysis of a large population through sampling. Standardized testing, in the context we're talking about, is an evaluation of performance based on the performance of a different group.
> It seems you don't object to standardized testing at all, but merely to the goals of the current school system
Here's the funny part. The goals are expressed through the school system. The kind of reform I'd like to see (read: infeasible in the short-term, at least) involve societal upheaval. I can talk about it, as long as you recognize that I've been working for the past few months on trying to figure out a good way of capturing my thoughts and have yet to succeed to my satisfaction.
I can't speak for wpietri, but my opposition to standardized testing is multifarious. For instance, I've experienced its detrimental effects on education firsthand. (In high school, there were a number of interesting topics that we had no time to cover because it was more important to teach to the test. My desire for education reform was born right there.) For the same reason, I dislike grading and GPAs and such. I don't consider this opposition very important, though, because I think we're teaching the wrong things to begin with.
> conveying knowledge/skills
The decision of which pieces of knowledge and which skills to focus on is a political decision and a form of indoctrination. Otherwise, you're not actually conveying anything so much as facilitating the ability to discover/develop knowledge/skills.
...evaluation of performance based on the performance of a different group.
I'm confused - what group of people should we be measuring, if not the students exiting the education system?
...there were a number of interesting topics that we had no time to cover because it was more important to teach to the test.
Again, I stand by my claim that your problem is not with standardized tests, but with the goals of the school system. You wanted schools to cover topic B, but the political system decided they must cover A.
The only role the tests played in this story is that they are an effective mechanism for making sure teachers actually do their job.
> The only role the tests played in this story is that they are an effective mechanism for making sure teachers actually do their job.
I am having trouble finding a word in that sentence that isn't wrong.
That's not the only role the play. E.g., they also cause people to spend more time on test prep and less on education.
There isn't just one story here, despite your attempts to pretend that teachers are the only problem and tests are the only solution. Which is the political angle you aim to work.
They are not a particularly effective mechanism. Which is why nobody sane uses standardized tests forevaluating, say, which developers to hire.
They are not an effective mechanism for making sure that teachers actually do their jobs. They could only theoretically cover a small part of the teacher's job, and it's not clear how well they cover that. And they lump together all of the other factors that go into even that small slice of a student's performance.
Can you be specific about what "test prep" means and how it differs from "education"?
Again, many larger companies (e.g., Google, Goldman, MS, DE Shaw) do attempt to standardize their hiring procedures and evaluation methods. But I suppose they are all "insane", right?
Are you seriously saying that any of those outfits uses a fill-in-the-bubble standardized test to make decisions about hiring, firing, and promotion for their software developers?
If not, I don't see what your point is.
I also don't believe you really don't understand where education and test prep diverge. I think that's a rhetorical technique.
Are you seriously saying that any of those outfits uses a fill-in-the-bubble standardized test to make decisions about hiring, firing, and promotion for their software developers?
They have collections of questions they ask on interviews, each assigned relative difficulties, and a standardized way to evaluate answers. They ask interviewers to stick to a certain structure, and discard results if an interviewer goes rogue. Standardization is about making sure results are objective and comparable to each other - "fill in the bubbles" is just one particular easy-to-grade method of accomplishing that.
In the past companies (pre Griggs v. Duke Power) did use even more standardized methods, which included "fill in the bubbles", they just don't get too close to completely objective standardized tests due to lawsuit risk.
I also don't believe you really don't understand where education and test prep diverge. I think that's a rhetorical technique.
For a decently designed test (such as the one I linked to) I don't believe they diverge significantly. As I suspected, you are unable to justify your claim that they do.
Ok. So your answer is: no, nobody uses a test like that for hiring. Why? Because, being cheap to grade, they're not a good measure for anything subtle. Like hiring decisions. And they certainly don't use them to evaluate anything after hiring.
As to the difference between education and test prep, I'm not unable to justify my position; I'm unwilling to try to do so to you. I don't think you're a serious conversational participant here, by which I mean one who's actually willing to learn anything.
But as a hint, your basic claim is that a fill-in-the-bubble standardized test is a perfect leading indicator for the decades-long effect we expect education to have. And that it can't be manipulated to diverge from that.
That's a very strong claim, and you're obviously smart enough to find the holes in that. You won't, though, because your whole MO on HN is as relentless debater on a pretty narrow set of political points.
Why is it suddenly my job to make up tests? If you'd like some, give it a go.
My point is that NCLB is scientific in the same way Taylorism was: it's a cargo cult activity that takes on the trappings of science and looks for something easy to measure. And then uses those measures in ways that aren't scientifically validated at all.
I'm asking you what evidence (if any) would change your belief. Perhaps no evidence is capable of changing your mind.
That's fine - you've just adopted a different philosophy, one of the "other ways of knowing" rather than science.
My point is that NCLB...
I'm not arguing for NCLB, I'm arguing that objective, comparable and reproducible measurements are the only viable way to optimize a system.
If you want to argue some specific metric is flawed, do it. Explain what should be measured and how the particular metric differs from it (i.e., details). So far, not a single person in this thread has actually done that - the closest we came was this: http://news.ycombinator.com/item?id=4713558
Science is awesome. It is also incredibly difficult and expensive.
I would love it if we did proper science with education. I would love it if we did it with software development. In neither case has anybody put forward sufficient resources to do real science in these realms. Not science sufficient to replace human judgment.
Until we can, I don't think cargo cult science is the way to go. Half-assed studies on software development methods, counting simple metrics for developers, measuring teachers through simple standardized tests: all cargo cult science.
For now, I'd much rather rely explicitly on human judgment. It's not as good as actual science, but it's much better than pseudoscience.
Standardized testing has nothing to do with science. There are studies in teaching and learning that do attempt to gauge the effectiveness of different teaching approaches, etc, and they typically use randomized trials and carefully controlled situations to be able to measure the effect.
But more crucially, the subjects (both teachers and students) in those studies are not rewarded based on the outcome of the study. Even if there is a correlation between the test and what we want schools to achieve (whatever that is), making teachers jobs contingent on that outcome will select for teachers who are good at achieving those results, which increasingly will diverge from your desired outcome. It's an example of dangers of "optimization by proxy" (http://lesswrong.com/lw/28r/is_google_paperclipping_the_web_...).
> Standardized testing is just the application of the scientific method to education. If you don't believe in the scientific method then you shouldn't apply it here.
Sorry, what's the null hypothesis here, and where's the control group?
How do you know clinical trials are better than observational methods? How do you know controlled experiments are better than post-hoc reasoning and logical deduction?
Standardized testing is just the application of the scientific method to education. If you don't believe in the scientific method then you shouldn't apply it here.
But if you are asking for an empirical method to compare the scientific method to "other ways of knowing", there isn't one.