How (not) to test language proficiency


The United States of America has no official language, although English functions as the de facto language of the government and media. This doesn’t mean English is the only language in use. In some regions, Spanish is spoken by the majority. Large cities often have a “Chinatown” area, where Chinese languages dominate.

There are many who view this multilingualism as a problem. Several groups exist to promote and lobby for legislation to make English the only official language of the US. ProEnglish  is one very prominent example. You can read an essay over here outlining their philosophy.

Regardless of what ProEnglish has to say about the merits of an official language, it’s clear that their main goal is preserving a particular brand of English-speaking American culture, and forcing immigrants to assimilate entirely. ProEnglish would like to see bilingual education banned, and they think that “[b]arring translations and interpreters for standard government services” is a good idea because it “would encourage immigrants to learn English.”

There have been several attempts to introduce Official English legislation in the United States, and the ProEnglish website lists three bills that they support. Two of them, Steve King’s H.R. 997 and James Inhofe’s H.R. 464, are basically the same thing. Both bills propose to make English the official language of the United States, but they do contain reasonable exceptions to English-only rules. Situations like language teaching, criminal proceedings, emergency situations, and use of Native American languages are explicitly exempted. The third bill supported by ProEnglish is Steve Stockman’s H.R. 5473, which is worse in some ways. It explicitly tries to repeal previous legislation that allows for the government to print certain material in other languages, including material related to voting.

I’ll focus specifically on the H.R. 997 (full text) for this post, which is Steve King’s bill. In particular, I want to look at how this would be implemented. If English becomes an official language, then it follows that citizens should have some degree of fluency in English. That means you need some way of testing immigrants, and assessing their ability to use English. What kind of things should we include in a language proficiency test? What would count as “good enough” English to get along in the United States? Really think about that for a minute, and come up with some ideas. What kinds of things would you expect on an English proficiency test for immigrants to the US?

Whatever you just thought of is almost certainly better than what’s included in H.R. 997. Section 3, Paragraph 164 is called “Uniform English language rules for naturalization” and it reads:

(a) Uniform Language Testing Standard.—All citizens should be able to read and understand generally the English language text of the Declaration of Independence, the Constitution, and the laws of the United States made in pursuance of the Constitution.

(b) Ceremonies.—All naturalization ceremonies shall be conducted in English.

Section 5 “Implementing Regulations”  says this:

The Secretary of Homeland Security shall, within 180 days after the date of enactment of this Act, issue for public notice and comment a proposed rule for uniform testing English language ability of candidates for naturalization, based upon the principles that—

(1) all citizens should be able to read and understand generally the English language text of the Declaration of Independence, the Constitution, and the laws of the United States which are made in pursuance thereof; and

(2) any exceptions to this standard should be limited to extraordinary circumstances, such as asylum.

I actually do language proficiency testing, and I often deal specifically with new immigrants. I’m familiar with how this should work. I can say without a doubt that the proposal in this bill is awful.

In language teaching and testing there are four basic components of language competence: reading, writing, listening, and speaking. A good test should address all four areas. The way that H.R. 997 is written, it sounds like they are only interested in testing reading.

But forget even that. The basic premise of the test is absurd: let’s assess someone’s proficiency in Modern American English, by asking them to read and comprehend a ~250 year old legal document. That sure makes sense.

I haven’t left out any relevant text either. Reading the Constitution and Declaration of Independence is the sole grounds on which someone’s English ability will be judged. This is plainly not an English test. This is a test on American history and culture. No wonder ProEnglish supports H.R. 997.

I would have expected a reference to existing standards set up by professional language assessment organizations. Or perhaps a call to establish a new set of testing standards, based on the latest research into American dialects. (This is actually an amazingly rich area of research, thanks in large part to the work of William Labov. If you are interested in American English, you should check out his work, such as the mapping of regional dialects.)

Even if you have no experience in language testing, it should be obvious that if the test is well-designed, most native English speakers will score pretty high. If native speakers routinely fail, it’s probably not testing the right thing.

So let’s think about a test, based on someone’s ability to read and understand the Constitution and the Declaration of Independence. I’m not sure that your average native speaker would do so well. American English has changed a lot since these were originally written.

First, a quick look at the Declaration of Independence (full text).

One thing that will pop out at you is that capital letters are sprinkled throughout the document in a way that’s really inconsistent with our current spelling practices. There are instances of both “form” and “Form”, “right” and “Right”. The word “Happiness” only appears with a capital, but never at the beginning of a sentence. This was fine in the 1700s, but it is certainly not considered acceptable today. If you stumble across a website where words are capitalized at random, what’s your first thought? Is it “this is a credible document”? Or is it “here comes the crazy”?

Another thing that’s really unusual from a modern perspective is the use of the masculine everywhere to mean ‘people in general’. The famous line “all men are created equal” is a prime example of this. Today, most style guidelines recommend that you avoid this. The trend is now toward using a generics like “people” or “humankind”, and for pronouns the preferences is for singular “they” or the combination “he/she”.

There are several weird past tenses to be found throughout the document as well. For example: “all experience hath shewn, that mankind are more disposed to suffer”. A more modern rendering would be “all experience has shown…”.

But beyond these particular issues, the style of the document as a whole is very strange from a modern perspective. Consider this sentence:

We have appealed to their native justice and magnanimity, and we have conjured them by the ties of our common kindred to disavow these usurpations, which, would inevitably interrupt our connections and correspondence.

This is not totally unreadable, but there is absolutely no way that your average speaker of American English would express themselves like this. The verb “conjure” is rarely used in this sense anymore, and the terms “magnanimity” and “usurpation” have become extremely rare. Steve King sure as heck doesn’t talk like this. I’m certain that no one working at ProEnglish talks like this. Could they pass their own test?

The Constitution (full text) also contains spelling inconsistent with modern-day English that I don’t think people need to able to read. It has the same weird random capitalization, and there are a few archaic verb conjugations as in “The House of Representatives shall chuse their Speaker and other Officers”.

I also learned a new word “Emolument”, which appears three times. Merriam-Webster’s online dictionary says it means ‘the returns arising from office or employment usually in the form of compensation’. But seriously, how often does this word get used in daily life? If you do use this word regularly, I’d like to know what your job is, and how common you think your experience is.

If that’s not obscure enough, how about “Erazure”? You can find that word in Article VII of the Constitution, but you can’t find it in Merriam-Webster. Is it really a good idea to test English fluency using a document with words so archaic they no longer appear in the dictionary?

Did you know that Congress can “grant Letters of Marque and Reprisal”? You didn’t know that? You don’t even know what a Letter of Marque is? Well, I guess you’d fail this English test, even though that has nothing to do with being fluent in English and is utterly irrelevant to modern life.

 

This is a test intended for people seeking naturalization. It should test someone’s ability to get along in society. Reading and understanding the Declaration of Independence or the Constitution is completely unnecessary for most people, under most ordinary circumstances. Along the same lines, being able to understand the entire body of US law that is “made in pursuance” of these old documents requires a specialized education. It is, of course, important to understand the laws and founding principles of the US, but I’m not convinced that this should form the basis of an English proficiency test.

A good proficiency test will look at someone’s ability to use English to accomplish things. The test should present scenarios like “call your boss and explain why you’ll be late for work”, “return a broken item to a store, explain the problem, and ask for a refund”, or “give your friend advice on how to prepare for an event”. If you aren’t able to use English to do any of these things, you’re going to have a hard time living in the United States, or anywhere else that uses predominantly English.

Suppose that a test scenario asks someone to talk about a birthday they remember. Based on my experience of over 5,000 tests, I would imagine a hypothetical fluent speaker would say something like this:

Once I organized a birthday dinner for my friend. He was turning 35. I knew how much he liked this one restaurant, so I booked a table there. I had heard the restaurant served a particular dessert, so I made sure they would have some for that evening.

A hypothetical less-fluent speaker, telling the exact same story, might say something like this:

Once I make a birthday dinner for my friend. He was turned 35. I know he likes some restaurant, so I make a table there. The restaurant have a nice dessert, so I ask if they have it that night.

Someone with very little English might say:

My friend he 35. He like eat. Eat very sweet. We go place together to eat. Together for his birthday.

The first person uses a variety of tenses and aspects to talk about the relative order of events, and knows specialized vocabulary like “book a table”. The response is clear and precise. The second person uses only simple present and simple past, and details about the order of events is lost. The third response contains numerous grammatical errors, and the vocabulary is basic and limited.

A response like this tells me a lot about someone’s ability to use English, and much more than I could learn from having them answer questions about the Declaration of Independence. I know that the first person could get along with ease in an English speaking area, while the second person may occasionally have trouble being understood. The third person is going to face significant daily challenges. A thorough test will contain a number of different scenarios, requiring a variety of different lexical items and grammatical constructions, so that we get a global picture of someone’s capabilities.

Someone’s ability to use language can be measured on several different levels: is their pronunciation clear or is it hard to make out some words? Do they use a wide variety of words, or do they repeat a lot of basic vocabulary? Are they making frequent grammatical errors? Does their response actually address the scenario, or is it mostly unrelated? A well designed test will have some way of rating these components independently.

There are several publications out there with very detailed descriptions of how to measure language proficiency. A big one is the Common European Framework of Reference of Languages. In Canada, we have the Canadian Language Benchmarks, which can be used for testing either French or English. In addition to these frameworks, there are many different language proficiency tests that exist based on these, and other, standards.

Note that this is quite distinct from administering a grammar test. We don’t want to ask people to identify gerunds, or make them explain to how to turn actives into passives. We want to test someone’s ability to use English, not describe English. Basing a test on explicit grammatical knowledge is a bad idea for the same reason as basing it on someone’s ability to interpret the Constitution. Grammatical analysis is not something that is useful for everyday living, and it is not something that most native speakers can do properly either. Even the people claiming to be grammar experts aren’t very good at it.  In fact, second language speakers are often better at grammar tests because they have to do grammatical analysis as part of learning English, while native speaker often receive no such training.

 

Organizations like ProEnglish, and people like Steve King, are trying to appear as though they have the interests of immigrants at heart. They just want to make sure everyone can use English and get along together. However, their proposals focus too much on knowledge of American culture, and lack any serious metric for evaluating English proficiency. Tests would be biased in favour of highly educated people from certain Western countries. Furthermore, many proposals seek to remove helpful services from people with less-than-perfect fluency, such as government translation. If this is really about English proficiency, and not just keeping out them foreigners, Official English advocates need to do some homework and find out what a proper test looks like.

Advertisements

2 Comments

Filed under ESL, Linguistics

2 responses to “How (not) to test language proficiency

  1. Emoluments? Salary or wages to the rest of us. Not difficult, but I did have to ask decades ago when I first heard it. Rather archaic English; but there are so many such words.

    English grammar is simple, compared to many other languages; the problem with English is that there are so many words; words with similar, but not quite the same meaning. Think of a bucket and a pail, for example.

    A further point—or should that be farther—is that bilingualism is good for the brain, discouraging senility.

    Like

  2. A good and valuable information

    Like

Tell the world what you think!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s