I think it's because we've become a society of "instant gratification," and so expect fast results from others.
If you talk to anyone in an old folks home, you'll learn that in general, first generation immigrants ALWAYS strugged with English and usually defaulted to their own language. They would, however, strongly encourage their children to learn English, and their children generally translated for them. Then that generation would be so embarrassed by their parents' "ignorance" that they wouldn't teach the original language to the third generation (and, because of that, could talk "behind the children's back" by defaulting to French, Swedish, Italian, etc). I know story after story about this--the Germans in the Hill Country of Texas, a friend who's Cajun (and whose great grandmother still doesn't speak English), and my father's childhood friend, "Swede" Davis.
The people who think English fluency should be a requirement of citizenship feel threatened by the influx of immigrants, and don't like being around people they can't understand "in our own country." But the real root cause impatience, and thinking the immigrants aren't speaking English because they're just being stubborn. Or, as a racist might put it, "uppity."