Question for Jonathan Berry

[Peter McKillop]

Your memory, while failing, is a little generous, I think. The Gallop survey is here
http://members.shaw.ca/berry5868/gall.htm

Jonathan, referring to John Coleman's post/link from another thread, what was the statistical significance of the pollster's findings? Was the pollster just having a 'look-see' or were the results considered applicable to the population as a whole (subject to their error factor)?

[Jonathan Berry]

Re: Question for Jonathan Berry

Jonathan, referring to John Coleman's post/link from another thread, what was the statistical significance of the pollster's findings? Was the pollster just having a 'look-see' or were the results considered applicable to the population as a whole (subject to their error factor)?

Not sure that I understand your questions, Peter. Failing that, I think that the link (
http://members.shaw.ca/berry5868/gall.htm
)
answers them already.

I have not looked at the other thread.

[Peter McKillop]

Re: Question for Jonathan Berry

Not sure that I understand your questions, Peter. ...

Well, for example, the 4th table in the linked report states that 2.1% of those interviewed played chess at least once per week. Was it Gallup's intention that that percentage could be extended to the Canadian population as a whole?

[Jonathan Berry]

Re: Question for Jonathan Berry

Well, for example, the 4th table in the linked report states that 2.1% of those interviewed played chess at least once per week. Was it Gallup's intention that that percentage could be extended to the Canadian population as a whole?

OK, I understand now.

Although I'm a math grad, I never took a course in Statistics (and even my Number Theory probably isn't up to snuff), but as I understand it, if you do a survey, your confidence level depends upon what portion of the total you asked the question. The bigger the portion, the higher the confidence. AFAIR, there is some sort of confidence interval where the results are considered good if they are correct 95% of the time to within 5% of the value. Or something like that. You can go higher than this gold standard, but there are diminishing returns because people change their minds, maybe the question is not formulated ideally, etc, etc.

It would have been nice for Gallup to have provided a number like that; I don't think they did. But you could work it out retrospectively by taking the number of national interviews (2331), and the population of Canada in 1979 aged 15 or older, then plugging the numbers into the standard formula.

The reliability of the figures divided over the various categories would be reduced from the national numbers; perhaps that is why Gallup did not want to broach their confidence intervals in the first place. They wouldn't be giving one number, but dozens.

A more general answer is that the FQE and CFC paid Gallup to do this study. Obviously, Gallup also asked 2331 people other questions too, with other studies paid for by other clients. It would hardly make sense for Gallup to do this if the results did not have significance. It would make Gallup look unprofessional, and it would reduce the value (i.e., the amount they could charge the clients) of the studies.

Sorry this is not a precise answer. But I hope that it works for you.

[Ed Seedhouse]

Re: Question for Jonathan Berry

but as I understand it, if you do a survey, your confidence level depends upon what portion of the total you asked the question.

Actually, no. For large populations like all the residents of Canada it depends almost entirely on the sample size.

For small populations the percentage of the population sampled can affect the confidence intervals if sampling is done "without replacement". If sampling is done "with replacement" then only the sample size matters. "With replacement" simply means that one of the items can be chosen as a sample more than once.

An opinion poll will be done "without replacement" but the population to be sampled is so large that the difference is insignificant and may be ignored.

All this on the assumption that the selection is random. Generally each item in the sample universe must have either the same chance of being selected, or if the sample is stratified it must have a known chance of being selected and the final results must be adjusted to reflect the odds for each strata.

interval where the results are considered good if they are correct 95% of the time to within 5% of the value.

That's the normal confidence level used. If you use higher levels of confidence, say 99%, then the confidence intervals are wider.

Mathematically you can choose any confidence interval you like and the math will spit out the probability levels for that interval.

That's for a poll done once with one question. If you add questions you are in effect taking multiple polls and the calculation of confidence intervals becomes more difficult.