How To Draw Inference From Survey

The chief objective of online engagement is to streamline civic communication in communities past improving how local government officials and residents communicate. Augmenting the status quo by hearing from those they wouldn't otherwise, getting more thoughtful feedback, or automating sentiment reporting to save city staff time over traditional practices should be the goal, and nosotros hope those pursuing online communications notice these to be truthful. These tools should meliorate current engagement practices, unify digital date channels and supplement traditional in-person events.

With that said, every bit the audience grows larger and more representative, beyond merely providing a tool for more residents to participate, results can also begin to behave statistical significance. Depending on your margin of error tolerances and representativeness needs this could happen with a city's first batch of questions. Nosotros've seen several cities hitting this threshold within days of launching, and while not the norm, should be something you lot're prepared to explicate.

statistical_validity_tracker_polco

While we hope any results from online civic date add value by bringing more thoughts to the conversation, we besides want to provide confidence that these results can be, and ofttimes are, statistically sound. Simply because a survey was done online or was less expensive than traditional statistical surveys does not mean its results are less valuable. There are often times that online results are as skilful, if not amend, than traditional analog surveys, and those situations are becoming more frequent as fewer residents answer landline polls or answer knocked doors.

The post-obit guideline provides a helpful starting point on how and when to draw statistical inference from results.

Assessing the Statistical Significance of a sample. The statistical significance of a sample is not a stock-still percentage of population. Instead, the significance scales nonlinearly with population size. For a small city, a significant sample could exist 250 respondents. For a city 100x that in population the significant sample could be 3x, or 750 respondents. The proper sample size besides depends on the margin of fault local officials are willing to tolerate.
Assessing appropriate Margin of Error and Confidence tolerances . How much margin of mistake a metropolis official should be willing to tolerate in guild to cite results depends on perceived or expected variation in sentiment. If a survey is 80% to 20% with a fair amount of results reported, then a city official can tolerate a higher margin of error and be relatively confident that the majority favors the position. A 10% margin of error should exist sufficient in this case, and hence a smaller sample size is sufficient to incorporate a statistically significant event for the city. If a survey is 51% to 49% with a fair amount of results reported, so nosotros should have a much tighter margin of error tolerance before confidently terminal that a majority favors 1 position. A smaller margin of fault needs a bigger sample size to cite the results confidently.
Assessing the Representativeness of the sample. Depending on the question at hand or the results a city official might want to cite, beyond simply the sample size, the representativeness of a sample is also often important. The appropriate fashion to do this is to decide which demographic or geographic breakdowns are important to cheque representation for. A urban center may want to cheque past age deciles, by district, by gender, or by other subsets. A point of circumspection is that given the nonlinear relationship between statistically significant sample size and population information technology can be very difficult, for any survey or polling method, to generate a statistically meaning sample for every subset, and even harder for all combinations (centre-aged females in District ii, for instance). This makes choosing which dimensions to seek representativeness important. On Polco, results and representativeness are viewable (and therefore assessable) in real fourth dimension. This means outreach can be adjusted to ensure the right people are participating existent time, and our squad has resources and information on real-time outreach aligning.
Assessing Option Bias and Comparing polling methodologies . 2 primary methods of surveying (or polling) exist and an understanding of their tradeoffs is helpful. Polco, for example, falls into the second method
1. Method 1: Probability Polling: Surveyor draws a random representative sample from the population, ordinarily registered voters of an area. Then the surveyors survey the sample via the means they have to observe these individuals: most commonly via landline phone calls, pre-bundled focus groups, or knocking doors.
  1. Pros & Cons: Traditional and tested. Suffers pick bias on who has landlines and answers landline calls or knocked doors. This bias is increasing in fourth dimension. Expensive. Slow. Limited meta-information insights.
2. Method 2: Not Probability Polling: Surveyor casts a wide net to collect sentiment from as many respondents as possible, creating a large sample, typically online. The surveyor conducts appropriate re-weighting and/or sub-sampling to account for differences between sample demographics and known population demographics.
  1. Pros & Cons: Newer. Can suffer less choice bias, and the corporeality of selection bias from who'due south online decreases every year. Currently ~lxxx% of U.s.a. adults are on social media, with a higher percent having online admission. Faster, less expensive. Also yields insights on social mapping of data flows and network effects on preference formation.