Survey design

SURVEY DESIGN

Knowing what the client wants is the key factor to success in any type of business. News media, government agencies and political candidates need to know what the public thinks. Associations need to know what their members want. Large companies need to measure the attitudes of their employees. The best way to find this information is to conduct a survey.

The Steps in a Survey Project

Following are the main steps involved in a survey building project.

Establishing Goals

The first step in any survey is deciding what we want to learn. The goals of the project determine whom we will survey and what we will ask them. If our goals are unclear, the results will probably be unclear. Some typical goals include learning more about:

·                     The potential market for a new product or service

·                     Ratings of current products or services

·                     Employee attitudes

·                     Customer/patient satisfaction levels

·                     Reader/viewer/listener opinions

·                     Association member opinions

·                     Opinions about political candidates or issues

·                     Corporate images

These sample goals represent general areas. The more specific we can make our goals, the easier it will be to get usable answers.

Selecting Sample

There are two main components in determining whom we will interview. The first is deciding what kind of people to interview. Researchers often call this group the target population. If we conduct an employee attitude survey or an association membership survey, the population is obvious. If we are trying to determine the likely success of a product, the target population may be less obvious. Correctly determining the target population is critical. If we do not interview the right kinds of people, we will not successfully meet our goals.

The next thing to decide is how many people we need to interview. Statisticians know that a small, representative sample will reflect the group from which it is drawn. The larger the sample, the more precisely it reflects the target group. However, the rate of improvement in the precision decreases as our sample size increases. For example, to increase a sample from 250 to 1,000 only doubles the precision. We must make a decision about our sample size based on factors such as: time available, budget and necessary degree of precision.

Interviewing Methods

Once we have decided on our sample we must decide on our method of data collection. Each method has advantages and disadvantages.

Personal Interviews

An interview is called personal when the Interviewer asks the questions face-to-face with the Interviewee. Personal interviews can take place in the home, at a shopping mall, on the street, outside a movie theatre or polling place, and so on.

Advantages

·         The ability to let the Interviewee see, feel and/or taste a product.

·         The ability to find the target population. For example, we can find people who have seen a film much more easily outside a theatre in which it is playing than by calling phone numbers at random.

·         Longer interviews are sometimes tolerated. Particularly with in-home interviews that have been arranged in advance. People may be willing to talk longer face-to-face than to someone on the phone.

Disadvantages

·         Personal interviews usually cost more per interview than other methods. This is particularly true of in-home interviews, where travel time is a major factor.

·         Each mall has its own characteristics. It draws its clientele from a specific geographic area surrounding it, and its shop profile also influences the type of client. These characteristics may differ from the target population and create a non-representative sample.

Telephone Surveys

Surveying by telephone is a popular interviewing method.

Advantages

·         People can usually be contacted faster over the telephone than with other methods. If the Interviewers are using CATI (computer-assisted telephone interviewing), the results can be available minutes after completing the last interview.

·         We can dial random telephone numbers when we do not have the actual telephone numbers of potential respondents.

·         CATI software, such as The Survey System, makes complex questionnaires practical by offering many logic options. It can automatically skip questions, perform calculations and modify questions based on the answers to earlier questions. It can check the logical consistency of answers and can present questions or answers choices in a random order (the last two are sometimes important for reasons described later).

·         Skilled interviewers can often elicit longer or more complete answers than people will give on their own to mail, email surveys (though some people will give longer answers to Web page surveys).  Interviewers can also ask for clarification of unclear responses.

·         Some software, such as The Survey System, can combine survey answers with pre-existing information we have about the people being interviewed.

Disadvantages

·         Many telemarketers have given legitimate research a bad name by claiming to be doing research when they start a sales call. Consequently, many people are reluctant to answer phone interviews and use their answering machines to screen calls. Since over half of the homes in the USA have answering machines, this problem is getting worse.

·         The growing number of working women often means that no one is home during the day. This limits calling time to a "window" of about 6-9 p.m. (when we can be sure to interrupt dinner or a favourite TV program).

·         We cannot show or sample products by phone.

Mail Surveys

Advantages

·         Mail surveys are among the least expensive.

·         This is the only kind of survey we can do if we have the names and addresses of the target population, but not their telephone numbers.

·         The questionnaire can include pictures - something that is not possible over the phone.

·         Mail surveys allow the respondent to answer at their leisure, rather than at the often inconvenient moment they are contacted for a phone or personal interview. For this reason, they are not considered as intrusive as other kinds of interviews.

Disadvantages

·         Time!  Mail surveys take longer than other kinds. We will need to wait several weeks after mailing out questionnaires before we can be sure that we have gotten most of the responses.

• In populations of lower educational and literacy levels, response rates to mail surveys are often too small to be useful. This, in effect, eliminates many immigrant populations that form substantial markets in many areas. Even in well-educated populations, response rates vary from as low as 3% up to 90%. As a rule of thumb, the best response levels are achieved from highly-educated people and people with a particular interest in the subject (which, depending on our target population, could lead to a biased sample.

Computer Direct Interviews

These are interviews in which the Interviewees enter their own answers directly into a computer. They can be used at malls, trade shows, offices, and so on.  The Survey System's optional Interviewing Module and Interview Stations can easily create computer-direct interviews.  Some researchers set up a Web page survey for this purpose.

Advantages

·         The virtual elimination of data entry and editing costs.

·         We will get more accurate answers to sensitive questions. Recent studies of potential blood donors have shown respondents were more likely to reveal HIV-related risk factors to a computer screen than to either human interviewers or paper questionnaires. The National Institute of Justice has also found that computer-aided surveys among drug users get better results than personal interviews. Employees are also more often willing to give more honest answers to a computer than to a person or paper questionnaire.

·         The elimination of interviewer bias. Different interviewers can ask questions in different ways, leading to different results. The computer asks the questions the same way every time.

·         Ensuring skip patterns are accurately followed. The Survey System can ensure people are not asked questions they should skip based on their earlier answers. These automatic skips are more accurate than relying on an Interviewer reading a paper questionnaire.

·         Response rates are usually higher. Computer-aided interviewing is still novel enough that some people will answer a computer interview when they would not have completed another kind of interview.

Disadvantages

·         The Interviewees must have access to a computer or one must be provided for them.

·         As with mail surveys, computer direct interviews may have serious response rate problems in populations of lower educational and literacy levels. This method may grow in importance as computer use increases.

Email Surveys

Email surveys are both very economical and very fast. More people have email than have full Internet access. This makes email a better choice than a Web page survey for some populations. On the other hand, email surveys are limited to simple questionnaires, whereas Web page surveys can include complex logic.

Advantages

·         Speed.  An email questionnaire can gather several thousand responses within a day or two.

·         There is practically no cost involved once the set up has been completed.

·         We can attach pictures and sound files.

·         The novelty element of an email survey often stimulates higher response levels than ordinary “snail” mail surveys.

Disadvantages

·         We must possess (or purchase) a list of email addresses.

·         Some people will respond several times or pass questionnaires along to friends to answer. Many programs have no check to eliminate people responding multiple times to bias the results. The Survey System’s Email Module will only accept one reply from each address sent the questionnaire. It eliminates duplicate and pass along questionnaires and checks to ensure that respondents have not ignored instructions (e.g., giving 2 answers to a question requesting only one).

·         Many people dislike unsolicited email even more than unsolicited regular mail. We may want to send email questionnaires only to people who expect to get email from we.

·         We cannot use email surveys to generalize findings to the whole populations. People who have email are different from those who do not, even when matched on demographic characteristics, such as age and gender.

·         Email surveys cannot automatically skip questions or randomize question or answer choice order or use other automatic techniques that can enhance surveys the way Web page surveys can.

Internet/Intranet (Web Page) Surveys

Web surveys are rapidly gaining popularity.  They have major speed, cost, and flexibility advantages, but also significant sampling limitations.  These limitations make software selection especially important and restrict the groups we can study using this technique.

Advantages

·         Web page surveys are extremely fast.  A questionnaire posted on a popular Web site can gather several thousand responses within a few hours.  Many people who will respond to an email invitation to take a Web survey will do so the first day, and most will do so within a few days.

·         There is practically no cost involved once the set up has been completed.  Large samples do not cost more than smaller ones (except for any cost to acquire the sample).

·         We can show pictures.  Some Web survey software can also show video and play sound.

·         Web page questionnaires can use complex question skipping logic, randomizations and other features not possible with paper questionnaires or most email surveys.  These features can assure better data.

·         Web page questionnaires can use colours, fonts and other formatting options not possible in most email surveys.

·         A significant number of people will give more honest answers to questions about sensitive topics, such as drug use or sex, when giving their answers to a computer, instead of to a person or on paper.

·         On average, people give longer answers to open-ended questions on Web page questionnaires than they do on other kinds of self-administered surveys.

·         Some Web survey software, such as The Survey System, can combine the survey answers with pre-existing information we have about individuals
taking a survey.

Disadvantages

·         Current use of the Internet is far from universal.  Internet surveys do not reflect the population as a whole.  This is true even if a sample of Internet users is selected to match the general population in terms of age, gender and other demographics.

·         People can easily quit in the middle of a questionnaire.  They are not as likely to complete a long questionnaire on the Web as they would be if talking with a good interviewer.

·         If our survey pops up on a web page, we often have no control over who replies - anyone from Antarctica to Zanzibar, cruising that web page may answer.

·         Depending on our software, there is often no control over people responding multiple times to bias the results.

Scanning Questionnaires

Scanning questionnaires is a method of data collection that can be used with paper questionnaires that have been administered in face-to-face interviews; mail surveys or surveys completed by an Interviewer over the telephone. The Survey System can produce paper questionnaires that can be scanned using Remark Office OMR (available from CRS). Other software can scan questionnaires and produce ASCII Files that can be read into The Survey System.

Advantages

·         Scanning can be the fastest method of data entry for paper questionnaires.

·         Scanning is more accurate than a person in reading a properly completed questionnaire.

Disadvantages

·         Scanning is best-suited to "check the box" type surveys and bar codes. Scanning programs have various methods to deal with text responses, but all require additional data entry time.

·         Scanning is less forgiving (accurate) than a person in reading a poorly marked questionnaire. Requires investment in additional hardware to do the actual scanning.

Basic Steps for conducting Survey

The following are the basic steps for conducting a survey:

1)     Determine the objectives of the survey. Arrange meetings, discussions with senior executives for whom the survey is to be conducted.

2)     When the objectives of the survey were pre-determined and provided by the sponsoring agency, it is still necessary to hold meetings with the responsible officers in order to seek clarification.

3)     There should be mutual agreement between the chief executives of the sponsoring agency and the survey team prior to finalization for research methodology.

4)     Preparation of research methodology with sufficient details about each part, so that an action oriented to control the activities of researchers within given time and cost is possible.

5)     Preparation of Questionnaire: There are two basic goals in designing a questionnaire for any study, to collect information relevant to the determined objectives of the study and to collect this information with maximum reliability and validity.

6)     Pre-testing: Pre-testing is a technique of examining the workability or accuracy of the questionnaire, before starting to collect the data from the sample. This step is taken before the preparation of the final questionnaire/interview schedule to be used.

7)     Sampling: There are numerous methods and techniques of sampling. In case of homogenous population a smaller percentage of samples will suffice, whereas, a heterogeneous population may demand a stratified sampling. While drawing a stratified sample it is better to give higher representation to smaller strata as compared to larger stratum.

8)     Data collection: Selection of Field Assistants/Interviews: Personal characteristics: Keeping in view the characteristics of the population under study, age, education, ethnic background and personal appearance of the candidate be given top consideration.

9)     Training: As there is no universally standard survey or interview; there can be no standard training programme. The intensity and nature of training will depend on the size of the interviewers, past experience, type and size of the questionnaire and the time and money available. Thus, the time required for training may range from one day to four days.

10)  Problems: location of the sample unit: the location of the sample is very important task for the supervisors as well as interviewer. The wrong selection of the sample will not fulfill the objective of the survey. The validity and accuracy of the data will certainly depend upon the right approach to the sample as instructed by the supervisors and project incharge.

11)  Field problem: interviewers often face the following problems:-

-         Peoples’ suspicion

-         No proper place for interviewing, disturb the interviewers.

-         Contracting leader is difficult in case of clashes.

-         Lack of ability to know language of the respondents.

-         Non co-operation from the respondents.

12) Field Supervision: the role of field supervisor is also of great importance. It is therefore, necessary that the supervisor should be the full-time, experienced, and preferably the senior members of the research organization. In order to accomplish the cited task efficiently, the supervisors should be sober, experienced, polite and responsible type of person. They should be provided with all the necessary facilities.

13) Quality Control: a daily check of the completed questionnaire followed by a prompt discussion between the supervisor and the interviewers concerned will ensure better quality right from the beginning. In absence of a real foolproof method of checking for quality control, it is possible only to avoid such situations by a categorically making it clear to the interviewers during the training that various checks will be made on their work on various stages in order to maintain an acceptable standards.

14) Data processing: The following are the major steps which are normally followed while processing data manually:

Ø      Preparation of an editing plan for checking and verifying.

Ø      Coding and filling interview schedules.

Ø      Interview schedules are verified from sample unit.

Ø      The filled in interview schedules examined for legibility.

Ø      Prepare a coding key for all questions.

Ø      Each question is given a code number.

Ø      Due care to responses from the respondent.

Ø      Transfer whole data on the coding sheet.

Ø      Simple and contingency tables are prepared.

Ø      Group data by tally method will be presented.

Ø      In simple & cross tables data presented-characteristics/variable-wise.

15)    Problems of Data Processing: There are some problems of data     processing, which need to be dealt  with due care.

   16)    Writing Research Report :

Ø      Prepare a comprehensive Outline.

Ø      Use tables, tables, charts, graph and pictures to show relationship.

Ø      Use simple language.

Ø      Be objective in presenting the research findings.

Ø      Effective communication of results.

Ø      Distinguish between technical and popular reporting.

Observational Method:

In survey research, it has been often complained that the interviewer has to contact respondents in every day activities and occupations. The respondent may, therefore, not co-operate whole-heartedly in spite of his good intentions because some other matters or obligations may bother him at the time of giving interview. In observational method, this disadvantage is not there. The observer is able to conduct his study without disturbing the respondent and is able to study the behavior of a particular respondent over even a long period of time. 

Importance of Observational Method:

According to Mr. Bailey, observation is decidedly superior to survey research as well as experimentation in collecting data on non-verbal behavior.

If we compare survey method with observational method, there are a few advantages of the observational method:

·        Observational method is superior to collect information about non-verbal behavior.

·        The researcher observes the behavior in its natural environment

Observational Method for Collecting Primary Data:

There are a number of situations where observational research method provides more relevant information. Observation technique has been used to collect such information where the overt human behavior can be observed by the trained researcher. The main quality of observation method is that the personal elements can be reduced to the minimum. While in personal interviews the personal elements of the interviewer as well as that of the respondent cannot be controlled to the same extent as in the case of observation techniques.

In observation method, even mechanical devices can be used to observe the behavior. It is a definite advantage that mechanical measuring and recording devices have greater degree of reliability. Human senses and judgments are used in observation and even in experimentation but the systematic procedures followed in these two techniques of data collection have a definite advantage. In a number of developing countries, business and social researchers are making use of observational and experimental techniques in primary data collection.

Certain advantages of the observation techniques may be listed; human error can be reduced; mechanical devices can secure more accurate data; observations may be made of actual, real conditions of occurrences; little effort and expense occurs in checking the results of observations.