|
Glossary of Research Terms in Systematic Musicology |
Quick Index: A B C D E F G H I J K L M N O P Q R S T U V W X Y Z.
See two-alternative forced choice.
See three-alternative forced choice.
The proposing of a supplementary hypothesis that is intended to explain why a favorite theory failed an experimental test. Ad hoc hypotheses are open to grave abuse -- and so researchers are encouraged to avoid them. Wherever possible, test an ad hoc hypothesis in another experiment. Further discussion.
Criticizing the person rather than criticizing the argument. Criticisms should focus on the quality of the argument. Further discussion.
analysis of variance
A technique for understanding a phenomenon by attempting to re-create it. Chemists might demonstrate that they understand some natural molecule by assembling such a molecule from scratch and showing that the molecule behaves in the same way as the natural one. Similarly, a music theorist might attempt to demonstrate an understanding of a particular style by programming a computer that generates music judged by listeners as belonging to that style.
Research employing existing data or existing information. Archival research may be based on sales records, historical documents, and musical scores.
See mortaility problem.
autophase method
A control measurement carried out before an experimental treatment.
A method for inferring subjective probabilities by having subjects bet on possible outcomes. For example, a listener might be asked to place cash bets on possible notes that might continue a melodic fragment. If listeners behave rationality, then their bets should be proportional to how likely they think the event will be.
An experiment where different groups of subjects receive different treatments. Also known as Independent groups design. Contrasts with within-subjects design.
Any systematic procedure, confound or experimenter behavior that causes a study to inappropriately favor one hypothesis over another. See also experimenter bias, positive results bias, interviewer bias, non-response bias, hindsight bias, confirmation bias, cultural bias cohort bias, sampling bias.
binaural
An experiment where subjects do not know whether are part of the treatment group, or part of the control group. Further discussion.
When the effects of one treatment are still present when the next treatment is given. (See Cozby, p.281) Carry-over effects can be minimized by leaving lots of time between treatments or by using between-subjects designs. Further discussion.
Descriptive study of a single person, historical incident, or event. Case studies can provide interesting insights into a phenomenon. However, case studies often fail to be representative of a general population, so they have a low degree of external validity. Introspective case studies are often prone to egocentric bias.
CAT
When a task is so easy that the experimental manipulation shows little/no effect. Running a pilot study is a good way to determine whether a ceiling effect is present. The ceiling effect can be reduced or eliminated by making the experimental task more difficult. See also floor effect, range restriction effect, effect size. Further discussion.
Any way of describing or characterizing typical, average, or common values in some distribution. See mean, median, mode.
A common non-parametric statistical test which compares an expected proportion or ratio to an actual proportion or ratio. Further information.
cluster sampling
A group of people who have a similar background in terms of age and historical experiences.
Differences between age groups in a cross-sectional study that are due to generational differences rather than due to the experimental manipulation. Cohort effects can often be reduced by using a more narrow range of ages. Cohort effects can be eliminated by using a longitudinal design instead of a cross-sectional design.
See cohort bias.
conceptual replication
An individual who collaborates with the experimenter, but who appears to other subjects to be either another subject or a neutral person.
confidence level
The tendency to see events as conforming to a hypothesis while viewing falsifying events as "exceptions". In carrying out observations, researchers should be systematic. Further discussion.
The failure to control an influential variable in an experiment.
A variable causing a confound.
The degree to which a test or measurement is able to accurately reflect a theoretical construct. The degree to which a test or measurement is able to accurately predict performance or outcome. See also criterion validity, validity.
The idea that reality is socially constructed. The view that reality cannot be understood outside of the way humans interact. The idea that knowledge is constructed, not discovered. Further information. Contrast with social mediation.
content analysis
The failure to take contradictions seriously. Further information.
Efforts made by an experimenter to ensure that the only differences between groups of subjects are attributable to the experimental treatments.
The failure to contrast an experimental group with a control group. control failure
A subset of the sample that does not receive the experimental treatment. A group against which the experimental group can be compared.
The happy situation where two or more approaches to testing a hypothesis produce the same results. Converging evidence is said to be strong when the results are replicated using (1) different samples, (2) different populations, (3) different operational defintions, of the theoretical concepts, (4) different experimental instructions, (5) different experimental paradigms, (6) different experimental tasks. See also replication.
The philosophy, associated with physicist Pierre Duhem, that the aim of science is not to explain in the sense of identifying "the way things actually are". The goal of science is to discover regularities in the world, and to express these in the language of mathematics ("conventions"). Science allows us to apprehend the logical order of things and to make accurate (and useful) predictions. See also problem of induction, falsificationism, positivism, normal science, methodological anarchism, and methodology without methodological rules.
A statistical value that represents the degree of similarity between two numerical variables. Correlation coefficients range between a maximum of +1 and a minimum of -1. A value of +1 means that the two variables are perfectly similar -- both values rise and fall in synchrony. A value of -1 means that the two variables are perfectly negatively similar -- when one value rises the other falls, and vice versa. A value of 0 means that there is no numerical relationship between the two variables -- the variables fluctuate independently.
A study that determines the strength of a relationship between two variables without determining whether the relationship is causal. See also third variable, and third variable problem, Contrasts with experimental study.
A method for minimizing order effects by presenting treatments in all possible orders.
The tendency to raise perpetual objections to all operational definitions. counter-operationalizing problem
The degree to which a test or measurement is able to predict performance or outcome. See also construct validity, validity.
criterion variable
An experiment which directly pits two or more theories against one another. An experiment where competing theories make contradicting predictions.
Research that compares responses between people of two or more cultures. Also called "comparative research."
A between subjects method used in developmental studies that contrasts individuals of various ages at one moment in time. Contrast with longitudinal study.
CRUDI
The inappropriate application of a concept to people from another culture. cultural bias
The failure to make a distinction that people in another culture readily make. cultural ignorance
Any mathematical procedure that attempts to fit a smooth curve to a set of data points.
Information collected in a systematic manner in order to test an experimental hypothesis. See also prospective data, retrospective data.
The tendency to ignore readily available data when assessing theories, assumptions or hypotheses. Further discussion.
The post-experiment interviewing of a subject in order to identify possible confounds, including unforeseen demand characteristics.
The intentional misleading of a subject during an experiment, in order to avoid demand characteristics or expectancy effects.
The logical process by which a conclusion (proposition) is generated from a set of initial conditions (axioms). A form of reasoning common in mathematics and logic. Contrast with induction.
degrees of freedom
Any aspect of an experiment that might inform subjects of the purpose of the study. (See Cozby, p.121) Demand characteristics can be controlled by: (1) using deception (for example, by adding "filler" questions that make it more difficult for subjects to infer the experimental question), (2) debriefing subjects at the end of the experiment, (3) using field observation, (4) avoiding within-subjects designs where all subjects are aware of all the experimental conditions. (5) asking subjects not to discuss the experiment with future participants. filler items.
The variable measured or observed by the experimenter. Contrasts with independent variable.
Various methods used to summarize and display data. These methods are used to help the experimenter become familiar with the data, and to make it easier for the experimenter to recognize irregularities or unusual properties. Descriptive statistics are typically not involved in the process of hypothesis testing. See also inferential statistics.
dichotic
Criticizing an idea because of its origin (for example, an idea given in a religious text). discovery fallacy
A technique used in perceptual research to determine whether a subject hears two stimuli as the same or different. Experiments employing a dishabituation paradigm typically repeat a stimulus until the subject becomes habituated to it. When habituation is complete then a new stimulus is introduced. If the new stimulus is perceived as the same (or similar) to the preceding stimuli then the subject will show no dishabituation. Conversely, if the new stimulus is perceived to differ from the preceding stimuli then the subject will show evidence of dishabituation or orienting to the stimulus.The dishabituation paradigm is typically used when studying pre-verbal infants or non-human animals. The paradigm is less commonly used among adults since adults can verbally report perceived similarity or difference. See also attention, passive attention, orienting response, habituation, dishabituation, heart rate, tachycardia, bradycardia, P3.
An experiment where neither the experimenter nor the subjects know who is assigned to the treatment group and who is assigned to the control group. double blind experiment
The use of a single data set both to formulate a theory and to "independently" test the theory. double-use data
Electro-cardiogram. Measure of heart-related activity including heart-rate. See also bradycardia, tachycardia.
The problem of generalizing results from controlled experiments to real-world contexts. See also mundane realism. Further discussion.
Electroencephalography ("brainwaves"). Measure of electrical activity at the surface of the scalp arising from neural potentials in the brain.
The amount of change in a dependent variable that can be attributed to manipulations of the independent variable. A large effect size exists when the value of the dependent variable is strongly influenced by the independent variable. A large effect size might be masked by floor effects, ceiling effects, and range restriction effects. Running a pilot study is often a good way to determine whether the effect size is masked by one of these limitations.
The tendency to assume that other people experience things the same way we do.
Electromyography. Measure of electrical activity at the surface of the skin arising from muscle actions.
Related to knowledge from observation. See also induction. empiricism
error detection & reduction
"No evidence is absolutely reliable, and, arguably, no evidence is theory-independent. However, the basic requirement for the scientific use of any evidence is not that it should be absolutely reliable and theory-independent, but only that it should be more reliable than the theories that it serves to confirm or disconfirm and therefore independent of those particular theories (or of any equally controversial theory)." -Dan Sperber, Explaining Culture, 1996, p.40.
The tendency to prematurely exclude competing views (see Natoli, 1997; p.151). Remember that "no theory is every truly dead." (Popper)
Any unconscious or conscious cues that convey to the subject how the experimenter wants them to respond. Expecting someone to behave in a particular way has been shown to promote the expected behavior. Expectancy effects can be minimized by using standardized interactions with subjects, automated data-gathering methods, and double blind protocols.
The claim or proposition that a researcher intends to test either using experimental methods or correlational methods. The experimental hypothesis is formulated so that all terms are operationally defined. The experimental hypothesis determines what is the population of interest.The experimental hypothesis is not directly tested. Instead, the experimenter formulates a "contrary" or null version of the hypothesis. Data are collected in an attempt to falsify the null hypothesis. If the null hypothesis is rejected, then the data are regarded as consistent with the experimental hypothesis. See also ad-hoc hypothesis, research hypothesis, post-hoc hypothesis, null hypothesis, hypothesis.
A formal approach to observation intended to test some theory or hypothesis. A method for determining the relationship (if any) between an independent variable manipulated by the experimenter and a dependent variable measured by the experimenter. The possible confounding influence of other variables is minimized by the use of control groups or through randomization. In contrast with the correlational study, the experimental method allows the experimenter to infer possible causality. ("No causation without manipulation.") See also pilot study, critical experiment.
One of several basic approaches to carrying-out an experiment. For example, between-subjects design, within-subjects design, repeated measures design, head-turning paradigm, betting paradigm, dishabituation paradigm, blind experiment, double blind experiment, internet experiment, single-subject experiment, cross-sectional method, longitudinal method, multiple baseline design, pager technique, participant-observation, pretest-posttest design, probe-tone technique, two-alternative forced choice.
experimental realism
Any unconscious or conscious action by the experimenter that biases the result in favor of the experimental hypothesis. Experimenter bias includes phenomena such as expectancy effects where the experiment inadvertently conveys cues to the subject as to how the experimenter wants them to respond.
The degree to which the experimental results can be legitimately generalized to the wider population, and beyond the narrow operational definition of terms. See also research hypothesis.
The process of observing the behavior of others. Compare introspection.
The degree to which a measurement method appears to accurately reflect a variable.
factorial design
When a subject wrongly identifies a stimulus as belonging to a target response group. See also hit, miss.
A false negative error occurs when we claim something to be false or useless or unknowable when it is, in fact, true, useful or knowable. (Also known as a Type II error.) See also false negative skepticism.
A skepticism oriented towards avoiding false negative errors. The false-negative skeptic tends to make statements such as the following:"It might well be true."In short, the false-negative skeptic tends to the view that there is insufficient evidence to reject some claim.
"It could yet prove to be useful."
"We might know more than we think."
A false-negative skeptic might also be regarded as a theory-conserving skeptic. Contrast with false positive skepticism.
A false positive error occurs when we claim something to be true or useful or knowable when it is, in fact, false, useless or unknowable. (Also known as a Type I error.) See also false positive skepticism.
A skepticism oriented towards avoiding false positive errors. The false-positive skeptic tends to make statements such as the following:"You don't know that for sure."In short, the false-positive skeptic tends to the view that there is insufficient evidence to support some claim.
"I really doubt that that's useful."
"There's no way you could ever know that."
A false-positive skeptic might also be regarded as a theory-discarding skeptic. Contrast with false negative skepticism.
The philosophy associated with Karl Popper, who argued that propositions about the world cannot be verified by observation, but propositions can be falsified. For example, no number of observations can prove the statement "All swans are white." However, a single observation of a black swan can prove the statement "All swans are white" is false. See also problem of induction, null hypothesis, conventionalism, positivism, normal science, methodological anarchism, and methodology without methodological rules.
An experiment that is conducted in natural settings rather than in a laboratory.
Descriptive observations made in natural settings rather than in a laboratory.
Questions or stimuli added to a measurement instrumet in order to mask or disguise the purpose of the experiment. See also demand characteristics.
When a task is so difficult that the experimental manipulation shows little/no effect. Running a pilot study is a good way to determine whether a floor effect is present. The floor effect can be reduced or eliminated by making the experimental task more difficult. See also ceiling effect, range restriction effect, effect size. Further discussion.
fMRI
frequency distribution
functional design
A computer program for plotting graphs from numerical data.
Galvanic Skin Response. A measure of electrical conductance on the surface of the skin. Conductance increases with increasing perspiration.
A method of sampling from a population based entirely on simple accessibility. Haphazard sampling occurs, for example, when an experimenter collects data from cooperative friends. Also called "convenience sampling." A good illustration of why haphazard sampling is unreliable is evident in the Paltauf case.
The failure to test important theories, assumptions, or hypotheses that are readily testable. Further discussion.
An experimental procedure, commonly used with infants, that determines interest in a stimulus by the infant's orienting (or failing to orient) in the direction of some sound or visual stimulus. A commonly used dependent measure in developmental experiments.
Theories and hypotheses carry moral and aesthetic repercussions. In testing theories, the burden of evidence can shift depending on the consequences of the theory. A high-risk hypothesis is one that carries great moral or aesthetic repercussions. See also low-risk hypothesis.
The ease with which people confidently interpret or explain any set of existing data. Further discussion.
Also known as a bar graph. A graphical display of data that indicates the number of occurrences (frequency) of observations within a given range of values.
Any external event, not part of the experimental manipulation, that confounds the results. A common problem in longitudinal studies that can mar the internal validity of an experiment.
Any change between a pretest measure and posttest measure that is not attributable to the experimental manipulation. In order to minimize history effects the experimenter might isolate subjects from external information. Alternatively, post-experiment debriefing can be used to identify possible confounds. Further discussion.
When a subject correctly identifies a stimulus as belonging to a target response group. See also miss, false alarm.
Holding others to a higher methodological standard than oneself. Further discussion.
A claim or proposition about the world that may or may not be true. According to Karl Popper, good hypotheses are, in principal, falsifiable. That is, in principal, one ought to be able to identify a class of observations that would be inconsistent with the hypothesis. Poor hypotheses cannot, in principal, be falsified. See also ad-hoc hypothesis, post-hoc hypothesis, experimental hypothesis, null hypothesis, unfalsifiable hypothesis.
An experiment where different groups of subjects receive different treatments. Also known as between-subjects design. Contrasts with within-subjects design.
The variable manipulated by the experimenter. Contrasts with dependent variable. See also mundane realism.
Statistical techniques whose purpose is the testing of some hypothesis. See also descriptive statistics.
The ethical principal by which participants in an experiment are given sufficient information regarding the experimental procedure to make an informed judgment when agreeing to participate.
A formal review committee charged with the task of approving proposed experimental research involving human subjects. The purpose of the Institutional Review Board is to protect the physical and psychological well-being of experimental participants.
Changes of measurement over time due to fatigue, increased observational skill, or changes of observational standards. Further discussion.
instrumentalism
interaction effect
The degree of certainty that the observed changes in a dependent variable were caused by manipulations of the independent variable -- rather than due to some confounding variable.
An experiment conducted on the internet, where participants are recruited and participate using materials disseminated via the world-wide web. See also non-response bias, haphazard sampling.
The elapsed time between the onset of one stimulus (e.g. sound) and the onset of the next stimulus.
interrupted time series design
interval scale
Conscious or unconscious influence of an interviewee's responses by the interviewer. Typically, interviewer bias tends to favor the experimental hypothesis.
The process of mental self-observation or self-examination. See phenomenology, cognitive penetrability. Compare extrospection.
Appealing to an authority figure in support of an argument. Researchers should cite published research rather than identifying authority figures. Good researchers also provide references so others can judge the quality of the supporting research. Further discussion.
item-total reliability
Just noticeable difference. Perceptual limen. JND
Likert scale
An approach to chronicling how people (or objects) change over a period of time. The longitudinal method minimally involves a pretest that establishes an initial state, and a posttest that determines a final state following the passage of some time. Although longitudinal research may take a long time, the longitudinal method is considered superior to the cross-sectional method. See also history effect.
A type of within-subjects study that makes use of the longitudinal method. Longitudinal studies may last several years or even decades. Contrast with cross-sectional method. Further information.
Theories and hypotheses carry moral and aesthetic repercussions. In testing theories, the burden of evidence can shift depending on the consequences of the theory. A low-risk hypothesis is one that carries negligible moral or aesthetic risk. Whether the hypothesis is supported or refuted has little moral or aesthetic repercussions. See also high-risk hypothesis.
The tendency to become preoccupied with significant results that have a small magnitude of effect. magnitude blindness
Same as effect size.
main effect
manipulation check
A specific technique for assigning subjects to the control and experimental groups. Subjects are organized in pairs so that each member of the pair is matched for a set of criteria (e.g., same sex, same age, etc.). One member of the pair is then randomly assigned to the experimental group and the other is assigned to the control group.
Any change over time that influences a subject's behavior. For example, a subject may become increasingly bored or increasingly hungry. A subject may become progressively more skilled at the experimental task. For longer experiments, such as longitudinal studies, a subject may become progressively more ill, gain weight, or become increasingly wise.
Any confound arising due to maturation. Further discussion.
A measure for characterizing central tendency in a set of numbers. The average value of a set of numbers, calculated by adding together all numbers, and dividing by the number of numbers. See also median, mode.
The amount by which a measurement deviates from the ideal or true value.
A measure for characterizing central tendency in a set of numbers. The median corresponds to the middle value when all values are arranged in numerical order. Further discussion. See also mean, mode.
meta-analysis
Any method of collecting dependent measures in which the subject directly manipulates a device, such as tuning an oscillator.
The view advocated by Paul Feyerabend that, in research, there is no infallible methodology. The only methodological rule that does not stand in the way of progress is anything goes. See also problem of induction, conventionalism, normal science, positivism, falsificationism, and methodology without methodological rules.
The view advocated by Jagdish Hattiangadi, that there is no absolute methodological rules that will ensure arriving at the truth. Nevertheless, methodology itself becomes more refined over time, in the same way that scientific theories become progressively more refined. Contrast with methodological anarchism. See also conventionalism, falsificationism, normal science, positivism.
When a subject fails to identify a stimulus as belonging to a target response group. See also hit, false alarm.
A measure for characterizing central tendency in a set of numbers. The mode is the most common item in a list of items. A survey of concert programs might establish that the most frequently played composer is Beethoven. We would say that the modal composer in printed concert programs is "Beethoven".When the list contains numerical values, each value may occur uniquely. It may be possible to collect the values in groups according to range. That is, to form a histogram. The mode for a histogram is that range which contains the most values. I.e., the tallest bar on the histogram. Compare mean (or average), and median.
The creation of a physical or computer analogy to some phenomenon. Modeling has four major uses: (1) Modeling can help in estimating the relative magnitude of various factors involved in a phenomenon. (2) A successful model can be shown to account for unexpected behavior that has been observed. (3) A successful model can be used to predict certain behaviors, which can then be tested experimentally. (4) A model can be used to show that a given theory cannot account for certain phenomenon, and so the theory must be judged as incomplete.
1. Presentation of a sound to a single ear.
2. Sound reproduction using a single channel.
Also known as attrition. In a longitudinal study, the bias introduced by some subjects disappearing from the sample. Experimenters try to counter the mortality problem by convincing subjects to continue in an experiment, and by investigating possible differences between continuing and non-continuing subjects. Further discussion.
multiple baseline design
multiple correlation
If a statistical test relies on a 0.05 confidence level, then, on average, a spuriously significant result will occur for each 20 tests performed. A research who carries out a large number of tests with a given set of data is likely to encounter a "significant" result that is not valid. In order to avoid the multiple tests problem, good scholars avoid excessive numbers of tests for a given data set. In addition, there are statistical techniques that allow researchers to compensate for multiple tests. If the research wants to test a number of plausible hypotheses, then it is useful to split large data sets into one or more "reserved sets." Once a promising hypothesis is identified, it can be tested using the reserved (unused) data. In general, researchers should collected data in order to test a priori hypotheses rather than collect data first, and then try a number of possible hypotheses. Further discussion.
The degree to which the manipulation of the independent variable in an experiment resembles fluctuations in the everyday world. See also ecological validity.
The belief that what IS is what OUGHT to be. naturalist fallacy
A variety of computational techniques that attempt to simulate the approximate behavior of an inter-connected set of neural units. Neural networks have succeeded in simulating a number of complete behaviors.
nominal scale
1. In survey studies, a bias introduced by the failure of some people to respond. For example, a survey related to nutrition and diabetes is more likely to be filled-out and returned by people who suffer from diabetes and people with an interest in nutrition.2. In survey studies, a bias introduced by a failure of the experimenter to contact certain people. For example, a telephone survey attempting to determine the frequency of concert-going may fail to reach people who are attending concerts (and so not at home to answer the telephone).
Experimental or correlational data for which the null hypothesis cannot be rejected.
A concept in Thomas Kuhn's characterization of science. "Normal" science consists of "puzzle solving" within an existing paradigm. That is, normal science engages in filling-in details, yet to be completed within a widely-held theoretical outlook. Normal science is contrasted by Kuhn with "revolutionary science" which aims to overthrow the existing theoretical paradigm. problem of induction, conventionalism, falsificationism, normal science, positivism, methodological anarchism, and methodology without methodological rules.
A hypothesis, formulated by the experimenter, that is opposed to the experimental hypothesis. For example, the experimental hypothesis might be that loud sounds tend to raise listeners' blood pressure. A suitable corresponding null hypothesis might be that loud sounds do not raise listeners' blood pressure more than the same sounds played quietly.
In the experimental method, the experimenter aims to falsify the null hypothesis. That is, the experimenter collects data for both a treatment group and control group, and tests whether the results are consistent with the null hypothesis. If there is no significant statistical difference between the treatment and control data, then the experimenter is unable to reject the null hypothesis, and so the data are considered inconsistent with the experimental hypothesis. On the other hand, if a statistically significant difference is found between the treatment and control data (in the direction predicted), then the experimenter is able to reject the null hypothesis, and the data are deemed to be consistent with the experimental hypothesis. more
The reformulating of a theoretical concept so that it is amenable to practical measurement or manipulation within an experiment. For example, the concept of "beauty" may be operationally defined in an experiment as "reported beauty". Similarly, the concept of a "tonic chord" may be operationally defined as "any major or minor sonority whose root corresponds to the tonic of a work, where the key is identified by more than 90% of university-trained music theorists." theoretical concept.
Any confounding effect arising from the order of stimuli or order of treatments, especially in a repeated measures design. Order effects can be minimized by either counter-balancing the order between matched subjects, or randomizing the order of stimuli/treatment for each subject. Order effects can also be minimized by using a between subjects design.
ordinal scale
A data point or value that is very different from the majority of other values. In data analysis, discarding of outliers is discouraged unless a reasonable explanation can be offered for the large deviation.
The tendency to assume that an experimental result generalizes to a wide variety of real-world situations. See also external validity. Further discussion.
A common technique in field experiments to collect information about what people are doing or thinking. Subjects carry a "pager" which is programmed to "beep" at random times throughout the day. Subjects are typically asked to record some self-observation (such as whether or not they have a tune going through their head). With a sufficient number of subjects, the pager technique can be used to collect data regarding the frequency of occurrence of various phenomena. See also introspection.
panel study
According to Thomas Kuhn's taxonomy, a dramatic and revolutionary change of world view encountered in research communities, in which a prevailing theory is overthrown and replaced by a new theory. See also normal science.
According to Thomas Kuhn's taxonomy, science done within an established paradigm. Equivalent to normal science.
partial correlation
participant-observation
Positron Emission Tomography. A technique for imaging the brain.
A formal method used in introspection. One approach to observation-based (or empirical) knowledge. See also cognitive penetrability, extrospection.
physical modeling
A preliminary or practice version of an experiment whose purpose is to help identify possible problems. Pilot studies often help the experimenter to identify such problems as ceiling or floor effects, carry-over effects, order effects, range restriction effect, maturation confounds, reactivity problems, and instrument decay.
1. A neutral or inert chemical substance that is believed by a subject to be an active drug. 2. A neutral or ineffective manipulation that is believed by a subject to have a facilitating effect. Placebos are often administered to control groups, in order to measure the effect of expectation on behavior. See placebo effect.
The positive or negative response arising from the subject's belief about the efficacy of some drug or manipulation. Further information. Further discussion.
The total, ideal group of people, events, or objects of interest in a study. Formally, the ideal set of items from which a sample is draw for study. See also shifting population problem.
A bias commonly shown by scholarly journals to publish only studies that demonstrate positive results (i.e., where data and theory agree). In order to avoid positive results bias, seek replications for suspect phenomena. Be aware of possible top-drawer effect. Further discussion.
The philosophy that regards reality as independent of the observer, and that truthful knowledge of this reality can arise from unbiased, verified observation. See also verificationism, conventionalism, falsificationism, normal science, methodological anarchism, and methodology without methodological rules.
The problem arising when a phenomenon is deemed not to exist because no evidence is available: "Absence of evidence is interpreted as evidence of absence." Further discussion.
Attempts to account for patterns after the data are collected. See also post-hoc hypothesis, and hindsight bias.
Following data collection, the formulation and testing of additional hypotheses not envisaged before the data was collected. Post-hoc hypotheses are methodologically inferior to a priori hypotheses because the experimenter is able to formulate ideas while knowing in advance how the data look. Post-hoc hypotheses are susceptible to hindsight bias, multiple tests, and double-use data.
A statistical calculation that can be used to estimate the sample size (number of subjects) need for a proposed experiment.The number of subjects needed in an experiment depends on seven factors: (1) the statistical significance level chosen for rejecting the null hypothesis, (2) whether the null hypothesis is stated with or without directionality, (3) the anticipated effect size, (4) the probability chosen by the experimenter of making a Type I error (i.e., statistical power), (5) the number of subgroups required in the experiment, (6) the anticipated subject attrition or mortality (in longitudinal studies), and (7) the reliability of the subject's responses.
prediction
predictor variable
preference rule
The tendency to rush into an experiment without first familiarizing yourself with a complex phenomena. In order to avoid premature reduction use descriptive and qualitative methods to explore a complex phenomenon. Use explorative information to help form testable hypotheses and to identify plausible confounds that need to be controlled. Further discussion.
The practice of representing others to themselves. (Natoli, 1997; p.151). To avoid presumptive representation, exercise care when portraying or summarizing the views of others -- especially when your portrayal causes a disadvantaged group to lose power. Further discussion.
An experimental approach or experimental paradigm which compares an initial state with a subsequent state. For example, an experiment might involve training people to carry out some task. The difference between a pretest and posttest allows a measure of the degree of learning that has taken place.
The tendency for a response to be influenced by the first events experienced. See also recency.
The likelihood of some event or observation. Probabilities are expressed numerically, and range between zero (impossible) and one (entirely certain). In experimental studies and correlational studies, the aim is to calculate the probability of the null hypothesis arising by chance for a given data set. If the null hypothesis is determined to have a low probability of occuring by chance, then the null hypothesis is rejected and the data are regarded as being consistent with the experimental hypothesis.
A technique by which a listener's musical experience can be probed at a particular moment in time. A musical context is presented -- such as several chords or the initial notes of a melody. Following the context, a single tone or chord is played, and the listener is asked to judge the tone or chord according to some criterion. Often, the listener is asked to judge how well the tone or chord "fits" with the preceding musical context. The contextual passage is then repeated and a different probe tone or chord is played. Following each presentation, the listener is asked to judge how well the new tone or chord fits with the preceding context.In probe-tone experiments, a dozen or more repetitions of the contextual passage may be presented -- each presentation followed by a different probe. In this way, a detailed picture can be assembled concerning the listener's musical judgement at that moment.
In some cases, exhaustive experiments are carried out to trace the changes in the listener's experience as the music progresses. For example, the first three notes of a melody may be played, followed by a probe tone. This procedure is repeated until a large number of continuation tones have been probed. Then the first four notes of the melody are played, again followed by one of several probe tones. This procedure continues for the first five notes, six notes, and so on.
The probe-tone technique has been used to trace in detail such phenomena as how a modulating chord progression begins to evoke a different tonal center. The probe-tone technique was devised by Roger Shepard and has been extensively used by Carol Krumhansl. See also Krumhansl and Kessler key profiles. probe-tone technique
The problem (identified by Hume) that no number of particular observations can establish the truth of some general conclusion. falsificationism. Further description.
Prospective data is data that is not yet available to the researcher. Prospective data includes data that will be collected in the future, but prospective data also includes existing data that a researcher has not yet seen -- such as data published in a forgotten article, or manuscripts in an overlooked archive. In contrast with retrospective data, prospective data allows researchers to more rigorously test hypotheses by attempting to forecast or predict properties of yet-to-be-collected data.
quota sampling
random assignment
random error
randomization
randomized response technique
Failure to vary an independent variable over a sufficient range of values -- with the consequence that the effect size looks small. Running a pilot study can help establish a suitable range of a variable. See also ceiling effect, floor effect, effect size.
ratio scale
reaction time
reactivity
When the act of measuring something changes the measurement itself. Further information.
The tendency for a response to be influenced by the most recent events. See also primacy.
The explanation of complex phenomena as merely the interaction of simpler underlying phenomena; explanation proceeds by accounting for complex wholes in terms of simpler components. Contrast with holism.
The tendency to interpret regression toward the mean as an experimental phenomenon. In order to avoid regression artifacts, don't use extreme values as a sampling criterion. Also use a control group (such as scrambling orders) to compare with the experimental group. Further discussion.
regression equation
regression to the mean
Falsely concretizing an abstract concept (e.g. regarding spatial representations of pitch structure as mental representations). reification
The belief that no idea, hypothesis, theory or belief is better than another. relativist fallacy
The degree to which a measurement produces consistent results. See also test-retest reliability. Contrast with validity.
reliability problem
repeated measures
See within-subjects design.
Finding the same results in a subsequent study. See also replication study.
A study carried out in order to determine whether a previous research finding can be duplicated. See also conceptual replication, replication.
The failure to make the fruits of your scholarship available for the benefit of others. research hoarding
The hypothesis about the population on which the experimental hypothesis is based. Showing that the sample data is consistent with the experimental hypothesis provides support for the research hypothesis. See also external validity.
response set
response time
Retrospective data is data that is already in-hand -- data that is known to the researcher. Contrast with prospective data.
role-playing
A collection of items drawm from some population. The population may consist of events, objects, people, etc. See also sampling, sample size. Contrast with subject.
The number of people or elements sampled from the population. sample size
The process of selecting people or items from a population. The goal of sampling is to select a group that is representative of the general population in all respects, and to select a sufficient number (sample size) that permits some degree of confidence in the statistical inferences to be made. See also sampling bias, haphazard sampling, quota sampling, simple random sampling, population, external validity, mortality problem, experimental method, correlational method, degrees of freedom, shifting population problem, subject.
Any confound that causes the sample to not be representative of the pertinent population. Sampling bias can be minimized by using random sampling. If there are identifiable sub-groups use stratified random sampling. Where possible, avoid "convenience" or haphazard sampling. Further discussion.
selection differences
sensitivity
The tendency to try to interpret every perturbation in a data set; a failure to recognize that data always contains some "noise". The sensitivity syndrome can be side-stepped by using test-retest and other techniques to estimate the margin of error for any collected data. Report chance levels, p values, effect sizes. Beware of hindsight bias. Further information.
sequential design
The tendency to reconceive a sample as representing a different population than original conceived. Further discussion.
significance level
significant result
simple random sampling
simulator group
single-subject experiment
The human disposition to resist learning new scholarly methods that may be pertinent to a research problem. skills neglect
An idea, belief, or "fact" that arises by social convention. For the social constructivist, an idea like "motherhood" is not some natural state or category. Rather, "motherhood" is an idea that is created and sustained by various social mechanisms, such as the media, educational curricula, religious beliefs, politicians, and others who hold power. See also constructivism. Contrast with social mediation.
The idea that reality is not perceived in a neutral or nativist manner, but instead that what we experience are social conventions or constructions. See also social construction. Constrast with social mediation, social mediation of reality.
The idea that our knowledge about the world is filtered through socially and culturally influenced lenses. The view that there exists an independent external reality, but that our apprehension of that reality is vauge, indirect, and biased. A weaker form of constructivism or social construction. See also social mediation of reality.
The idea that our perceptions are mediated by social conventions and constructions, but that these constructions do not determine reality. That is, perception is mediated by social conventions, but reality is not dictated by social conventions. See social mediation. Contrast with social construction of reality.
Exploring a phenomenon without ever testing a proper hypothesis. Further discussion.
standard deviation
statistic
statistical regression
stratified random sampling
stratum
A participant in an experiment. See also maturation, population, sample.
survey research
syllogisms
systematic observation
systematic variance
t-test
testing effect
The consistency of a subject's responses when repeating a test, or when responding to a repetition of an earlier stimulus. A technique used to measure reliability.
Transformational-Generative Grammar.
A broad term or concept that is central to some theory -- such as "accent," "sadness," "meter," or "popular."
A theory typically makes a claim about the interrelationship of such concepts. For example, a theory might claim that slower harmonic rhythms are associated with greater chromaticism. In attempting to test this theory, one must first reconceive of these concepts in ways that make them measurable. For example, the theoretical concept chromaticism might be operationally defined as "the percentage of notes within a given passage that lie outside of the prevailing key as identified by a minimum of three out of four professional music theorists."
It is understood that such operational definitions do not entirely capture the meaning of the corresponding theoretical concept. Rather, the operational definition at best provides a rough estimate. Researchers may properly disagree about the merits and limitations of various operational definitions. In the best studies, a researcher might investigate the reslationship using several alternative operational definitions. Converging evidence is found when the predicted relationship occurs no matter what operational defintions are used.
See also theory, hypothesis, operational definition,.
An account, story, or proposed explanation for some phenomenon. The advancement of knowledge relies on (1) the creative generation of new theories, and (2) the testing and discarding of theories that do not adequately account for the phenomenon in question. Theories may be discarded for two reasons. One reason is that the theory fails to be logically consistent -- as where the theory contradicts itself. A second reason is that the theory fails to correspond in some non-trivial way to observations about the world.
According to Karl Popper, theories may be broadly divided into two kinds. Some theories are constructed in such a way that observations of a certain sort would contradict the theory, and so render the theory implausible. Other theories are constructed in such a way that no observation could, in principle, ever contradict the theory. That is, the theory could never, in principle, be falsified. Popper dubbed the first type of theory "scientic theories" and the latter "non-scientific" or "pre-scientific" theories.
Theories typically make claims about the interrelationships between theoretical concepts. In empirical research, theories are used to make predictions about what might, or might not, be observable. These predictions are stated in the form of hypotheses that may then be tested using either an experimental or correlational method.
According to Popper, the advancement of knowledge is served when the researcher (1) reformulates a previously unfalsifiable theory so that it is stated in a form that is, in principle, falsifiable, and (2) where experimental observations are made in an effort to falsify and existing theory.
See also hypothesis, falsificationism, theoretical concepts, operational definition, converging evidence.
A hidden or invisible variable that accounts for a strong correlation between two variables. The scenario that establishes the fact that that correlation does not allow inference of a causal link.
By way of example, there is a strong correlation between consumption of ice cream and death by drowning. Whenever ice cream consumption increases, drowning deaths also increase. However, eating ice cream does not cause people to drown (or vice versa). Hot summer days cause people to both swim and eat ice cream. Drowning deaths increase when more people swim. Hot summer days is the "third variable" -- the potentially invisible causal variable -- that is responsible for the strong correlation between drowning deaths and ice cream consumption. See correlational study, third variable problem. Contrast with experimental method.
The presumption that two correlated variables are causally linked; such links may arise through an unknown third variable. The third variable problem can be circumvented by avoiding interpreting correlation as causality. Carrying out an experiment (where variables can be directly manipulated) can test notions of probable causality. Further information.
three-alternative forced choice
top-drawer effect
true score
Research carried out on identical twins. When twins have been separated at birth, studies permit researchers to infer aspects of behavior that can be attributed solely to environment or solely to heredity.
two-alternative forced choice
A Type I error occurs when we claim something to be true or useful or knowable when it is, in fact, false, useless or unknowable. Also known as false positive error. See also false positive skepticism.
A Type II error occurs when we claim something to be false or useless or unknowable when it is, in fact, true, useful or knowable. Also known as false negative error. See also false negative skepticism.
The formulation of a theory or hypothesis which cannot be, in principle, falsified. See also falsificationism. Further discussion.
A prejudice against the possibility of cross-cultural universals. Further discussion.
A measurement made without the subject's awareness. A measurement made without the subject knowing that the measurement is being collected for an experiment.
The degree to which a test measures what it purports to measure. For example, a test intended to measure "musical skill" would be deemed invalid if it really only measures "finger movement speed". Validity may be regarded as the degree to which an operationalized measurement accurately reflects the target theoretical concept. See also operational definition. Contrast with reliability.
When an operational definition of a variable fails to accurately reflect the true theoretical meaning of the variable (See Cozby, p.31). Further discussion.
variability
variable
variance
The view, promoted by the Vienna Circle and the Positivists, that hypotheses can be verified by observation. This view was challenged by Karl Popper, who proposed that observation can only be used to falsify hypotheses. See also problem of induction.
An experiment where each subject receives all treatments. Also known as a repeated measures design. Contrasts with between-subjects design.
A standardized way of characterizing a set of numerical data. A mathematically transformed value, so that a measure has a mean of zero and a standard deviation of one. Further information.