Tải bản đầy đủ - 0 (trang)
Hack 9. Show Cause and Effect

Hack 9. Show Cause and Effect

Tải bản đầy đủ - 0trang

Researchershavedevelopedframeworksfortalkingabout

differentresearchdesignsandwhethersuchdesignsevenallow

forproofthatonevariableaffectsanother.Thedifferentdesigns

involvethepresenceorabsenceofcomparisongroupsandhow

participantsareassignedtothosegroups.

Therearefourbasiccategoriesofgroupdesigns,basedon

whetherthedesigncanprovidestrongevidence,moderate

evidence,weakevidence,ornoevidenceofcauseandeffect:



Non-experimentaldesigns

Thesedesignsusuallyinvolvejustonegroupofpeople,and

statisticsareusedtoeitherdescribethepopulationor

demonstratearelationshipbetweenvariables.Anexample

ofthisdesignisacorrelationalstudy,wheresimple

associationsamongvariablesareanalyzed[Hack#11].This

typeofdesignprovidesnoevidenceofcauseandeffect.



Pre-experimentaldesigns

Thesedesignsusuallyinvolveonegroupofpeopleandtwo

ormoremeasurementoccasionstoseewhetherchangehas

occurred.Anexampleofthisdesignistogiveapretesttoa

groupofpeople,dosomethingtothem,givethemaposttest,andseewhethertheirscoreschange.Thistypeof

designprovidesweakevidenceofcauseandeffectbecause

forcesotherthanwhateveryoudidtothepoorfolkscould

havecausedanychangeinscores.



Quasi-experimentaldesigns

Thesedesignsinvolvemorethanonegroupofpeople,with



atleastonegroupactingasacomparisongroup.

Assignmenttothesegroupsisnotrandombutis

determinedbysomethingoutsidetheresearcher'scontrol.

Anexampleofthisdesigniscomparingmalesandfemales

ontheirattitudestowardstatistics.Atbest,thissortof

designprovidesmoderateevidenceofcauseandeffect.

Withoutrandomassignmenttogroups,thegroupsarelikely

notequalonabunchofunmeasuredvariables,andthose

mightbetherealcauseforanydifferencesthatarefound.



Experimentaldesigns

Thesedesignshaveacomparisongroupand,importantly,

peopleareassignedtothegroupsrandomly.Therandom

assignmenttogroupsallowsforresearcherstoassumethat

allgroupsareequalonallunmeasuredvariables,thus

(theoretically)rulingthemoutasalternativeexplanations

foranydifferencesfound.Anexampleofthisdesignisa

drugstudyinwhichallparticipantsrandomlygeteitherthe

drugbeingtestedoracomparisondrugoraplacebo(sugar

pill).



DoesWeightCauseHeight?

Earlierinthishack,Imentionedawell-knowncorrelational

finding:inpeople,heightandweighttendtoberelated.Taller

malesweighmore,usually,thenshortermales,forexample.I

laughedoffthesuggestionthatifwefedpeoplemore,they

wouldgettallerbecauseofwhatIthinkIknowabouthowthe

bodygrows,thesuggestionthatweightcausesheightis

theoreticallyunlikely.Butwhatifyoudemandedscientific

proof?

Icouldtestthehypothesisthatweightcausesheightusinga



basicexperimentaldesign.Experimentaldesignshavea

comparisongroup,andtheassignmenttosuchgroupsmustbe

random.Anyrelationshipsfoundundersuchcircumstancesare

likelycausalrelationships.Formystudy,I'dcreatetwogroups:



Group1

Thirtycollegefreshmen,whoIwouldrecruitfromthe

populationoftheMidwesternuniversitywhereIwork.This

groupwouldbetheexperimentalgroup;Iwouldincrease

theirweightandmeasurewhethertheirheightincreases.



Group2

Thirtycollegefreshmen,whoIwouldrecruitfromthe

populationoftheMidwesternuniversitywhereIwork.This

groupwouldbethecontrolgroup;Iwouldnotmanipulate

theirweightatallandwouldthenmeasurewhethertheir

heightchanges.



Inthisdesign,scientistswouldcallweighttheindependentvariable

(becausewedon'tcarewhatcausesit)andheightthedependent

variable(becausewewonderwhetheritdependson,oriscausedby,

theindependentvariable).



Becausethisdesignmatchesthecriteriaforexperimental

designs,wecouldinterpretanyrelationshipsfoundasevidence

ofcauseandeffect.



FightingThreatstoValidity



Researchconclusionsfallintotwotypes.Theyhavetodowith

thecause-and-effectclaimandwhetheranysuchclaim,onceit

isestablished,isgeneralizabletowholepopulationsoroutside

thelaboratory.Table1-9displaystheprimarytypesofvalidity

concernswheninterpretingresearchresults.Theseconcerns

arethehurdlesthatmustbecrossedbyresearchers.

TableValidityofresearchresults



Validityconcern



Validityquestion



Statistical

Istherearelationshipamongvariables?

conclusionvalidity

Internalvalidity

Istherelationshipacause-and-effectrelationship?

Isthecause-and-effectrelationshipamongthevariables

Constructvalidity

youbelieveshouldbeaffected?

Doesthiscause-and-effectrelationshipexisteverywherefor

Externalvalidity

everyone?



Evenwhenresearchershavechosenatrueexperimental

design,theystillmustworrythatanyresultsmightnotreally

beduetoonevariableaffectinganother.Acause-and-effect

conclusionhasmanythreatstoitsvalidity,butfortunately,just

bythinkingaboutit,researchershaveidentifiedmanyofthese

threatsandhavedevelopedsolutions.



Researchers'understandingofgroupdesigns,theterminologyusedto

describethem,theidentificationofthreatstovalidityinresearch

design,andthetoolstoguardagainstthethreatsareprettymuch

entirelyduetotheextremelyinfluentialworksofCookandCampbell,

citedinthe"SeeAlso"sectionofthishack.



Afewthreatstothevalidityofcausalclaimsandclaimsof

generalizabilityarediscussednext,alongwithsomewaysof

eliminatingthem.Therearedozensofthreatsidentifiedand



dealtwithintheresearchdesignliterature,butmostofthem

areeitherunsolvableorcanbesolvedwiththesametools

describedhere:



History

Outsideeventscouldaffectresults.Asolutionistousea

controlgroup(acomparisongroupthatdoesnotreceivethe

drugorinterventionorwhatever),withrandomassignment

ofsubjectstogroups.Anotherpartofthesolutionisto

controlbothgroups'environmentsasmuchaspossible

(e.g.,inlaboratory-typesettings).



Maturation

Subjectsdevelopnaturallyduringastudy,andchanges

mightbeduetothesenaturaldevelopments.Random

assignmentofparticipantstoanexperimentalgroupanda

controlgroupsolvesthisproblemnicely.



Selection

Theremightbesystematicbiasinassigningsubjectsto

groups.Thesolutionistoassignsubjectsrandomly.



Testing

Justtakingapretestmightaffecttheleveloftheresearch

variable.Createacomparisongroupandgivebothgroups

thepretest,soanychangeswillbeequalbetweenthe

groups.Andassignsubjectstothetwogroupsrandomly



(areyoustartingtoseeapatternhere?).



Instrumentation

Theremightbesystematicbiasinthemeasurement.The

solutionistousevalid,standardized,objectivelyscored

tests.



HawthorneEffect

Subjects'awarenessthattheyaresubjectsinastudymight

affectresults.Tofightthis,youcouldlimityoursubjects'

awarenessofwhatresultsyouexpect,oryoucouldconduct

adouble-blindstudyinwhichsubjects(andresearchers)

don'tevenknowwhattreatmenttheyarereceiving.

Thevalidityofresearchdesignandthevalidityofanyclaims

aboutcauseandeffectaresimilartoclaimsofvalidityin

measurement[Hack#28].Suchargumentsareopenand

unending,andvalidityconclusionsrestonareasoned

examinationoftheevidenceathandandconsiderationforwhat

seemsreasonable.



SeeAlso

Campbell,D.T.andStanley,J.C.(1966).Experimentaland

quasi-experimentaldesignsforresearch.Chicago:Rand

McNally.

Cook,T.D.andCampbell,D.T.(1979).Quasiexperimentation:Designandanalysisissuesforfield

settings.Boston:Houghton-Mifflin.



Shadish,W.R.,Cook,T.D.,andCampbell,D.T.(2002).

Experimentalandquasi-experimentaldesignsfor

generalizedcausalinference.Boston:Houghton-Mifflin.



Hack10.KnowBigWhenYouSeeIt



You'vejustreadaboutanamazingnewscientific

discovery,butissuchafindingreallyabigdeal?By

applyingeffectsizeinterpretations,youcanjudgethe

importanceofsuchannouncements(orlackthereof)for

yourself.

Somethingismissinginmostreportsofscientificfindingsin

nonscientificpublications,onTV,ontheradio,anddoIeven

havetomentionontheWeb.Althoughreportsinsuchmedia

typicallydoaprettygoodjobofonlyreportingfindingsthatare

"statisticallysignificant,"thisisnotenoughtodetermine

whetheranythingreallyimportantorusefulhasbeen

discovered.Abigdrugstudycanreport"significant"results,but

stillnothavefoundanythingofinteresttotherestofusoreven

otherresearchers.

Aswerepeatinmanyplacesinthisbook,significance[Hack

#4]meansonlythatwhatyoufoundislikelytobetrueabout

thebiggerpopulationyousampledfrom.Theproblemisthat

thisfactaloneisnotnearlyenoughforyoutoknowwhether

youshouldchangeyourbehavior,startanewdiet,switch

drugs,orreinterpretyourviewoftheworld.

Whatyouneedtoknowtomakedecisionsaboutyourlifeand

realityinlightofanynewscientificreportisthesizeofthe

relationshipthathasjustbeenbroughttolight.Howmuch

betterisbrandAthanbrandB?HowbigisthatSATdifference

betweenboysandgirlsinmeaningfulterms?Isitworthitto

takethathalfanaspirinaday,everyday,toloweryourriskofa

heartattack?Howmuchloweristhatriskanyway?



Thestrengthofthatrelationshipshouldbeexpressedinsome

standardizedway,too.Otherwise,thereisnowaytoreally

judgehowbigitis.Usingastatisticaltoolknownaseffectsize

willletyouknowbigwhenyouseeit.



SeeingEffectSizesEverywhere

Aneffectsizeisastandardizedvaluethatindicatesthestrength

ofarelationshipbetweentwovariables.Beforewetalkabout

howtorecognizeorinterpreteffectsizes,let'sbeginwithsome

basicsaboutrelationshipsandstatisticalresearch.

Statisticalresearchhasalwaysbeeninterestedinrelationships

amongvariables.Thecorrelationcoefficient,forexample,isan

indexofthestrengthanddirectionofrelationshipsbetweentwo

setsofscores[Hack#11].Lessobvious,butstillvalid,

examplesofstatisticalproceduresthatmeasurerelationships

includettests[Hack#17]andanalysisofvariance,aprocedure

forcomparingmorethantwogroupsatonetime.



Evenproceduresthatcomparedifferentgroupsarestillinterestedin

relationshipsbetweenvariables.Withattest,forinstance,asignificant

resultmeansthatitmatterswhichgroupapersonisin.Inotherwords,

thereisanassociationbetweentheindependentvariable(whichdefines

thegroups)andthedependentvariable(themeasuredoutcome).



FindingorComputingEffectSizes

Thishackisaboutfindingandinterpretingeffectsizestojudge

theimplicationsofscientificfindingsreportedinthepopular

mediaorinscientificwritings.Often,theeffectsizeisreported



Tài liệu bạn tìm kiếm đã sẵn sàng tải về

Hack 9. Show Cause and Effect

Tải bản đầy đủ ngay(0 tr)

×