big-data-Application.docx

ApplicationofbigdataAbstract:Sincethemid1980s.theworldhasexperiencedanunprecedentedexplosioninthecapacitytoproduce.More,andcommunicaccdata,primarilyindigitalfmuts.Simultaneously,accesstocomputingtechnologiesintheformof(hepersonalPC.SnlarlPhone.andotherhandhelddeviceshasmirroredIhisgrowth.WiihlhcscenhancedCapabililiesofdatastorageandrapidcompulationaswellasreal-timedeliveryofinformationviatheinternet,theaveragedailyconsump<ionofdatabyanindividualhasgrowncxpcmcmially.KeyWords:Bigdata;education;application;UniverseDiscovery:UseBigdatatoanalyzehugeamountofdatastoredintheGlassplatesinthetuiitionalwayandPIX)Videusabee,pictureofIheoriginandevolvenentof(heuniverseEnvironmentalresearches:AnIrishconxnycalledTreeMetricswithelevenmilliontreesindexedonthreecontinentsdevelopsasystemusingbigdatatokeeptruckofallIhctreesinorderIomeetthelogmarketdemandwhileatthesametimeminimizingthecflectontheforest.EducationSupport:BigdatatechnologyisalsousedasanassistanceofcomputeralgorithmtotransformlargeamountofbookssimultaneouslyintonativelanguagestoI)CUerIhBeducationinunderdevelopedregions.EnergySaving:Bigdatacanhelpx>wercompaniesand(heircustomersconserveenergybygivingfeedbackmessagesFromamalfunctioningmetertothepowercompanyoralarmingcustomerswhentheyarcwastingenergy.CrimePredictionBigDataanalyticsishelpingPOliCCdepartmentstodeveloppredictionalgorithmstopreventprobableCrimCcommitments.TechnologyHadoopEcosystemVolume:largeamountofdataVariety:heterogeneityofdatatypes,representation,andsemanticinterpretation.Velocity:boththerateatwhichdataarriveandthetimeinwhichitmustbeactedupon.Veracity:referstothebiases,noiseandabnormalityindata.Isthedatathatisbeingstored,andminedmeaningfultotheproblembeinganalyzed.(Inscopingoutyourbigdatastrategyyouneedtohaveyourteamandpartnersworktohelpkeepyourdatacleanandprocessestokeep,dirtydata,fromaccumulatinginyoursystems.)Challenges:Privacy:BiSDQIaprofoundlychangextheprMenofprivacyanotherreasonwhyIheUS,xdata-drivenCtvnpaniesareM)byinfsohardtousethereviewofdataprotectionlawtoweakenit.AbigdataSOCiCIyseemsl>beinevilable.andPrVJniSCSmuch,butprivacy(pn>perlyImderSkKMI)mustbeanimportantpartofanysuchsociety.First,weneedtthinkbroadlyaboutprivacyasmorethanjust(hekeepingofasecret,butastherulesthatmustgovernPerSOnalinformation.Second,weneedU>realize(hatinformationdoesnlloselegalPrUkXtionjuslbecauseitisheldbyanotherperson.Confidentialityruleswillbeessentialinanyb设datafuture.Ttiini.wcneedtodevelopsomebigdataethicsasasociety.BigdatainthefieldofEducation:Studeinsusingonlinesystem>SystemtocollectandrecordtheStUdCnlS'onlinelearningbehavior.savinginlodatabaseFordataanalysisandprocessing,predictingthestudents,academicperformanceTheresultoftbcforecastandfeedbackforvisualprocessingToniecetherequirvmentsofthestudents*individuallearningmaterials÷Teacher¼.dminisirators.anddeveloperstimelytogivestudenlsBUkklnCCandhelpBigdatahaschangedthewayfinancialinstitutionsoperateintraditionaldatafromfourdifferentways，achievinggreatbusinessvalue。Thesefourareas(4C)include：CompatibilityConnectedness*Cost*Capitalization.Theapplicationoflargedatainfinancialindustryisgraduallyexpanding,Overseas,bigdatahavebeenfullytriedinthefinancialindustry,suchasriskcontrol,operationmanagementalessupportandbusinessmodelinnovation.InChina,theapplicationoflargedatainfinancialinstitutionsisstillbasicallyattheinitialstage.Keychallengessuchasdataconsolidationanddepartmentalcoordinationremainthemainbottlenecksthathinderfinancialinstitutionsfromtranslatingdataintovalue.Example:AnonlinestorehasanexistingSQ1.Databasethatmanagesitsauthorizationsystemandstorcf11Hit.SQ1.isnecessaryinordertoensurenolosttransactionsandfullACIDcompliance.Thestorealso(racksitssocialaclivilyusingaKEY/Valuedalabasc.ThisdalabasePrwiIkSspeedandcaneasilystorelargeamountsofdatabutPrOVidCSlessconsistencythanaSQ1.SOIUtion.However,associaldataisno<ascriticalastransactional,thissolutionisappropriate.Thestorealsoprovidesrvconuncndalionstocustomersaboutproductstheymightlike.TheyuseaColumndatabaseforthispurpose.SQ1.isnotnccessar)becausetheinformationisnotrelational.TheMinistryofEducationisresponsibleforInanagingdataaboutstudentsaround(heprovince.Theyhaveover125millionrecords(students).241millionrowsandover15yearsworthofdata.hcdataisstoredusing450tables,buttheonlyvarietyisatattributelcvcl(i.c.StandardizedtcMing,pcrtbnnancc.ccc).Theyreceivethedatafromindividualschoolsingiantchunksfourlimesayear.TheMinistrydeckledagainstusingbigdatatoolstomanagetheirdataandisstillusingaRDBMS.Ahhoughtheyhavalareamoun(sofdata,the(yeoftheirdataandIbevelocityatwhichIheyreceiveitmeansthataRDBMSisstillbest.TheyalsohavecomplianceandprivacyissuesthatmakeNoSQ1.anunsuitablechoice.EachschoolandschoolboardwassendingthedataoIheMinistryusingdifferentschemasand(hisinconsistencymadedatamanagementdifikuhanddecreasedthevalueofIhedataforBipurposes.TheMinistrylookamasterdatamanagementapproach.Overthecourseof4years,theycreatedcommonmclh<iok>giesa