From 098bb90f07e36f21e705936a4374b7606e189f8e Mon Sep 17 00:00:00 2001 From: NotXia <35894453+NotXia@users.noreply.github.com> Date: Sat, 25 Nov 2023 14:15:15 +0100 Subject: [PATCH] Add ML/DM ensemble models and regression --- .../img/ensemble_error.png | Bin 0 -> 38058 bytes src/machine-learning-and-data-mining/main.tex | 1 + .../sections/_classification.tex | 70 ++++++++++++++++++ .../sections/_regression.tex | 56 ++++++++++++++ 4 files changed, 127 insertions(+) create mode 100644 src/machine-learning-and-data-mining/img/ensemble_error.png create mode 100644 src/machine-learning-and-data-mining/sections/_regression.tex diff --git a/src/machine-learning-and-data-mining/img/ensemble_error.png b/src/machine-learning-and-data-mining/img/ensemble_error.png new file mode 100644 index 0000000000000000000000000000000000000000..a93393fcee921d79d0d4551949b2d936e5ce3f86 GIT binary patch literal 38058 zcmce;cU;cx{|4MpNz+Vd(NJkfNJFXITt!iYb}EV%nxa9;tXxG&DrpD}X=y8~K||8e zqExg?dpyV4t^4Quyq>?Gzqs9WeXh^>9>@DQj`w-(fYu(iC2N+@(9p0^_Nwd9&@76e zp`m-Rm=1qZ8FauNztEpkr5swkcyVK!b}Rmq+i~|%M_v09jxJ_rEon~L+1px5oHIXb zX=!)P%HFY`zF;Q}4L=PcJPbAx^tOLXqO)e}$`yJ>M%(B~gZYwl_b zolnAvTWnM9ch^$)CA_D~h+6N~5Zf&%U|FIg;Hy#Z>6u#)IlQHlKk0>Gr+nv`x$oIK zzBoOo_$|-nlvL3<;Zb0`gMFQxmmq~ID0;i&rVA|P)rXZ z|HG#<;Q@XX6jIdgCBKTcxt$WhuN3AAT?O*1hHFFv`BkTA`2U9=k^6N}l7EZ5Lo3Zd zF^4kE%d;I~de4sLip%>f-S7`hfby)|fi$Zs*N_e5QH|OwBKbmhUX(wO<YR}r4G6EW_ieNAg44#ezT8g*e}Naa)9esANv^R&K8R$kY9wd;Yr zlUM%5spIos+tTRP+U<-xC%ZO8{c~%LpL^J!x0BnG|E2G~eZF;N6m8&!J?yPRWl~Id zw8cIW&O6Swj}N}%^WJ*8mPRUne7&^O&qJoQ&yE!)#7C?NDLvQuaewf}*$yesYnp82 z-B%mj5j*Vv!@7Q^#;!7ImsxB_`MoXPcru=eiJ7S}CjAT>=9_D``&*rfBPa6aHj9i9 z`MPbEV>Nbz-`;#NJjb@@_7?`Xp>|q%r=N>%uG_Kf-?O^s_21)0G<;4r@odg@EByV% z#B-!&tNW-;ludnV_u;$bhw`v2oNm2R`mf4|YBQ6+z4zbWMtj7_X=Rn-0_a`LE`Vx#jgl!Gkbd2t+Lou65*WH(%bF;-t z1=gS93JvV@7-Ub?Y`pv=7#|t4LFP}yg*53ZMCrt86h1rl!mBCAh5lD<;)Vvp%##%n zGM1$@9i?|RG61Qj5FWgR+e^zUKnd~p=9E3FTuI9Oe2sABWFiM`1km)V^TJo80t_-(24b0t|iyj z>R03u-r_OE_E;~m;T4B%V+PxJ&x;DLa}`PBtm=={r&MZ(O_LIfj6hpV>LsBXWp}|k(K*uc7N;XyQ!_2Egowh>!*tarnXk>oU-|nWK_7L zGdyVMqFj-g_x5JjM`t@q3$KYpES{d7>Q`?}$grtjHXA;a!sjtAAZ_2Q80FmCfU{aO z+S?d1vaD@!`~zqG(o2yZlh*vKsw|fMO*z|pDzYbj+@}m`i}&sOljipt-h68h(iYB+ zmbNSnKE!K_Z@FQQzhZ5FLy8%FRcu&>ZDU+2BMxpP#clWp(t$|_^sLybPGGA85T?5I2=^+b_bo-WwjaqKQAGULNZ1g#37X3NI#DjT-g#JjY zh@o4@9hI@ZChs!IoSs%cVeN>0X=10w2ilmF(@uSU64M{H*_`pf;{y*bM{-7ZHaUGc z^SGujF|&CR-+RZQQ<)w!=S}NU%mn!G&Rg+9Xp-_q8-fMudg@vH=E%e@O{+0$O`r|-?QRL2x zs_DpILH#9}7pI(~?3%I#3^d*9IPcq$v z_>ScW9sSwqi6CD2+41i|{=)&D(;F;bwMjIjSubabedRvdLplHEfc6iwKV?7ruJA#{rpFn`{!f z@rC@+mi&uHl8xDJ@+fSpR})CTzs@6>{-P?xuzjE7bY?|ks z2T2s}+)RmfCwfL!ehse*x$ieU=cZazN~l6xQmJdUo}&5w_Lk=4u)u`LB~na;6m@gd zEy?$*)~Fj=L3giU_IGj#uTh50G5keuUluMpBsD_i-j~Q294T?5hUrztd=pW(S75~=#R2BtN8Z21~W+%m^J!dbpUSg4y8-Lgt>B;Ll^ZN@T z=m;V>rMt_as7}^$nSjd@gVvAozn%qaFP>Z0S1{)hJ$P}YxZa|q+@aE;jGJpsJ?YxbKXPXrIG>Lry{M{oIwqgtMl<rO)wU?A2VJ77jYtp|Y8f@=yx%b8LL}R9X!3qg;M zRr6!F{gT9!;yz{k+h@~x04vB*mrsaHpPJoP1sP1dH4 zbG5wV#)Fe#BH9t-wRfY6Zu-6Bb-($T?Zk~$5~_>N|4fK~o0;MGZ|kNoOW{-4)^20n zqNJ+!@Z!`cE$X*Rco}|veRJ5>(_YC1v)RX9oG9U4Qlge*`10Fy!^J!zIn+L1d|Dk7A; ze}o}5b^v6>p~4Vdno zeGvCH$tcI`9iQiPn;)NW!x?!rX!cLXWSXeD+jFZj7+j6TU4HrB1H}hCEWp9^|1V@b=c)FHet1MSt8Vl{d2NK)luioNq$t zA6T)4AO(#*4+Bi*W@oOw-NaM=UUK>5^l*i@v}peEH`fafA2Hf$Qzspzer?%cqQq2P zfv2*x$FwWIkJBCTBWSlW2a=8R@Dr^UfXbhrxLia_@MllF52y2oi^vcafPwFZ`<<7) z&G(zLP^B-gj@6_`58hOnVpg1`7ee!Y%@VIJHkR~%gf^>5#;MOs#;h#a1y-B~*zhLd zPGYSz{&1rH*2^qqKZ;t)f<<+UBC_$cwB~Pb(xPgvXlYn>V;+86n-h5Ww!$WrW=yki(MzX1;=Vd zSLw=B%d+uRcNzZYy5jI^9)MtD|GMAV{-geYg=|#Lf!lqT7d@tNv+}- z)^8-htIZ_z(-uEI{8TS>YWT;8wE<^S(-qx@XiP2)G8{k=#$19!Wl|h(yD*H{`0a+&+nnpq?3=Qk)FpK8*&fdO#}-a!KCkGks$-a?PuD(+akm)@ z(fUz-k3P-%6XSTN^4!GdBPO&YZtM@!u{UdQg_!}5fT?_O;@MnE)B={1)flLaNWl-x&Iy-)4^DZD>y9_b{-cCU6NXS!$ru;FcvPfzsj z8>azHC$pI{jAm_NO@XR|NuKs1ISCO?` zt9P6gJUGQ>x$NnYWXAKOJquO#y_+q-Jj%BFyXF^E~CEc#Jc~8^u7<`tPxsxE zdA{O7$<5W9>BzGcded*2{an1Smnx*WO6<_pE-4|w=L7>f{J3|L4z2j(n*LF~hHNMC z0kkrDZKdBU4*DZmb)pU>=$uK^8vLK{Zn=N=^%WD1Ewif{$F2lEL8TGpo9{eZKFV$A zP$O8a=ulV6jOfxtgG~0UzU*IKW6kayu3ZoPFpCEd-bu4-T7!eC5DGC=t$8Qk&CzJM|2>jc#x*LAbB z#rwdu_PLwu)y$HOR@LUZor?mcP}q=u+8dz#O)T|`u{)bwo6qv1kQXODtZy)PMm?rS zWuzbNsb3y4RQ(xE#`~d~A05WKMVP!&2P%dz`T5)1mZtsP;9{PApXiKa@|fu0ivB9P z4G@d2FL(Ik*O%w$K{f1{>}TVVDc+ekG29t>?&A)M$*PS9_`9u5JSKz&-fsx-sj3#z z2qq($eiX8iqV3-IvQt2nat^J9-vgA{eX7o(ZSI?_H^{JY zYuDLs-^>aGxoJALGs4}@bvGGm3rWabjx0FWsbvbHq-~F8b^!{UOS9X_>a{B~He8vx z-g0rQsXIQBU2&k0e%d~=dO#H|Uh!+ds!_>|(_bX_eVYYp^-5{!0it69)?>Ob)tf0` zs04z^;l0cE*39)iwfK`3FAU)dxnH}g9(k7piTF`<^4lK9IIf&+x)NgtP+N#&T(aqC z3Jr!+)$?3Y>i-)v3}$O}uHKoQ9V>WuAw1G^miuVl`K{9UqCl3*800QajtKnx; zM^AMu%RsW{%v1Z8d=>P;Yr;LgJjX#6(4sB#ojOV=0cB?VeO0E)mjRX%ugwDmH_ zvfgu=KTshVNBdi}gVnA$hh;W78z6y$jJL=5qx5YjIEORe$hqFKFxPF^i~Q4RtHQeH z<{V>>e8*xi2AiEXSp2ETU`1->fdRZfwNozrd1G{b7q+u|i0_>0*<$)WP(Wq;Er^>7 z08~sN!9#2uLa~0*kL+JxU5ZhAL&LAtKNFB?@_CO_1qsKfqsKYf>epCPhXL7PcA}Gh zZt_ckv1=Pcv~KFtr0gY+jq~IW4$r7PLHDalv5&&Qp)PW^t<}uBCF{)htCLgomSuMt z7&&-ZNrhD(PB_<5x^hjXtW~Ag<2!r7c##2}79Z{Y{nmCbR6UW;oLXm!>orq#bEGTy zpDDAR3_wjXlQWewIS+Ic#2&FvE}iAqHsl~PLA;@|);gi=1up%@(Zg>Y%1iHv`w0aD zyx3Nsp?b*jd(KXadZt+v<#6$0czofdtF@z}RYWjvWuZXJtzi4z>AsbrB4neP&1Nadx{wduwMWKvjr)O?2xWZE%o^q-J@@IpcQmp zO=x=MwsZhplv}PZ>-*JOx7w#iIB+;j4gf=v1{tj5pCV@$^)zM%Y@8zV`v6*arfuV0 ztAkcqTQ&`m!GiE>e(C;g%)8{YtUjm)FP)icnG?PqUsU(&07XY|VB8%;zBKeZ3z8ns)JPJkbPG5{%XyYT!|9qvqrV^+tSh?Z(616p| zn-L1s=IeqFse&aM-wxQjiuu+`X)6vck~S@*iFR-W{i0&EZA0u8((C@E3Y1rkE=fN2 z;)<1-OV?!}fNxzW;h*BQnY84)1*6c{tk2eA4(E*B{_qt=#!S3vsSleI1q_IGjo2eZ z5*egaYb-6kW}Pw9J`#F^GwipY@NCTr>eR*oq2it?JBq*TioU^~zh#~DBI?60@6|Yz76pqRVYkAo za82Z&RtLSp1Ri=2G87bCZ`ny)~US?lkmKC`oCVk-A!EQ=QPR%UPMIqZ(axi*v()7}11 z!E=0L>d6Y;=!|fbDdy!*i+g%1v*rrsX6$tm03g`5ye{^;H*mWLK$*W`dzS0qx5`ID zt(2}dtIV%wH6M$}e&z7}-Bq=rdoiVG-&JW{1)CErzx~{PZ8`ao_5PU0^x&Ro-T<@3 zuRuXvEN0uzxpj{9K24B^tm&&u40w6wcSiI$_aD+8we!hbpH`_DV~YF)$Jqq%H6@+%gfdwZ_^J>dYOr4s>n)Nh*}opS=)!P~b-K@Xp!ecnAN zZ*5zm+$9U4GvnK*9aMF=nyUxSKJ2!<>UJ6|%W#DdkI(|eze8t~MP37g)`_Ffc}AZc zPBvDO_*J?Qv=Zy`6RMHXm3PGqZ!Zxqav$lcdU3MCqGhGNqHk0`2iMp5AcvKwP~p{& z{6^spW4WeVSL!tII8AxQ&f~VtxjWj8{Dv`V_y+Lpu`=KsjrX;+c}P{;DWHC%%}U^m zyaphB3&xN(_6QwtIipQHU&Dq1x7jzZJBzx&@n9orSN~#V{qmyC=C0f2u5VZD2znR5 z*8*mM%wy{AqeaciqQ6hoJf?*pBH3Nxw_@s2IjEc~;YzaAwx8nI4tiXCKexRJDZRNs zZ}J%QBH&|}=v~pfy0X`5^%71xxMig?2cMZp5-0@@q6$qWUYs6$cd}4C!u8v==NHZ8 zI24E6cpQqDMW)*x;$z>mRcy|m9x6+z9V`)vRQw@E`~KdR&dm?>e4jRYy6SLEx2+8o zKa!NDl)&PjnyV&1v{p-&TimA2ZQvD?_mY)jJ{?hJ1D98DNxq-gJ$$&_Pi0zFMMDx1 zy2zbD$G7XpzTVPc(Je%g0V_grx#E5a{C$k!={2=c!d9KDF{T_m+O|r*itlWt^R^X} z56D2;Q}=u@N$3}gxQ&|f9cjMxvQ%b!4=afcZ$kc5*4#SX95BGslfRO!k7^ixb^U%z z!XRT&(PE#J4=#v(_08r59y6-qjg>`mX|LJNEmb{dfCfZc|L&!`<*J^O+6%(J& z(l!kafoJzIsCQ9y9v=|kzv;!n#~xc={AX%0?ny9`F>{!}yv`OZX|(3CPHbsg_SX}< zn2keaESDVQYJa>2g=t{x0X0eayFHA!W(z1Fhrz;-mZxt2r~P z&d0K2P}7#bc>^|vCGXSy?e@H#sWHW6J@Lk*MaC!xKdF&qoysRz8?{!X{t+s9Zrhc$ zUSK+X%)^K7-7ad+R{NMt9|6oME&hWHzaQnkv$jHVDJi7GC(c%+e=%k`^yH8z=!5-R zPL$IiD*ur+S9%?jsGok_-mJQ6VKOGCg{cDq&vK_z*#M(ngw0mz`})!wK;IrEUxi={ zkoa-CU!CF1V86mAF%H~(&7+)-Vae;K(iCcHO8twI0c*A{36(puzUtjI8%rd2Oq~Yk zHQr_%OG>;Q$h3LxaA4OF4gTkm4IjdH3`0m^DzZJA;F@y&_fOy5p=P^Ie2Zv^bp*#t zk1|NF%5}HKwd2l_qSi9=RDy|taS|>#;eFmGhzAdS5r4Jhu80$4_3Dzx7 z4t4u_{#T+3=$}vRR*}`nJ(_b*h+XWHIVWK1Cde{X+QVUQl=8#LeN<}*nHNhgYXnLzSy#-H#vbR5ocwl%Ga-mT8o7bW*S(vWgL^P}v)oCZQpuF?wBJKq*fOF5{nkR=@ z7GLgB{mCF}+puEs@IPA;iPT1zzW)#HT3z|z-VH1(#a4t|5~7rzS3R*(-uh%vu4EmM zarlmNa>vWg@WdiJIBz{+xg4UlrQCD+JD;#qDf?eff+(=x zGb0%4MD{<~r`!V4_-HHd&|7JhIMA0Wb;m<6IfAIZZ6(=qLn?pqz6bIvL${t@+S2n-NOx;94zGKYHNosoWvH8&Rz zTt>hvcW?7%T_d~kSsET+xawxj28{H|;3YLp0q@jrY(et+^FQNI1Z)aw>UT4;xhrX; z;kar$P&lFcNWs}Nct)%A$pW7sHCmGdTDXk^vIP^cd&stxme0UInVkEub=_}<2+m(F zKWmr?=R@E`_W))iK58t#<*E2bT-#=S#?BGVoWLK`>KKaI_Q2o|3(U_ArSzwe5B%79 zyoNt21XxMBb%~yRzo_Sn#}=mp|`Y*#^!Oj_Wpf7!a{@y;mq;}}ssr37a^KK9}Vm=~|Lc3HHGn`F@NJ+nHulUP2& zy>{!V;9Z9`sKLHI6gy~d*Ifj z>>_gBWo{D{`0Lvft`{$tJq;7iqX?D|W@zrrBn*k76fHDlcCHCgh{mcWnIK^HEk5P>YmZagLDGY_qWkqFS;TD5 zU6G*e_Qugcc@E+bHA_+1_K)0q2pI^~lnuQ1c9*{P$T-D>DK!Ep^xY?lSfoh9RZr9< zC1Si*SV(fgsDOhLKNzh(UJN)`B1W3E1_w}A>5}P@PuhjB_^?i6zNMWV%`Bh=i@qB}$~Oet z{=c3`Q&`Xy;uitni`z?rIs+47htY9%D3i#fxwU31T~gM!>v)S|Z7)6$nlgMxp3LkV zv2wV!-`>OPIen%zwM>$p)H>pS0smTRr{i=g)2x^=`bUA8<{2Utn|^zJ4PAio{8%5` zicLqQw2Z7t560k33zb9xL(jn_Fo`z%1*6oNSMYAg1mN7tCZSMd(g-nk<` zol<`_rhJo40pW2E#A@6DZ}c`N zT6vW`ogtV*6{4#eX~l6;v%%ESUS{H81p7HcLWbB<5XFv*{u<-(6N!Vx6l#kKf>C$p z$)WZVl~B9RIt`S~3>dDsH-c2II@Gw&4iLy}+!)a2YJ>6#ZG7SNKTkFnczU*i>SrJ# ztB|7e!t=z$FR<#XCN(BhsFJlY zMIr^JFh~uH>~aU+%5kj_0d>p>&AkM#kPeeD1E3olafKm-s-It%UHAnJO(> z7kAk*sJk7IU)oF+Mi@e930+{W!4QUk);PCtT-E$UJF5Q#w++jRjVf{Fzr1=IGyOz= z3dENR7YRNQ)s3S8+r*?1BB|uub2Q4V_zK>XPWFQBYo0PaUW4KixBotLE48DUc082` zACP`x&FL25fi>U}=rT6s19%b-A_1K81>VvXl3ob%As?*kih`Z_@;}1bq}fYlqUOv znSas>Bro)pLM{Q`3V!)lA^ra-?K-OBhFQ{q+$VzJ661RRV9ImnTfr%rhaV!8)^;`L zDGZR|UbLOy6pRePFaLu=Gn0BK?nN|!1Ti-U%cKfu2=agH&$G7wY;`pxzCCGo>@c6G zEdC|$*wq>#PucU|{1}#~!NJ^&{(Ch{x8uKpew?|WW_ZsWmu>r3ZD!Fg4N>%DwXCxsyTi6zige%%=UsY8Zq}Oyk^-#<7PaQz43g%;7zpDQSE|UwiJ}Gt>e}PK) z@Ltsk0ZtNo)rkdj4-hwqNsM3gaLdsJZ#pK^agLb#WFN+&<8Cgvi|Vu2%%=%Q4%Le5 zoy-ul^LIy-g;CF~^}97IQ<&y;z6SrUI6UJV+Mvv=`1!r)od9GevYCV?4wFn)^+IBMw_ zzWk#;gFE4Is6BR)mSxo@8hCj2rmfgwIPesW2k5vu>aF7_#AAAhuZ%?NhAVJ2u{RTe z)q|+XXs%wweuu=OU$f0-b!M~>MZ*CcmnK{ySK~JN)}FqnH!fY7Y@*x>34M`WPVY8T zw2wCP%YttnEbv5!Vx4zk>bMpcM zuky`C>Ev|9BS`1q9Sfrfc_0$KD;*`^6C6zH;)D=Yhn{MRvR8=NV#zb(yftz#Lt<(W z6f*x4B>h&pIVl3R7>kI^&0*7+(d)})EH`;18>~t|UasSI>*^Tr*0jm6TvR_GmKlvZ zVhrVx%E~tlIf^>6(KP5P6o*+68ZocOP;ofZb3GtAUGrd$3nKgn0_0#_S42ITh8CMO z0V)>1i@-3pX5+u`IcCEG#p7a=&;*KKJ&lvIBYe3{+B#xfDn)AY^tl@LI$+S^hc zgBS1`kL@Y=aO9)Yex`lWqF=IN;{b^~bg})JL)A^_^(&oOuiQ9CCh(8Sa~FupSqKYj zk$MgcT=a!x+!1_Tvl9f?0o0G`{?@i5ov+xWIcDFiF(JAsilVnjVUj6wt6_Nz%$@BP`G7_<8^R5N>=a)I5?=VKWGGsHX@g~3k+*uop2X68HJ9DkVk zPP3v#tsl0xEJDb0WoCn97%2*lBI9n?H|rAv5N_!Rrh`kV>ACqHQ?x^s?&?@_(y><- zY9zk&m#j19#0SAdS|P&Nl^|Bw63)r)S$ylL%zdJnLCd5^db0Rvy1)~sFusE+{v5Jk zeONH(H+VyRx} zI0_?jZWtP>2U3WFRhenu0wQm?fbEb>9cDu~epO}tl=j)92QPIf6^(!WRE(k_vQVr1 zx23a?StOV_$hmQ?OQQB^@T&da^0!%_$FCR6$B*cOuS35pG<=y30y zf7Y@xgVOW$<(50ik~kISg5@_>h*CF8WumvF^eVGHJ54-7E`5ekP%61~FAG}ozH>e` zR;k9Z^f$XPXuR>Ejh3;xIq>eRe2i%*Y@I|#Y$8M$MlHp`*Rlz_sKNW68Re`36)#eC zr?k!|fw_cntwkZF>5q+aohy=o*jb}BD`05U?mrR|Q(nHnapG`(!Q5AE>(F2wKZAj- zTjx+5@oBn-@KdUvPr&p@Typf#fUix+Jv>GWG459$%BR{ zDGluQfiXV+*81AMr-8#*IDI)G1ns^N=Ssb|eqF z6ml<2oHa%I7D8HMLqU~*PhFj>4$*o0=c!L`h~I}S%^*%LR4bVY%N;C8Mv~>fv zYnW{l4POosMoZqMZ(o|SL^&&5b=x6;@a>CGyl3@#0x$DZbgS$rY6(&1t&&?wavpki zbm?M)p0n-m2T3V`WsKGsQ$PQ{)sY}_x}EEFFkH4GBbd zGrG*?^AKq=u;T(t%hIidtTSnm7e|jp2Oa>wc3e5VG27`h=GdTo)?#fn&5o20Yw7@a zswT^U9^&FE9aF(Rr{uPCX#c!Ke1q`it^djQ9qBFlX-P4CkyNrdM5Vs`DZ40~lC_R@ zUH%}LB{Eh?VJyPXfJz-(qPbTmqvNlkaqFoXGWmGTo7ohcH(?uwHVRB-A&M3m5o;0+ zLacmLPUu4a0pwl!`}__K1R7)MU5Wd~mMFEWP-m&78)9}Qvrvpyn-0J`LI)2I0~S5F zcS`T2R{JbeAk1Kht4!Em4I8*L7eGGT}B ze2E-}QEvbS%mFO(1pyLYOm>4~2;nE!Nqq&ms^Z-Dm6A5yj&xKQ3W@~i1X0EL;k44= z>*WEzJ5E$YG{HYiD!EF3bDlRE#b8018*!D7>gHgIy3)c8W3%v@z4BI-{QNF`UkjP| zE=ao#Swvg6f)3Tb0Vb*1Y~`k-oQvZP_6&d|rpEvhuHYgG0>t-`=QYG#h}HNw&Zfxb zGRzsv-rxK5S%tb{`>fPaqTnq9>}`Ujwb-h^>d_vNCXAb~sjPa8Sp|~ce32q1h6C`% z9Qq8xRmT?alo*m!pQb3IFe$>qN=gOsW*WjzE9^uKri>Su|Cjb~=}8<*wWtm;8yJI; zi-r7~uP@>MicWMf`&}SNxt^1<3lgoq(7u?#4e7#R6=WUZM zUq_h1r;v~Rh6(wGc={fd%U%aSvLQf`ZW}0_izeC-KH4*Z~lq_w%k` ztmr~XOwgBAwk%ySkJB(kdSj3hQaTUj6jnP{mwKdrgqA{Yn#Gt8`)8~kDi2k@g!%G{ z-aQmjroOx}RmB?51c7sFn=C>*kRf|~^PNBj_l%!cQ? zV^si@MYO_0dzj&x5m_U*6NT169`Ig}-z5J6x4|<(QHlTNA#hY!3Ze^88eI))N|^@e zPV>AYaUeCcf#JNi%5N_&XU0=CjVPJV4s}@6)UjDiPQP3uYXA!wGET)Qpx34K4L#al#Pv75vBP}p%3$N zFKq3thR1uA^zqk=O$qC$-GozNoc?7pM+eIQa}>gUP|ScDthG=r&=2!)24phDf$mN# zOMA?}Wvf$&>i;W^4|ApTB zXn!xMVKKzDp@E^ljPUs2`RL}^s2klW1vB`UWOusipD`i?c9ew#7`(yfOB^odC`t~O z-uYX~bAn|~p|cfzi2a5%NY1b);FU|45!0_J9GgVfAp6IdyCb`}x*$L>E5d%J0)ByP zHh|7y(Ece%@qE1itaa`*KvfTaT&$7B#>Jx+HU=>I;LKZYb%SD%k)wxd@sf+`U!D^g zs51|sB_tT3ST6u60*q;#Zy?+(XtZTB8VqY>tpXjzDb=_CV;Pmwr(j$m`iWN=X+|@m`4jvVUlkc zC?KP6PQfB;^ex(wT$(Af9SdFnkaF|<4*U>rr@>cRYGHL$ph1)?VXc-AS42NkL#qdn-4See9gs{_){4RBWiuVgo1Xg%Aa!Yhroq)2G%ddt5*y;awoY);gd-(wnhNE+rD%{0Ia3DkknMG{Ob)N$NwxV?r2o z>*2x!XEc+PYe@hfD%X|?c_G#5(C_6-FC(L*)D^1LYF&JcoMFsj0GoNq4cJ_GzG+v_ znfK>^j}fbwn=9zrUASa@)Fqx2&f(1)qr5IHe*Wd^KL=&Qjy`{<%EeNrvh>wP&gYL- zuV8+2{W{yOO}iFZFVQ)qqB7YZ@XP&2W)WRkMt%K7rMb{|GF#k#G~6@$7(SfeKSh;^ z_`o4Zxn!Z=MA8Cs@L`BMM8@1JTwoi5bbj~xB}y~(@|$RI;igge4vj{cf|NgN|10SU zIY#Zn+6_m^(4CcHhiJ$`6_~SJo8c>K?dmK;j8a}c6--ERwaT=GgN6ntFGBU;pvLUm zjz#@>ivr}E{W7bE5aQ(_y?j_j0c)d`Zv)jY+(BjjLIO*y``WsLBlFUFAWsXU>I{V#6H1j-`@M! z`%c0CB)k7NO1gF*gA4_MGX3||zWdv5wR2!0S6ufCG@&AcpMMp>xodry8l1~@{tN)i+|N5(5IdL^6zR8wvhuz z?0IlQh!-vWUcCpBK)@a=3|-f!!(aJx>lZ_{Wm+24CG~iWU8x$SBpiv5MHauvY*qA3 zPoj$=1wJA!F+mJqI9uu%LM ztP)D-3MEUc$Q=B?pqMgXR5j7p0?UBe=&`qP0cLfCrukb_)Loqrs$hrILWu6z-y40W z5~+26ct6pqF@Eiz873GYsr35%c1WG)+|01$Y3xm3!1v1-%Umy$h!&h*!%yiR0=gs9 zH4Ru221C!8-(<67Z*9;f+4(*J8kQb!*ww>!+m9u)gzv7ur$If$dG@^37+I{c}XibH#;?J3aatP}Q7V+`jvFG&TB|G{)j!YDhq zz5N$vxkWw5PpnRsB=>TK&pR)%p*sM9GdAhk?_{zuMoZ{_*jjeWD92!ovKBo`mDzhC znot^C-Tjd1Bh3c^IS0og_(=>mENn6bNk)78x)G>m(s5dexJ!6HQ?gn`Ogq@(+HUkl zQ>RLn+}n_p(?>UpN=FZ?DtC$TqrLl0{W+$wDYva9QxT=lDx(?9?S+;sLCXFAUc;22 zEyx7uIrPLRr0!nd^^IrdB<^hBiMKId(~P%rpO2yQMRT6}JHc$+6hn*lST{N24ki?i zvQ<|T4~MNM;PQ*FiUk8w~MQvhCc61Y-p3ELaTxejm=Hwg9sA9-z)q*TMj{a>vC zwe*$TLV_Xu_P!&FA;8Kyz;=>H*8a(Q8jSUq*S$XOYXd!&%h*+5ha@~%XH|%m5L_89 zXGpLxK;xQklH`&jB#a-7uuEXBJ8}MW<7>Py%~U_JMNo@9+Y|!@DFS1D&Y02d!EzNS zKW==DeO4yYc!_S1f~L?pxs*i}UKwIIY^~Om`Y>Uihr3Hu@P1gHY@&!Y;2h$Q@>meT z$z*njs}-ld2uFCQxnbKF)QP}NCuJbWIpU%v9q)y{pZj&*`A2w^y$!)98|ewBD||1; zxMnE35-u5NmNeKW-9Ys2+tU(cTbpb}TX|qNcoVs~rSS2=M|<^*Z^tbAhW#j7Ue{LI zD3byYY=&$jd{Y$gM0cXCCR8C!>X$s+Sko)POqbtbJ$cB4>RAabKI~5V2!WU$ z*FE`JoNa<~VxFEEN?EpiKrq48;Q8_0f=-)X^pWY0tORFdi=C}{4g^jRDp}ffZnVw3 zdF34VE6I8XGDYwyWh)rRymZ?T2%z+UntRrqzM5x%Dp7s(vLHq7U(x)! zH^L5M)0uWBZQQt2qF({>t~|L^kTS6`EJqPSGjY-*16j$swp>q2akYVg| zQbJR*Ud zf_!L&T}x?tp!k!4fpIR93r@B*C3Mo(&(RezA#-P8PpkSco_Jv`{%zUCQ(s`t>oVO5 z#mf|`RC#8!AjK^O%|k%#3Juv}a44i@q9MCVvm^1!v6&f)pw2?bls1j=lfhqUw9fIc z4zlNFABO-%%;T?OqX1r!iFLKgS@v-fEBpN7~fa764AhwlkmYr^90MlF8XXHdfiq5VZv2!`t=6-qBk{};Cve7OZa;F7sG`P6*{Ym@m zjk(bJ`Al&mQ@5l#$fG}=C~`{R;@+X7T(jug6%L}RBY6&}Q1H`FOhIg4VPhgv{kdZd z<{+d=wIwx?{2HC$Ny6#~heH(Ju∓EF|+-R3` zD~8Jr?1UFc8qqx8mEmMtjWjH>Wj$YMfBf}R?a(R^3}=5v;!e45wf)42Yp@-Ki+H4u zrk`2~e08K1wt`1i&p3{$5Xj@$RY|)V305KM5moiBh=dLCr6ya`uq%L_wyadDyP0ZsZ?Y--d6ZszBje$@mplt798f*a_R~F(jkiBaJFkKTiaU%U9MQj_&KMziI9(};;Mx2R(j&d8BaU_i7>LV54 z@oJ-wao<$P9Xj)8vQAXggrBt2gwSSZwV#9;{Y zVtB|q46S2C*ar7I1z@(9nhh5h*G3>OeN9b7aouHVunzED)YpG6R3cwJ6H_J0SRy+i zi#P&h>)6n&9ryVM)US9RlP-8?BOz!Uuo6x-QUfBTvRdi*-);@&1betV1^1R&=rmM%c!m&$I6 z*w7^7f4fut^&c+?MSpyE*7~}aC|Pe{mLLk{Td`N03+zDTsi3l>4(iX>L`Zes7;5C3 zI@)W9`{CMhkuaR1%#DTafeH(o5aNBz3%VaIDBYInon%@sc}+!|>YRLzLUBF8vFf26 zHcpAlXAR#)0I0q8w$?S7_9m@_ti@2hH44beI!W~1C`fBZNb5Rw?X#7Ap#BnJB(6*{ zQfSTol|)ajno2Dk!<`POV>D^@Exa4&2cu0CYiQB3^OX4fB_2@t3dzg~oErNgYdvwp zRrLz%r}bl0p76CAQkC}HE( z!Ns^LWi1ral2fy!mk{&wNUdHB2H0~LW(mPqZKSCM@T%bCR=)XN3~_Co{i6)Y7)Qu= z@}BQ{SV%C)1>h&alj-LAMvx$(l0|ZC)zAqe7s4UgnWWFQ?~Q_+Y`&Nlvn#}8W~~2( z{;TD}+-OF!hWlhVEi130+=tI-MNEH0p}`^S^JDSpwxdWu8M!r$4?O8Srhop9*UfVU zMQq*XwkNnIu@JixiwO1>WHv@YuGTkNIs`Le5NpeO!`u092bSY(|J+<@nX4G%6JV{S3gDg3z&#H1jkMK?V zl01kz#^?s5hJV9Sj1jN>BCKhf=VFMXmUs?;u;(}Qn=z>Y*{#YNsn#S{Z2j2tiW+PW z`Hh2kV+)b2OonGmgpj=wB(caVvVz(|bObMKt^mQUB}XhcGk4b0Y7C|d!18>o-_h$F zdHb!EgtJ44Eh`Gy?e+wN+7FC}i-@?=iYqwj2ys9?*pbtlVW9t1JUUvf z5U-P^2O&g+&cLc$4VYj*z!Pvg7v^PW)*IjtUT%TomP}V@iV*aPp=4%@-m6)%7={pi zeFs=Zm^<=bRi}%PWj4tfnD+di#o>~(Lf|k4I5ajS##(xbM7Yus5{Q(438Y^MvWdKm zW@x7Kg>&!_1@kU9z-yUv9oDPtP>EVvQ@rHTLcDtJwJ%m~ ze*2p(Xhrym96N3Y{;z%PgD3w=k=@y!foN>(XE4Ht zhTP%`BQ{)IDJalg<^ohpk19zUw#_zsA{mt>!s1@~RY{?h`SU=9AbjGtB1fV;(Y~Bq z^k^+1QM?bA9?dK5V6tO#!l%ao6{!~*3P=pjk07_>)dxLT~y2T_a&x*%JIol|D z{0W@I={bMXAlVHf`}NPfpT~Z_AAF89QHt@&~St&}^ z#B%OFA#*0VrY|GcULOXJX@WSHM70-fBRAfm8WplD^c)D~$ir_Ta}Q0WpZe}ZwiO;N zwQTQw)6EYg);1IRpFt%JlWeV=9g;!GTtX?`OqWdk%aY`$;V&MBo7#C7I zRnk%#1=Gb1GBy!nhZdvs4&r*=*Bg_2ZFrJ2Hp-R~1l0Kacy1D(8;X@`v{5Dk{H>Et zOp!YVx;{VUIsm$x+I(fR)MVJ{q-!~Ee9BQfE)Gio#XE^N!WQKzXry+9wWOu=Q#900 z72r9MkH>j+2{l9DN?|UflR-9t)G^GaVKEP*1rzwu_U-OK4$ zHZ)J76h(tJg;FRIrGc_dNk~W~3Yn9kWGEd&6B`*qB}2+QPemsxQ=(FaRHQuuIn>>ArORiaC7(NmZ1!(C_fiASWGs# zGBmbX3B~%?b$s3&wiJ3b{G;Q-X}vf-lky*BQJ9IAn_?FQz&U*K8vX9I@$8`Dmq%NC$iq5ep?0j2U;!FU=7z|fK!XgQ9}QQ- z8-m1a7Ta}xq4I(s&q_y*HZYe4e=t7Kn#)Lj^?TN2COKx zaQQO=O1R*(;6lRD##i|EFHkpk8lSl;N#SOdMk`6hK27Kxb4ouKamWmfar$7pQH#kV zlW*D$cEX2drLRlLlldMRmLdD*Rx*zf5_WlxnVY%N9_bp)K*`eA*Tmi;S2I$A=2wnB z#+XgPZ}Im}d=On(_=fd^n}oOJ!e)9JsL{H{$I%R!3D{0M{u5PE?eC{gvV)IFwAVGg zX>vnnu?zWR6jS?ElWHB~@iJf<9kp;<@%?7Q+1Hw7gOQU_eKY>yhquij369iHUN5#n zMO2q}_My%tJ^q5JDBCT#*b~TvhfBR3kK6>Nd=zGz;Q4E6p*aJ=jVZjLXwRZ0e(Fg>SwyY0>RcX1CF2dlv6`;62_Tg zYDb7+@gd_Zuk6V{TIAcd@1~z3;6I<+K3jJrD3$$;B*5m1805e{#m@YRfG70 zjcP8R4a&fk2QFRpE&UhdB2xf4E#CY9df=et%CkP6b9+yz(_rE_&TkmByaE{^qt+7; z-7U8Q%MFfz2{_1lH)POVxi{BZ-#je1xnj`Tu~s}Ons1Y(-u&w8D%Ro0I-iS}B zyN8xPOFRDTy~(8BvI?Nzw=vaiT&>SipFOtc0#I*5a?zr{6V`HG{tPHA#8(ZV*n?w_ z!6<3Vd|%QiExEufYa5_>AG8u1E*KjZIP8zMX_Lxy@X$OE6zH$k9p|&jLr2Az?{L=bx zwI>^GUeK2|3QIqry{E?L>~61q1SB^d3aR6v-|^7`X0jrY$TTAAmL&VW=ImuBZx z(lXBbVpHDZhIuHlZ%(H5jT`@S;R5f&F6(PA@lEJ0W5l`E%2q8ByOY@c%PcF&fon0+``?OD_czwL02_s6CXPz?~{#+;O+{O`+EBBNUjW=KDH#&f+FWj zr*CN>O%=IJFpKXopA8|tZdA3z?G2+tNA|&j4_rjUb;Wqe_5x`5rxG-)W)>MrPbXvd z?k}rbR5TM>0tpoiIc!#6L2^D2C1=;j9gx_Q66F41u^-1M_G52~*Y=c5jM%!eAYzI@ z_G7^P{IlA!v@6e_L9YbQd3v*We}xHb22>H5@@p|O^Vso}>kD4^cKG)p zv4m;Aze3+bq{4b(zuvZi_hvQ_U9b^Hv-z26Cp^UbAlbaBx1H0|v8yJkm61a#H195jGGsfn#r>G*(!T7t=p zvE&^=r-YA^fS&Se$^(~6Z@SFSIS$Gh zrEL9ZlychkBdS}-|v(ztF3|VjXtt}{4 zX7Bf%wLi=0)-OrA4j4JI!g86Lzwr2P|0*yAV46N#=ZGr6-*bjSDwhiC3A&!KJdAz` z#aE;<%IxVy!+|A6i$b9w|0ez#Aw%+PmSfrlVC>e*TUp)LG78;Nug!Ohx0P0rHR7En zunKl_t?wP!78NTwDxjWv1{u$Fy=1;h@0$1h!rZ!`KYT1@7L1;u+%>)xTUqY(SiudD ztZtTL?daokY&>lPYR~b;N^Qe?p~Dd#3lFWooQ#dydjsrUVKf_s{4*dYKlVw&l$?CN^DRyd;v(5rLZ}1>uD=8 z`tWWO3hvu1vqi-%Ua_t(d_%DBOXe;Y{ zgm{t%@8wV;;3{X0QvRE3=XB1|Bne@pYD?||^VL6YpatI)Cdwa36Z7(0KOlvn^9#=F z92+>eyULX^5=`AY`B@h&stmqtY(b+a4T{LB+G((NMG;rIg=51%99__I8{w&{D$1uU zCmDlaFpP4?tzicmVEPSCDG{-_h|MABi>&{aV!##@88D8#NI3?(bP2v?^oCu_&B^jH&VN+s0DAXJ^wxfo{(C?u{7(;_FSgHoS{Il~1aNzvKPb9J2| zg`QJ@@IZj_Yl+liqmpNT$L$LutXX0PWfLF(_$WwVHn>Hw4OBKvds}ts^p-@7GQrL> zVdic59iT0PjN;H_Gu~$~b8!=omIk})$ndWNy5VBv8zs5`lXyb$94w-CH_N3KEFel; zL@Iq6L+#j+>$vt-scDR62o6?(Xk+Q?#tD}=QIdBVERvCF3UjHLF6IK>jP}fV);KKc&+1jY6`@2*?0e zDoM*bH1prf%2a&*I?Lt=6C*(VwMw@~;F z8@OsO`%SXqqUiZ92S!BcILyQa@R@M_z&<>YpthKbuJ5L3o%dlJ(xg;-=5L)$1*CDI zo^W+>BH9C~uNzt8xl74BjIDONy{)K8ik1D8w#xeK5j2YJ&zn1-b-Io7cx zy67Go!j&z<_B)Z}uq23*$0C}A@s2fkYyV@ z%Ferxp6A$cHAc@3iQhB+6kANI6Mx{1eGt_mODl^MGWg}W)0dhV+6cW8|49DH@*uE! zdNYOo2~1|MDF=4GG1R8td!F$Sn8j>fO$RolCQN2!o~+++9QY3#&4-PBRV^$&3|pT6 zd>bGC_%tNYmSIGi1o6W+vi*`4tF1XF0FOw}@`H~tIXQ>#^k7@CNTvV~d)l%+%>w~= zGG~9!BE%xJABclTwo|7%>(kPUcWg)#bR?e$<{X2i-VktNkz0jGcFY?n@|gYcCwg%% zKOv_{kTEg%mA>cxd5e-k=ytq8!+gQznn3Df654Gf3K6S=s9_>~Kq5tx_oeU(A1%8w znh8T!^qk+yivN2|6jc|JtZP>u6-bV3wzeJ*B$4hj5Wn=LQEVTOT!!od9Lg|>M!k*b z=^Bg;9r;I>5LSxPZjG5m+Z8JEu(b=CnF{=SO{v2Kl?4V$h|Ulpr))y*x?yD?^PCO< z5IDVlR`2x&L&rs_@W-Y~wvp%h6=pguT83dJW3(ldGq!hWB!Nu){yL#d)-OsiWi=GP5ci~f zxnPwe`13D@5(%AoRLu)v;l?73AVzZhm_7z?FszJX%mIL60(K*Qa^G}EfeK8_Ud#Cj zMd>BR^fKTU-n6~djt^7A9Loi`I=pW)qT)Y}6)V}?oy>eB6VRD5=M;`&nB-uaA|R*e zO4TZ(R6xKakeuv6#%dX0<%qXg>hXpX20nU?Dz1X2nV3$NC&gu84bfhw9;enV@-rYK z4Uf=;4TH;QYyHxg7)nh&DW9_>kVn#$hBeda{~JYB z+NsEI94x0Y&`;dPMGXa{XgxO8pZpy$AVG|y^$tUnSVJUcV^K~rLV$<}dZ7TF5#=pI@yj-HrP@}!Ctec^RrHpu#e0!a zATIy!{+qnk4R|T`To($%Dg9(OCs3+v=|TgE5EG3MXKO;V<0|Ws~l` zeq(q_AnL~VvA97(Q~!)U(%0u5MefV}__RkSVCpEt!8o|O)`X-twbedIn`DhxPB4AN z&U4{({am$0(SilOkqb=wL`o707HZxDR${?J{wORNP7o&JX-T4uW?Kqj7mjIb4rPbn zK^<`@`|A#+>miKf=bu_K{(Ks5!Z-&7No?Ss8BWO%IvAzEx~R6*IC zPpVDWBqgz=2H&BzJ4deS?INX#i&qLcYdyRTZrOK+DW}-^o^Y zLc~L&_{(cf>Ui!ZqYmgAZerk8A>|a49T(P2??un*k~B*rYz)ke@`y4?8MSahcb^l6 z2%srrq%A)G?a`6rS7;+guNPi)5agz#A4FWi&in0f8y}${5VB<@Uhv~5G?ZqXHSafj zKGu@wq#OmZz~G9RsIH!C`+mmSR~)PWLomK6=PG3XrlyRRd0If<$;^XYpddrj{>G9h z8ruXUyOLi(?*_BPR+$s;Q?eg&k-}n8;&U*l-RK**Xt-d{RABIDV8B@L9J|Y7eUGic zVCOwT630I9Cc#e#h{i~yd+e~}1UhKK11oAH*1LS^mgOgY0y*UX^R}|JJM_486q%b! z$mWXWl4y4Wy=u2EcG!)g!;jJs^djBhglVlz?pZe>+vNOZbp#1SCrA5-DYQHEfnuMC zV5hBgGLGVhj4fcxOr&#Uj_IOOwvf7t-KBka9-O+M*GNfqzV!dKpL9;3+gK_)L?UXe5xUOv=N5`Y|3&d$$}3@iKTV~X!3#s`D zC>p#`AA=c#{+<)Y#C~f_H~d%ZH|MFIdd{(rz+69(k(o_ZptGL8j2j(HPh(3fNj(nU zuh1+cSvavd-l(-DI@WrgMvyWxAK>r&N|7JNP{lY`k)0>ZgB6xF-k-s2HvP|h@C@ug zME&e0wUmp65M`}+QHpX%kGwbKSm=hSwg70y+OcD8_3GE$y7GhNFHMu4iu6umv&8V60)A-tBQ1q;Zf^;cjttpV2aKKo$S`ZFi`eJ} z@qx{VG4W!cFF4^oJsUs19o3Ded_Gh45w;=b$woW}7tynXD3XDvENqwQWFhdtyX^-d zH1I|jX@9$ky7KWmv5o7>Ddqf%F|1oO<6FCoF^riLVgA-=*i^uv97pD$AVL;qF2I!>+mD{yUgN_2 zn+^9H1_<%cK~qgFBZ}^wi=LAA-mlYjy#)Qw)~wc1u`~YYq=?L$otSGP` z0q>VmI)CyZlZ%LdHv}J;L9Gtx+yIXc7*U(vZ2AQqMXU2;Ep%(5mG13XXf+rI7JY=Q z1>eRs?Idl9EE-BzS^_e%YIKTX1`coX;}xQrA?N}#7A+eQ&}JSkvi1kkquqH zSmoL`TFtU5c5?*Zf_%UHHIZBt4B?f)0K)LZd@mgx53*q#E5bRX7J{3Z{!d-ytnx;% zIJjsYqElwHMlPcLPy$KCxF#Zh;ThH%uYE>sjB^8XBW|Evskls8K~1AoQcPaLB~;(Q zNn=|;f{0wbOBdR%%4Pxgz2f`9u1SBjEM~h!Fmr6=tDc?125Tc*Fd3_{3KKm2E0&6Q zwU{psOsO<9fy5w+aUV^|OFG%G*~WmkZy5HgCcBG_wx<1l!Xj1G5+`eI@}rwST%2;g z_MV)4zyq^v+|ebK9P&*#Mni*X&|2w|oq4Brswp}@8yAhbyZCAL3mfZdbkE5;SN|51 zS6$!l3m)p|K<(Cyi?&IDZinvqHfS|uL<|dr*@!QL#6!2OB9N!vYB`XmKiylg=25Wh zoJ1RRA0iZ};=(`wFKjlj>vL){bN{LDVl4_N{us@i$I7B??k}%%g(P2Y_$rwekUNE~ zx3Axoiie1TES{(6hf@=SPp_PF3FHGN5Vb}JWs+G8s^4E*1M!dQ(mrfdGu)sL^C23A z<`=e_il2S{(;vMPP!Y;65wKD~8r;4Cm%2-T>R#uz0}gDbRzmO?vHw3y6>Kd+#0Qi3 z`k&7(!8O(`hJAUG27n3F#{$YhdOMWQ&OO9dHvXTZWB;8b|7RWGT?@&En$go|zdat=O5}o;Jt-x63c`*F5jgFza#9WaD~(8Jz7FMD z<+9p1$_S-y*C)$j>bIFEFiWyyY_ZxiHZIhts3Bd|kL_q?67aWbGLZPbHs_3RQjS*U zFFJ(AK{=uK3mH^U4f=_=#yQ>Y*@Rszc%K>WoQ&$0MRVA0H^x+T8BPVafVesF&RJ0=HL z-;3O3vfw~N%mZ&%K1n2^zJG<#MDBG=eNbd-4dQqBu3P2 zQKZ)@H;l)4)xGz>@fHaXgGBVr(|ba#78za~9&MVKP%LD@B5;T_COZGQmY8?Bal{A~EL~4X=f=2sGUk(LJ#dkSaQv4X#hB%gSG#}}7eHJ|r-{Cq zuUj&=L-Dae3eKS=T8sEF_J)HQPR%7w`gR{$V9A0~)ze~2SALh<+e2q}QNW!K##v#q znr8qe=LE$)On@_6ONNpw9|zOwq7jI^Mkaa@A{B z)&nIBXRG(oW(4SG1~kX1bT}AN95~S1&vmr$8QvsegKlR6GU}S2@81IfcT4fZa0&b~HDgsb=+Q&o)SWuh@xZ!^sr?> znB5`YS*$$Dk+4PEw2wx4WA{RY&rGHqT#BF^C*)Ypn`|7u!g$V-YZTJ>d1S}iC?wUA zrGj-eA}QKQz4Z&a&k&C}=iuD1#5VUZU&(Xmhk({x`#cQf!V}HstV|w$@;12Hr8f!6 z6lajKAu94vn;q&-Tr2H`2@3Yq`;3lBp@q7N;)_~_^|}=&8^73&gjeq8*hbXa_L0N0 zb#|mCWP3gomOw1H&yci)3GSjefQY#qq{dqw!>7>VGW|hKI!{{3@0`H1?@WqWqS^*H zm%{MrzFCgi>-~~ujUao43o^^vV`j^HT+GDo;?lIDXed2R6WSnF+`d`c+Mg&J~1(FnYIi|c#t z=Gx&Ta}JK#)0rxp^q$XM`Lp=)JX}Bkpm*_>q@vyjKZ5(sU8<_qODpgrno;6!hh#h* zeuCwol4V#12ra7(N6{5Mse70uscp8-;_Uom5V1T2(%k!E^N6QI=jneN9+m|J6Umpe z_w*^GnZPis@Lpt?N>%0@hrE`_nV)iG_TV1d-foocm5`E`-~YES##?H}4=j+--@*Ga z(olI_OIkXrnMfpm$QB$GB@D3bRK#rFiP3Oa}hlM}Z zz**UT<3GD1+Bu+YgcCvqVSa^vi zcpkTX7_!)h$N^|63BMsC(kAOz5S!|zXghFC?Bde^?h)cZd*tBI-Nn*R>ZlUfnESKg zUDa%S547%lI#@``L$7&;My^P)r3FPUz4LCqK5iM_HMUO9D9y2O2O!UE4(Hw`d{nR7 zhd}687|m-gG>7$n{yx>k^gjb(C0|n1-+%ysE1liqpI6_al+ABk?IQd-W&X<{lVq74 zSl&Xn8_DEBe)4@31q8U68`(~aE(hud&paf3AoX2@qA4jhT>bNRqIBlnjv)gNa5-TI zluKJrzxR>6rxM%WkrOKYsif9tS7r45sf0dC^5`MD$GB}Sk8u+-bN#Pmf>W>WE#0vD zbJ5Hl4%6f!k-cpyX&}uiU1;L|FMF6~9uV>#6>S`OD_F6q;)E1Be8{~2*8FgA5h8iK z1_t^w`vo*VjQgv7@julG|C|FGl&GNVf~6a34FeO@o@1`tB*QBeR*3NUSrq7lBkAo_ zHBVGR#DbZMDOOhQlGV$9b6QrVr*#NcNa^K3p6hO-=6m9-K(CbnX+ZB8t(+j4ifuM) zz8EFAb$(K0mY38YvH+jB5&v~0QltT97pck|Q}H(+C+;#Gn~Nm**RR))gCv53&IDp- zpdu&zd|$L<^Dhj_;{Z!tF7RqaY81}`r3mu^_i19JiW{X<0Vplas?#0lcgSkrTX5P< z`C7O0&V$P8n6J6#J1f|93ns^j;Efc(?@`#l#vTerwI4Ag;AQgn(5`ldMNPq(xAj<$ zcgN2)*~k}Pu34Qj5JPdDlU^bssWH`l2=d~^k*Uwp_m*<~B`djwiD5sjL6302)POI z$SaqLKu7TbXScFR*tZkmGp^7)J5B)xB<6#*_BP;9pnhB3h3K zXZ42Cuf!L?OgaC|CfJ$|*to3Q4_=Vb@ep8D|C6uM86@=WL+((eRRGR%98XP*59p<= zRU-@d1A9*t9uTZ?YbkfHMPD{!c(l2A;Q93%@@G4E4Ec4+bNvIup0Esem|Kn8)Y@_T zig%;vbQQ-tZVOT@u0262l-Zu zidMWluXx6|aKrxn`>(zXwytg7zl+~XyJDekVBnm_o+A?L*AIhSVTFc-T)W(bQX zrC(8GA3W=w*Ulr()c4fZAGyrvI#Tsf1N-LAP|k1jLU6ps@e;*|x(MklZ+b>b-8)P= z>yig=I}Xo%(9Frf!Ljkv^TXdq(@IKAbgitcj>dR(|D7J!-fpH^96uDZ%!R2bKd;7i z8bGM-kM%xA^ikC3%H6|Go^m}ov`5q|JjYf!sTKN&lGgb@Eq(osY+K~hjxEmN#(R6e z%3L%HuN)`0Sx%(aJ4=55ocH(Np43Y}UgYNMLa#HGHY?KWqy<-n>2=(NM>FX4|Nr&> zhf61}IEYJB_U#+)iJFACI8#e2>RaqAnnjMVUy^dLdA5>Wkcp;v>@qTn ziib|uv;kPu%4F2?mhbNuF~LWsdmON`N{)|?7Gg$5My@$Kj>Gr#f8J!@eDVl)ymWN_9P+~uhHIGxd`q$m`*T$MQiMptT9^jQ&RrpH-1N;`kSVKcm7p|f}v*2=H zW`zcrMGD64<*;xr?t+<`xh@apmqE@G?lA*642KnuDYUpqmdjiCBp%jBGtax2tbPgb zhGB)`=kD&#qNEEZ1&k<_?Po$PiI)A!vPP$mAAqqgGiqxB=Fj7>$8)Y(zgPE+c@(V_ zqP>pU053$`vS|*6PMBmg)h}GR zub+v>RBg}za(h`9{3KS5DUsXmJ$vL)1$}8V9*#v_!0tkv(akXDDSMX0r=%=rVl;8B zd0V-s#4R)K;S^N7&l+)mU6HbCoLzCNzg$0d&B8hy(;_a8J$+?1q`=pu?!R)N#OY9H zs43LDp&-B>A6GC8=}$vr^-j!#ZxbqTa-X?J5ab|gM-PqTP@FkchPLQMZsyn~r}v|~ zU|Iv(>nt|eJ_?MUSu~7!iWM3aHTWUB&br694SI>?;!Bp?y&Ii-#I73)Y**--EI<&X zkYs^Dl~)y%SZaeIdABX= z-Z6q6+Ti;nU?f-LAt&7j2X_BChz<>MmOuU(2RM)%h+o&&N=z)?=Fxd46RUH31BBk9 zU10Y0o(z@c>vh579+ZQ0mz5QoCLR$H%TW61GtGV3X_6*;a6@gBXl+z{e85e+#?~dC z@IIJU9mc?(+9%+NlTuZGuFLoOG)0q%D;FD&waKiit<}$AJP%2@KI_)rI;~5eW7c}d z%>AfC#bTyXmJ5gXUk+U}?6+jWQD>~nWCplyje(alkWbvs^OX$&VoILCjRIYap`=jT zp^jTx!(B4I2XG9F@KGkbYggkbjvb3aZQX*T?C4UQKx@^$M=JL9#8LQi^ba(WQ7j}h zHa0dQT{daX2}bj=Lq;MW=bUjLXo1%;!FTxR2CYNA-ABhBXS=iT9&OY`?~A4^c^QA? zgO%P@`I60Fcg6LtAk(e5K+E2J%U4M>xVHc-yx-qB-x3M;x+c%Ye%;p=WKu$lIA z9fzWFTM~WG->c%aN_6!KF-I-6@Cq*^28*5(loh+}3Olk??#CBeZK>~OU%n|bm=up4 z&O}#TU43Y+)c8VxTW*&kUELLo3vh4<6TDo2mqF!M{==_KOt7zsqPE}a}Rd_=MkDIf-}HV<2# zlvOZptw~dVxJVcV!>7|f8KlL<#Q~(*Sig0xtLi0%BZ+bM4u4&*_1%5@JfRDEdLz<~ z4hIVE9Yiq+xtVL4uX40T5J#|<+{rfR*~ysx$)bOa>w>4ZM$FoNTJ<@Zet*a19W!qC z@Ad?*WpdxJ=V%}*Bq9F8hYvqoT-B;;YJ}wNOd60aSh+NiCBe_n-`Uy8lDhRpSCvUF zL1q!8*TavQtMX%ys4tb3-Pr&k&@v)X1Kbxpd*du!S5l(HgsdyQ@71ll63WV@IXQZ| z*4EY@;rXwB21H9~WKku8+A+wP_CQ#jncRVaf$cAQjz{UwIKNFoecCaH4uviw9i3P0 z2?4uou(|h3JNLCVX}z0w1^fjwJ27<@C0<BMa%2op zhql`HnP|}szn{t@z?R*bu69J1uam?c$6Af%z}3q|>n9vfmOaP5SXfxVo4hl}pH71h zY@Ap$G>m-W5_JdEr2f6`5vQUd(EKH}%g`~T4^Xv6RaMoEpCu5qF0w)z+lU41RsAoI zgUMheS5k2A;Dfs!hMW`TvNHf~wnfrn3({JXtR<}NpU^&#Q-wd1+wXd4(1TPTd&y2CoCrQ1;c z)UmO9nX>X;)<6)zm?COC(hDgwNEBe=u}ez493N&<{4b)7T)Y-ZmeaJ>LCSa)y0XQ!-aF~Z~b3i|Ial1C8L~Mj@ilQWHT?& PS7NGatE6q-ck+J$k>b@X literal 0 HcmV?d00001 diff --git a/src/machine-learning-and-data-mining/main.tex b/src/machine-learning-and-data-mining/main.tex index d9d0867..3593190 100644 --- a/src/machine-learning-and-data-mining/main.tex +++ b/src/machine-learning-and-data-mining/main.tex @@ -30,5 +30,6 @@ \input{sections/_crisp.tex} \input{sections/_machine_learning.tex} \input{sections/_classification.tex} + \input{sections/_regression.tex} \end{document} \ No newline at end of file diff --git a/src/machine-learning-and-data-mining/sections/_classification.tex b/src/machine-learning-and-data-mining/sections/_classification.tex index 36eb7ee..01c8909 100644 --- a/src/machine-learning-and-data-mining/sections/_classification.tex +++ b/src/machine-learning-and-data-mining/sections/_classification.tex @@ -794,3 +794,73 @@ Inputs are fed to the network and backpropagation is used to update the weights. to predict a new observation, the $k$ most similar entries in the training set are selected and the class of the new data is determined as the most frequent class among the $k$ entries. \end{description} + + + +\section{Binary to multi-class classification} + +\begin{description} + \item[One-vs-one strategy (OVO)] \marginnote{One-vs-one strategy (OVO)} + Train a classifier for all the possible pairs of classes (this will result in $\frac{C \cdot (C-1)}{2}$ pairs). + The class assigned to a new observation is determined through a majority vote. + + \item[One-vs-rest strategy (OVR)] \marginnote{One-vs-rest strategy (OVR)} + Train $C$ classifiers where each is specialized to classify a specific class as positive and the others as negative. + The class assigned to a new observation is determined by the confidence score of each classifier. +\end{description} + + + +\section{Ensemble methods} +\marginnote{Ensemble methods} +Train a set of base classifiers and make predictions by majority vote. +If all the classifiers have the same but independent error rate, +the overall error of the ensemble model is lower (derived from a binomial distribution). + +\begin{figure}[h] + \centering + \includegraphics[width=0.6\textwidth]{img/ensemble_error.png} + \caption{Relationship between the error of base classifiers and ensemble models} +\end{figure} + +Different strategies to train an ensemble classifier can be used: +\begin{descriptionlist} + \item[Dataset manipulation] Resampling the dataset for each base classifier: + \begin{description} + \item[Bagging] + Sample with replacement with a uniform distribution. + \item[Boosting] + Iteratively change the distribution of the training data + prioritizing examples difficult to classify. + \begin{description} + \item[Adaboost] \marginnote{Adaboost} + Iteratively train base classifiers on a dataset where samples + misclassified at the previous iteration have a higher weight. + \end{description} + \end{description} + + \item[Feature manipulation] + Train a base classifier using only a subset of the features. + + \item[Class labels manipulation] + Train a base classifier to classify a partition of the class labels. + For instance, class labels can be partitioned into two groups $A_1$ and $A_2$, and + the base classifier is trained to assign as label one of the two groups. + During inference, when a group is predicted, all labels within that group receive a vote. +\end{descriptionlist} + + +\subsection{Random forests} +\marginnote{Random forests} + +Different decision trees trained on a different random sampling of the training set and different subset of features. +A prediction is made by averaging the output of each tree. + +\begin{description} + \item[Bias] \marginnote{Bias} + Simplicity of the target function of a model. + \item[Variance] \marginnote{Variance} + Amount of change of the target function when using different training data (i.e. how much the model overfits). +\end{description} + +Random forests aim to reduce the high variance of decision trees. \ No newline at end of file diff --git a/src/machine-learning-and-data-mining/sections/_regression.tex b/src/machine-learning-and-data-mining/sections/_regression.tex new file mode 100644 index 0000000..1b65d57 --- /dev/null +++ b/src/machine-learning-and-data-mining/sections/_regression.tex @@ -0,0 +1,56 @@ +\chapter{Regression} + +\begin{description} + \item[Linear regression] \marginnote{Linear regression} + Given: + \begin{itemize} + \item A dataset $\matr{X}$ of $N$ rows and $D$ features. + \item A response vector $\vec{y}$ of $N$ continuous values. + \end{itemize} + We want to learn the parameters $\vec{w} \in \mathbb{R}^D$ such that: + \[ \vec{y} \approx \matr{X}\vec{w}^T \] + + \item[Mean squared error] \marginnote{Mean squared error} + To find the parameters for linear regression, + we minimize as loss function the mean squared error: + \[ + \mathcal{L}(\vec{w}) = \Vert \matr{X}\vec{w}^T - \vec{y} \Vert^2 + \] + Its gradient is: + \[ \nabla\mathcal{L}(\vec{w}) = 2\matr{X}^T(\matr{X}\vec{w}^T - \vec{y}) \] + Constraining it to 0, we obtain the problem: + \[ \matr{X}^T\matr{X}\vec{w}^T = \matr{X}^T\vec{y} \] + If $\matr{X}^T\matr{X}$ is invertible, this can be solved analytically but could lead to overfitting. + Numerical methods are therefore more suited. + + Note that: + \begin{itemize} + \item MSE is influenced by the magnitude of the data. + \item It measures the fitness of a model in absolute terms. + \item It is suited to compare different models. + \end{itemize} + + \item[Coefficient of determination] \marginnote{Coefficient of determination} + Given: + \begin{itemize} + \item The mean of the observed data: $y_\text{avg} = \frac{1}{N} \sum_i \vec{y}_i$. + \item The sum of the squared residuals: $SS_\text{res} = \sum_i (\vec{y}_i - \vec{w}^T\vec{x}_i)^2$. + \item The total sum of squares: $SS_\text{tot} = \sum_i (\vec{y}_i - y_\text{avg})^2$. + \end{itemize} + The coefficient of determination is given by: + \[ \text{R}^2 = 1 - \frac{SS_\text{res}}{SS_\text{tot}} \] + + Intuitively, $\text{R}^2$ compares the model with a horizontal straight line ($y_\text{avg}$). + When $\text{R}^2 = 1$, the model has a perfect fit. + When $\text{R}^2$ is outside the range $[0, 1]$, then the model is worse than a straight line. + + Note that: + \begin{itemize} + \item $\text{R}^2$ is a standardized index. + \item $\text{R}^2$ tells how well the variables of the predictor can explain the variation in the target. + \item $\text{R}^2$ is not suited for non-linear models. + \end{itemize} + + \item[Polynomial regression] \marginnote{Polynomial regression} + Find a polynomial instead of a hyperplane. +\end{description} \ No newline at end of file