Interactively shaping agents via human reinforcement: The TAMER framework

To appear in The Fifth International Conference on Knowledge Capture (K-CAP 09),
Redondo Beach, California, USA, September 2009.
Interactivel yShaping Agents via Human Reinf orcement
The TAMER Frame work
W.Bradle yKno x
Depar tment ofComputer Sciences
The Univ ersity ofTexas atAustin
bradkno x@cs .ute xas .eduPeter Stone
Depar tment ofComputer Sciences
The Univ ersity ofTexas atAustin
pstone@cs .ute xas .edu
ABSTRA CT
As computational learning agen tsmoveintodomains
that incur real costs (e.g., autonomous driving orﬁnan-
cial investmen t),itwill benecessary tolearn goodpoli-
cies without numerous high-cost learning trials. One
promising approac htoreducing sample complexit yof
learning atask iskno wledge transfer from humans to
agen ts.Ideally ,metho dsoftransfer should beaccessible
toanyone with task kno wledge, regardless ofthat per-
son’s exp ertise inprogramming and AI. This pap erfo-
cuses onallowing ahuman trainer tointeractiv elyshap e
an agen t’s policy via reinforcemen tsignals. Speciﬁ-
cally ,the pap erintroduces “Training anAgen tMan u-
ally via Evaluativ eReinforcemen t,”ortamer ,aframe-
work that enables suchshaping. Diﬀering from previous
approac hes tointeractiv eshaping, atamer agen tmod-
elsthe human’s reinforcemen tand exploits itsmodel by
choosing actions exp ected tobemost highly reinforced.
Results from twodomains demonstrate that layusers
can train tamer agen tswithout deﬁning anenviron-
men tal reward function (as inanMDP) and indicate
that human training within the tamer framew ork can
reduce sample complexit yoverautonomous learning al-
gorithms.
Keywords
human-agen tinteraction, shaping, sequen tial decision-
making, human teac hers, learning agen ts
1. INTR ODUCTION
Ascomputational learning agen tscontinuetoimpro ve
their abilit ytolearn sequen tial decision-making tasks,
acentral but largely unfulﬁlled goal ofthe ﬁeld isto
deplo ythese agen tsinreal-w orld domains, making de-
cisions that aﬀect our lives. However,with real-w orld
deplo ymen tcomes real-w orld costs. Forsuchadeplo y-
Cop yright ACM ...$5.00men ttobeviable, agen tswill not beable touse hun-
dreds orthousands oflearning trials toreac hagood
policy when eachsub optimal trial iscostly .Forexam-
ple, anautonomous driving agen tshould not learn to
driv ebycrashing intoroad barriers and endangering
the livesofpedestrians.
Fortunately ,for man yofthese tasks, humans havedo-
main kno wledge that could speed the learning process,
reducing costly sample complexit y.Curren tly,most
kno wledge transfer from humans toagen tsoccurs via
programming, whic histime-consuming and inaccessi-
ble tothe general public. Itisimp ortan ttodevelop
agen tsthat can learn from natural metho dsofcomm u-
nication. The teac hing technique ofshaping isone such
metho d.Inthis context, wedeﬁne shaping asinterac-
tively training anagen tthrough signals ofpositiv eand
negativ ereinforcemen t.1Inashaping scenario, ahu-
man trainer observ esanagen tand reinforces itsbehav-
iorthrough push-buttons, spokenword (“yes” or“no”),
oranyother signal that can beconverted toascalar sig-
nal ofappro valordisappro val.The keychallenge, then,
istocreate agen tsthat can beshap edeﬀectiv ely.Our
problem deﬁnition isasfollo ws:
The Shaping problem Within asequen tial decision-
making task, anagen treceiv esasequence ofstate
descriptions (s1,s2,...where si/epsilon1S)and action op-
portunities (choosing ai/epsilon1A ateachsi). From a
human trainer who observ esthe agen tand un-
derstands apredeﬁned performance metric, the
agen talso receiv esoccasional positiv eand nega-
tivescalar reinforcemen tsignals (h1,h2,...)that
are correlated with the trainer’s assessmen tofre-
centstate-action pairs. Howcananagen tlearn the
bestpossible task policy (π:S→A),asmeasured
bythe performance metric, giventhe information
1Weuse the term “shaping” asitisused inanimal learning
literature (in whic hitwasinitially develop edbyB.F. Skin-
ner). There, shaping isdeﬁned astraining byreinforcing
successiv ely impro ving appro ximations ofthe target behav-
ior [6]. Inreinforcemen tlearning literature, itissometimes
used asinanimal learning, but more often “shaping” isre-
stricted tometho dsthat com bine the shaping reinforcemen t
signal and the rew ard signal ofthe environmen tintoasingle
signal [14]. An imp ortan tfeature ofour tamer framew ork
isthat the twosignals are not com bined.
contained inthe input?
InaMark ovDecision Pro cess (MDP), the environmen-
talreward signal, along with the rest ofthe MDP speci-
ﬁcation (see [16] fordetails), unam biguously deﬁne aset
ofoptimal policies. Since our goal istoallowthe human
trainer tofully control the agen t’sbehavior, the envi-
ronmen talreward signal isnot used. Following Abb eel
and Ng’s terminology [1], wecall this anMDP \R.
Exp ected beneﬁts oflearning from human reinforce-
men t2include the follo wing.
1.Shaping decreases sample complexit yfor learning
a“good”policy .
2.An agen tcan learn inthe absence ofacoded evalu-
ation function (e.g., anenvironmen talrew ard func-
tion).
3.The simple modeofcomm unication allowslayusers
toteac hagen tsthe policies whic hthey prefer, even
changing the desired policy ifthey choose.
4.Shap edagen tscan learn inmore complex domains
than autonomous learning allows.
This pap eraddresses the ﬁrst three ofthese beneﬁts.
Init,wereview previous work onagen tsthat can learn
from ahuman teac her through natural metho dsofcom-
munication (Section 2). Wetreat shaping asaspeciﬁc
modeofkno wledge transfer, distinct from (and proba-
bly complemen tary to) other natural metho dsofcom-
munication, including programming bydemonstration
and giving advice. Shaping only requires that aper-
son can observ ethe agen t’sbehavior, judge itsqual-
ity,and send afeedbac ksignal that can bemapp ed
toascalar value (e.g. bybutton press orverbal feed-
backof“good”and “bad”). Wethen presen tanovel
metho dbywhic hhuman trainers can shap eagen ts(Sec-
tion 3). This agen t-trainer framew ork, called Train-
ing anAgen tMan ually via Evaluativ eReinforcemen t
(tamer ),mak esuse ofestablished sup ervised learn-
ing techniques tomodel ahuman’s reinforcemen tfunc-
tion and then uses the learned model tochoose actions
that are projected toreceiv ethe most reinforcemen t.
The tamer framew ork, the insigh tsthat motiv ate it,
and anextension todela yedreinforcemen tare the key
contributions ofthis pap er. Furthermore, wedescrib e
twospeciﬁc, fully implemen ted tamer algorithms that
provide proofs-of-concept for twocontrasting task do-
mains (Section 4). Exp erimen tal results inthese do-
mains showthat tamer agen ts,under the tutelage of
human trainers, learn a“good”policy faster than eﬀec-
tiveautonomous learning agen ts(Section 5).
2Inthis pap er,wedistinguish between human reinforcemen t
and environmen tal rew ard within anMark ovDecision Pro-
cess. Toavoid confusion, human feedbac kisalwayscalled
“reinforcemen t.”2. RELA TED WORK: LEARNING FROM A
HUMAN
Work onhuman-teac hable agen tshas takenman yforms,
allwith the aim ofreducing the amoun tofprogram-
ming required byanexp ert, using more natural modes
ofcomm unication. Inthis section, wedescrib epast
work onagen tsthat learn from humans. Weargue that
the relativ estrengths and weaknesses ofour training
system put itinaunique space that isnot curren tlyoc-
cupied byanyother approac h.Furthermore, webeliev e
that man yofthe approac hes wereview are complemen-
tary toours: an ideal learning agen tmigh tcom bine
elemen tsfrom several ofthem.
2.1 Lear ning fromAdvice
Advice, inthe context ofMark ovDecision Pro cesses
(MDPs), isdeﬁned assuggesting anaction when acer-
tain condition istrue. Maclin and Sha vlik [12] pioneered
the approac hofgiving advice toreinforcemen tlearn-
ers. Giving advice via natural language could bean
eﬀectiv ewayfornon-tec hnical humans toteac hagen ts.
Kuhlmann etal.[11] created adomain-sp eciﬁc natural
language interface for giving advice toareinforcemen t
learner.
The informational richness ofadvice isclearly powerful,
howeverthere still remain anumberoftechnical chal-
lenges. The ﬁrst ofthese isthat general natural lan-
guage recognition isunsolv ed, soman ycurren tadvice-
taking systems [12, 13] require that the human enco de
her advice intoascripting orprogramming language,
making itinaccessible tonon-tec hnical users. The natu-
rallanguage unit ofKuhlmann etal.[11] required man-
ually labeled training samples. Moreo ver,work still re-
mains onhowtoembedadvice intoagen tsthat learn
from exp erience.
Additionally ,there will likely betimes when the trainer
kno wsthat the agen thas performed wellorpoorly ,but
cannot determine exactly why.Inthese cases, advice
will bemuchmore diﬃcult togivethan positiv eorneg-
ativ efeedbac k.
2.2 Lear ning fromDemonstration
Another wayforahuman toteac hanagen tistodemon-
strate atask via remote-con trol orwith his ownbody,
while the agen trecords state-action pairs, from whic h
itlearns ageneral policy for the task [3].
InAppren ticeship Learning [1], atypeoflearning from
demonstration, the algorithm begins with anMDP \R
(as doesthe tamer framew ork; see Section 1). The
algorithm learns areward function Rfrom ahuman’s
perio dofcontrol, and then the agen ttrains onthe MDP .
Considering the demonstration typeinwhic hahuman
controls the agen t,there are some tasks that are too
diﬃcult for ahuman trainer. This migh tbebecause
the agen thas more actuators than can beput inasim-
ple interface (e.g., man yrobots) orbecause the task
requires that the human beanexp ertbefore being able
tocontrol the agen t(e.g., helicopter piloting insimu-
lation). Inthese cases, ademonstration isinfeasible.
But aslong asthe human can judge the overall qualit y
ofthe agen t’sbehavior, then heorshe should beable
toprovide feedbac kvia tamer regardless ofthe task’s
diﬃcult y.
2.3 Lear ning fromReinf orcement (Shaping)
Within the context ofhuman-teac hable agen ts,ahuman
trainer shap esanagen tbyreinforcing successiv ely im-
proving appro ximations ofthe target behavior. When
the trainer can only givepositiv ereinforcemen ts,this
metho dissometimes called clicker training ,whic hcomes
from aform ofanimal training inwhic hanaudible click-
ing device ispreviously asso ciated with reinforcemen t
and then used asareinforcemen tsignal itself totrain
the animal.
Previous work onclickertraining has involvedteac h-
ing trickstoentertainmen tagen ts. Kaplan etal. [9]
and Blum berg etal. [4]implemen tclickertraining on
robotic and simulated dogs, resp ectiv ely.Blum berg et
al.’s system isespecially interesting, allowing the dog to
learn multi-action sequences and asso ciate them with
verbal cues. Though signiﬁcan tinthat they are novel
techniques ofteac hing pose sequences totheir resp ectiv e
platforms, neither isevaluated using anexplicit perfor-
mance metric, and itremains unclear ifand howthese
metho dscan begeneralized toother, possibly more
complex MDP settings.
Thomaz &Breazeal [18] interfaced ahuman trainer
with atable-based Q-learning agen tinavirtual kitc hen
environmen t.Their agen tseeks tomaximize its dis-
coun ted total rew ard, whic hforanytime step isthe sum
ofhuman reinforcemen tand environmen talrew ard.3
Inanother example ofmixing human reinforcemen twith
on-going reinforcemen tlearning, Isbelletal.[8]enable
asocial soft ware agen t,Cob ot, tolearn tomodel hu-
man preferences inLam bdaMOO. Cob ot“uses rein-
forcemen tlearning toproactiv ely takeaction inthis
complex social environmen t,and adapts his behavior
based onmultiple sources ofhuman [reinforcemen t].”
LikeThomaz and Breazeal, the agen tdoesn’t explicitly
learn tomodel the human reinforcemen tfunction, but
rather uses the human reinforcemen tasareward signal
inastandard RL framew ork.
The tamer system isdistinct from previous work on
human-deliv ered reinforcemen tinthat itisdesigned
both for ahuman-agen tteam and towork incomplex
domains through function appro ximation, generalizing
3As indicated inFootnote 1,com bining human reinforce-
men tand environmen tal rew ard intoone rew ard signal is
the narro werdeﬁnition ofshaping inreinforcemen tlearn-
ing. Asargued in[10], hand rare fundamen tally diﬀeren t
signals that contain diﬀeren tinformation and thusshould
betreated diﬀeren tly.Human Environment
AgentReinforcementActionStateSensory
Display
Supervised
LearnerReinforcement
ModelModel
Parameters
Reinforcement
PredictionReinforcement
PredictionsAction
SelectorAction
Figure 1:Framew ork forTraining anAgen tManually via Evaluativ e
Reinforcemen t(tamer ).
tounseen states. Italso uniquely forms amodel ofthe
human trainer’s intentional reinforcemen t,whic hitex-
ploits for action selection inthe presence orabsence of
the human trainer.
3. THE TAMER FRAMEW ORK
Consider the Shaping problem description giveninSec-
tion 1. The agen t’s role isbynature somewhat un-
derconstrained: itmust learn from reinforcemen tsuch
that itmaximizes performance (whic hitcannot mea-
sure). Wehypothesize that the agen tbest ﬁlls itsrole
bylearning amodel ofthe trainer’s reinforcemen tand
exploiting the model inaction selection.
Atamer agen tseeks tolearn the human trainer’s re-
inforcemen tfunction H:S×A→R.Presen ted with a
state s,the agen tconsults itslearned model ˆHand, if
choosing greedily ,takesthe action athat maximizes
ˆH(s, a).Figure 1shows interaction between ahu-
man, the environmen t,and atamer agen twithin an
MDP \R.Since the agen tseeks only tomaximize human
reinforcemen t,the optimal policy isdeﬁned solely by
the trainer, who could choose totrain the agen ttoper-
form anybehavior that itsmodel can represen t.When
the agen t’sperformance isevaluated using anobjectiv e
metric, weexp ect itsperformance tobelimited bythe
information provided bythe teac her.
3.1 Moti vating Insights
When wespeak ofautonomous learning agen ts,wemean
those that receiv efeedbac kinthe form ofenvironmen tal
reward (i.e., reinforcemen tlearning agen ts[16]). The
principal challenge for suchagen tsis,uponreceiving
environmen tal reward, toassign credit from that re-
ward tothe entire history ofpast state-action pairs. In
domains where the only discriminating reward isatthe
end ofthe task, assigning this credit can beparticularly
diﬃcult. Akeyinsigh tofthe tamer framew ork isthat
the problem ofcredit assignmen tinheren tinreinforce-
men tlearning isno longer presen twith an atten tive
human trainer. The trainer can evaluate anaction or
short sequence ofactions, considering the long-term ef-
fects ofeach,and deliv erpositiv eornegativ efeedbac k
within asmall temp oral windo wafter the behavior. As-
signing credit within that windo wpresen tsachallenge
initself, whic hwediscuss inSection 3.3, but itismuch
easier than assigning credit foranarbitrarily dela yedre-
ward signal asinreinforcemen tlearning. Assuming, for
now,that credit isprop erly assigned within the temp o-
ral windo w,weassert that atrainer can directly label
behavior. Therefore, modeling the trainer’s reinforce-
men tfunction Hisasup ervised learning problem.
Intuition and some evidence [8]suggest that ahuman
trainer’s reinforcemen tfunction, H,isamoving target.
Intuitiv ely,itseems likely that ahuman trainer will
raise hisorher standards asthe agen t’spolicy impro ves,
exp ecting more toget the same reinforcemen t.Though
wehavenot exp erimen tally studied this claim that a
human’s reinforcemen tfunction isamoving target, it
isdiscussed infurther detail in[10]. Considering our
conjecture that Hisamoving target function and that
the tamer agen tshould beable toexploit new data as
itbecomes available, the sup ervised learning algorithm
should both beable tohandle changes inthe target
function and allowincremen talupdates.4Additionally ,
for best results, the algorithm should generalize toall
unseen samples (i.e., tostates and possibly actions).
Examples ofsuchalgorithms include gradien tdescen t
overthe weigh tsofalinear model (whic hweuse in
our exp erimen ts)and backpropagation inamulti-la yer
neural network.
Aswewill report inSection 5,compared toautonomous
learning algorithms, tamer dramatically reduces the
numberofepiso des required tolearn agoodpolicy .One
result oflowering the sample complexit yisthat ifthe
incremen talupdates tothe model use asmall step size,
the model’s output will not movefar enough toever
bethe “true” human reinforcemen tfunction. But ide-
ally the learned function ˆHshould map tothe human
reinforcemen tfunction Hsuchthat
ˆH(s,a1)>ˆH(s,a2)⇐⇒H(s,a1)>H(s,a2).
Inthis case, the policies created bygreedily exploiting
ˆHand Hare equiv alen t.
3.1.1 Explor ation
The tamer framew ork isagnostic toexploration, leav-
ing action selection asablac kboxtobeﬁlled bythe
agen tdesigner. However, wehavefound that greed-
ilychoosing the action that isexp ected toreceiv ethe
highest reinforcemen tprovides suﬃcien texploration for
man ytasks, including those describ edinSection 5.This
coun terin tuitiv eﬁnding iseasily justiﬁed. Inagiven
state, agoodtrainer negativ elyreinforces anyundesired
action, eventually dropping itsexp ected reinforcemen t
belowanother action, resulting inanew (exploratory)
action choice. Anon-greedy action selection metho d
isneeded ifthere are actions ofwhic hthe trainer is
not aware ordoesnot kno wthe value. Otherwise, the
4Sup ervised learning algorithms that havebeenused suc-
cessfully asfunction appro ximators for reinforcemen tlearn-
ing ﬁtthese tworequiremen tsand thusmak egoodcandi-
dates for modeling the human.Algorithm 1Ageneral greedy tamer algorithm
Require: Input: stepSize
1:ReinfMo del.init(stepSize)
2:−→s←−→0
3:−→f←−→0
4:while true do5: h←getHumanR einfSinc ePreviousTimeStep()
6: ifh/negationslash=0then
7: error←h-ReinfMo del.pr edictR einf(−→f)
8: ReinfMo del.up date(−→f,error)
9: end if10:−→s←getStateV ec()
11: a←argmax a(ReinfMo del.pr edict(getF eatur es(−→s,a)))
12:−→f←getF eatur es(−→s,a)
13: takeA ction(a)
14: wait for next time step
15: end while
trainer should beable toguide the tamer agen t’sex-
ploration suﬃcien tlyunder greedy action selection.
3.1.2 Comparison with Reinfor cement Learning
The tamer framew ork for shaping agen tsshares much
common ground with reinforcemen tlearning, but there
are some keydiﬀerences that are imp ortan ttounder-
stand inorder tofully appreciate tamer .
Inreinforcemen tlearning, agen tsseek tomaximize re-
turn, whic hisadiscoun ted sum ofallfuture reward. In
contrast, atamer agen tdoesnot seek tomaximize a
discoun ted sum ofallfuture human reinforcemen t.In-
stead, itattempts todirectly maximize the short-term
reinforcemen tgivenbythe human trainer. Itdoesthis
because the trainer’s reinforcemen tsignal isadirect la-
belonrecen tstate-action pairs.
Corresp ondingly ,the human’s reinforcemen tfunction H
isnot an exact replacemen tfor arew ard function R
within anMDP .Although itmaybepossible for are-
inforcemen tlearning algorithm touse Hinlieu ofare-
ward function, itwould beunnecessary extra computa-
tion, since Halready deﬁnes apolicy .Weuse MDP \Rs
because the environmen tal reward within MDPs func-
tion dictates the optimal policy,soRisremo vedto
mak ethe human’s reinforcemen tfunction the sole de-
terminan tofgoodand bad behavior.
Man yreinforcemen tlearning algorithms can beeasily
converted toTAMER algorithms. Weencourage inter-
ested readers tofollo wour step-b y-step instructions at
http://www.cs.utexas.edu/users/bradkno x/k cap09 .
3.2 High-Le velAlgorithm
Ahigh-lev elalgorithm for implemen ting tamer isde-
scrib edinAlgorithm 1.After initialization ofthe rein-
forcemen tmodel ReinfMo del,the state vector−→s,and
the feature vector−→f(lines 1-3), the algorithm begins a
loopthat occurs once pertime step (line 4).
Inthe loop, the agen tﬁrst obtains ascalar measuremen t
ofthe human trainer’s reinforcemen tsince the previous
time step (line 5). Ifthe reinforcemen tvalue isnonzero,
then the error iscalculated asthe diﬀerence between
the actual reinforcemen tand the amoun tpredicted by
the agen t’sreinforcemen tmodel (line 7). The model,
left undeﬁned forthe general tamer algorithm, can be
instan tiated inmultiple waysasdiscussed below. The
calculated error isthen used, along with the previous
feature vector, toupdate the reinforcemen tmodel (line
8). The update isnot performed when h=0because
the trainer maynot bepaying atten tion ormigh teven
havequit training altogether, satisﬁed with the trained
policy .
After the agen tobtains the new state description, it
then greedily chooses the action athat, according to
the human reinforcemen tmodel, yields thelargest pre-
dicted reinforcemen t(lines 10-11). Although wehave
found that greedy action selection issuﬃcien tinour
work thusfar, more sophisticated action selection meth-
odsmaybeneeded insome situations, and would ﬁtjust
aswellwithin the tamer framew ork. The agen tcalcu-
lates features−→fofthe state and action ofthe curren t
time step (line 12) and takesthe chosen action (line 13)
before restarting the loop.
3.3 Credit Assignment toaDense State-Action
History
Insome task domains, the frequency oftime steps is
toohigh forhuman trainers toresp ond tospeciﬁc state-
action pairs before the next one occurs. Insimulation,
itistechnically possible tolowerthe frequency oftime
steps. But doing somaychange the character ofthe
task, and inphysical domains (e.g. helicopter ﬂigh t)
itmaynot bepossible. Although wehavenot studied
exactly where the frequency threshold is,our exp eri-
ence and studies ofhuman resp onse times (Figure 2)
[7]havemade itclear that having several seconds be-
tween time steps isenough for speciﬁc labeling and,
conversely ,having several time steps persecond istoo
frequen tfor speciﬁc labeling.
Figure 2:Hockley’s
study of the distri-
bution of human re-
sponse times for vi-
sual searc hes ofdiﬀer-
entlevels ofcomplex-
ity.Forthese faster domains, credit
from the human’s reinforce-
men tmust beappropriately
distributed across some subset
ofthe previous time steps. Our
approac hforcredit assignmen t
with alinear model and gradi-
entdescen tupdates isshown
inAlgorithm 2,whic hisan
extension ofAlgorithm 1with
the model inline. The error
for an update isthe diﬀer-
ence between the projected re-
inforcemen tpredicted bythe
linear model, where the input
isaweigh ted sum ofthe state-
action features foreachpreced-
ing time step (lines 9-13), andthe actual reinforcemen treceiv ed(line 14). The weigh t
for anytime step isits“credit”. The model isthen
updated with that error and the weigh ted sum offea-
tures (line 15).
3.3.1 Credit Calculation
Toassign credit, weassume that the reinforcemen tsig-
nal (receiv edattime 0)istargeting asingle state-action
pair. This assumption will undoubtedly bewrong at
times, but the eﬀects ofthe assumption will probably
besmall since the credit isstill spread out amongst re-
centevents,asdescrib edbelow.There are ntime steps
that migh tbethe target, beginning attimes t1,t2,...,tn,
where ifi<j,then tioccurred more recen tly than tj.
(Therefore the time step beginning attiends attime
ti−1ifi>1.) Wedeﬁne credit ctfor atime step start-
ing attime ti5and ending atti−1(or 0ifi=1)tobe
the probabilit ythat the reinforcemen tsignal wasgiven
forthe event(s,a)that occurred during that time step.
Wecreate aprobabilit ydensit yfunction foverthe de-
layofthe human’s reinforcemen tsignal. Then for any
state-action pair, credit iscalculated asthe integral of
ffrom ti−1toti.
ct=P(event star ting attiwastargeted )
=P(targetevent betw een ti−1and tisec. ago)
=/integraldisplayti
ti−1f(x)dx
(Ifi=1,then ti−1=0inthe aboveequation.) We
observ ethe constrain t
n/summationdisplay
i=1P(event star ting attiwastargeted )=1
An example probabilit ydensit yfunction isshown in
Figure 3. Inpractice, wemain tain awindo wofre-
centtime steps, pruning out those time steps that are
older and havenear-zero probabilit y(lines 10and 21).
Wealso alwaysuse functions that havezero probabil-
itybefore 0.2 seconds, assuming that human trainers
cannot resp ond inless than 0.2 seconds (supp orted by
Figure 2).
4. IMPLEMENTED ALGORITHMS
tamer isfully implemen ted and tested intwocontrast-
ing domains. Though the tamer agen tactiv elyseeks to
maximize predicted human reinforcemen tH,thetrue
objectiv eofthe human-agen tsystem istomaximize
some performance criteria, whether itbethe trainer’s
sub jectiv eevaluation oran explicitly deﬁned metric.
Forthis reason, and because measuring directly against
Hisimpractical toimp ossible, weevaluate tamer us-
ing Rtoprovide the performance metric, whic histhe
cum ulativ eMDP reward receiv edthroughout allepiso des
5Atime step can bethough tofasaslice intime oran
interv alintime. Considering that achoice totakeaction
ainstate schanges the displa yseen bythe human trainer
untilthe next time step, weconsider atime step tobean
interv al.
Figure 3:Probabilit ydensit yfunction f(x)foragamma(2.0, 0.28)
distribution. Reinforcemen tsignal hisreceiv edattime 0.Iftand
t/primeare times ofconsecutiv etime steps, credit for the time step att
isRt
t/primef(x)dx.Note that time movesbackwards asone movesrigh t
along the x-axis.
Algorithm 2Agreedy tamer algorithm with credit assign-
men t,using alinear model and gradien tdescen tupdates
Require: Input: stepSize, windowSize,
1:Crediter.init(windowSize)
2:−→s←−→0
3:−→f←−→0
4:−→w←−→05:while true do6: Crediter.up dateTime(clo ckTime())
7: h←getHumanR einfSinc ePreviousTimeStep()
8: ifh/negationslash=0then
9:−−−−−−−→credF eats ←0
10: for all(−→ft,t)∈Crediter.historyWindow do
11: ct←Crediter.assignCr edit(t)
12:−−−−−−−→credF eats ←−−−−−−−→credF eats +(ct×−→ft)
13: end for
14: error←h−(−→w·−−−−−−−→credF eats )
15:−→w←−→w+(stepSize ×error×−−−−−−−→credF eats )
16: end if17:−→s←getStateV ec()
18: a←argmax a(−→w·(getF eatur es(−→s,a)))
19:−→f←getF eatur es(−→s,a)
20: takeA ction(a)
21: Crediter.up dateWindow(−→f)
22: wait for next time step
23: end while
inarun. Ineachdomain, R(ignored bythe agen t)was
easily comm unicated tothe trainer, who wasinstructed
totrain his orher agen ttoearn the highest sum of
environmen talreward pertrial episo de.6Also, ineach
domain the state and action are displa yedgraphically
tothe human trainer. Both task domains were imple-
men ted within the RL-Library .7One domain isTetris,
apuzzle computer game with alarge state space and a
variable numberofactions pertime step (as isstandard
inthe literature, aﬁnal placemen tofafalling Tetris
piece isconsidered anaction). The time step frequency
inTetris, atless than one action persecond, allowed
human trainers toeasily reinforce individual actions.
The other domain isMoun tain Car, inwhic hasimu-
lated car must get tothe top ofahill. The car begins
between twosteep hills and must gobackand forth
togain enough momen tum toreac hthe goal. Moun-
6Our Tetris speciﬁcation follo ws that describ ed byBert-
sekasand Tsitsiklis [2]. Exp erimen tal details and videos
ofagen tsbefore, during, and after training can befound at
http://www.cs.utexas.edu/users/bradkno x/k cap09 .
7http://co de.go ogle.com/p/rl-library/tain Car has acontinuous, two-dimensional state space.
Inour exp erimen ts,actions occurred appro ximately ev-
ery 150 milliseconds, prev enting trainers from labeling
speciﬁc actions. Also, agen tsstarted eachepiso deina
random location within bounds around the bottom of
the hill.
These twodomains complemen teachother. Tetris has a
complex state-action space and lowtime step frequency ,
and Moun tain Car issimpler but occurs atahigh fre-
quency .
4.1 Tetris
The algorithm used toimplemen ttamer onTetris isa
speciﬁc implemen tation ofAlgorithm 1,using alinear
model and gradien tdescen toverthe weigh tsofeach
feature. The function getF eatur es(−→s,a)on line 13
ofthe algorithm considers feature vectors deriv edfrom
both the curren tstate and the next state (before the
new Tetris piece app ears). The next state ismerely
the previous state with the new blocksinplace (whic h
the action dictates) with anyfull horizon tal lines re-
moved.The diﬀerence between the twofeature vectors
isreturned and actually used bytamer asthe features
overwhic htolearn.
4.2 Mountain Car
The Moun tain Car tamer algorithm follo wsthe algo-
rithm givenfortamer with credit assignmen tdescrib ed
inAlgorithm 2,also using alinear model with gradien t
descen tupdates. The function getF eatur eVec(−→s,a)
onlines 19and 20isimplemen ted using 2-dimensional
Gaussian radial basis functions asdescrib ed in[16].
Credit assignmen tfollo wedaUniform(0.2, 0.6) distri-
bution (describ edinmore detail inSection 3.3).
5. RESUL TSAND DISCUSSION
Our exp erimen tstest the ﬁrst three ofthe the exp ected
beneﬁts ofshaping that are describ edinthe Introduc-
tion: shaping decreases sample complexit yfor learning
a”good”policy ,anagen tcan learn inthe absence of
acoded evaluation function, and layusers can teac h
agen tsthe policies they prefer. Forthe exp erimen ts,
human trainers observ edthe agen tsinsimulation ona
computer screen. Positiv eand negativ ereinforcemen t
wasgivenvia twokeys onthe keyb oard. The train-
ers were read instructions and were not told anything
about the agen t’sfeatures orlearning algorithm. Nine
humans trained Tetris agen ts.Nineteen trained Moun-
tain Car agen ts.Foreachtask, atleast afourth ofthe
trainers did not kno whowtoprogram acomputer.
5.1 Tetris
Tetris isnotoriously diﬃcult fortemp oral diﬀerence learn-
ing metho dsthat model afunction suchasavalue or
action-v alue function. Inour ownwork, wewere only
able toget Sarsa( λ)[16] toclear appro ximately 30lines
pergame with avery small step size αand after hun-
dreds ofgames. Bertsek asand Tsitiklis report that
they were unable toget optimistic TD(lam bda) tomak e
”substan tial progress”. RRL-KBR [15] doessomewhat
better, getting to50lines pergame after 120 games or
so.8The only successful approac hthat learns avalue
function (of those found bythe authors) isλ-policy it-
eration [2], whic hreac hes several thousand lines after
appro ximately 50games.
However,λ-policy iteration diﬀers intwoimp ortan tways
from the rest ofthese value-based approac hes. First, it
begins with hand-co ded weigh tsthat already achiev e
about 30lines pergame. Getting tothat levelofplay
isnon trivial, and some learning algorithms, suchas
Sarsa( λ),fail toevenreac hitwhen starting with all
weigh tsinitialized tozero. Second, λ-policy iteration
gathers man ystate transitions from 5games and per-
forms aleast-squares batc hupdate. All other successful
learning metho dslikewise perform batc hupdates after
observing apolicy for man ygames (as man yas500
inthe best-p erforming algorithms), likely because of
the high stochasticit yofTetris. Additionally ,ofthose
that havethe necessary data available, allprevious algo-
rithms that modelafunction are unstable after reac hing
peak performance, unlearning substan tially untilthey
clear less than half asman ylines astheir peak.
Inour exp erimen ts,human trainers practiced for two
runs. Data from the third run isreported. Ofthe al-
gorithms that model anactual function, our gradien t
descen t,linear tamer isthe only one that clearly does
not unlearn inTetris.9Ofthose that also perform in-
cremen talupdates, tamer learns the fastest and tothe
highest ﬁnal performance, reac hing 65.89 lines pergame
bythe third game (Table 1and Figure 4).10
Policy searc halgorithms, whic hdonot model avalue
orreinforcemen tfunction, attain the best ﬁnal per-
formance, reac hing hundreds ofthousands oflines per
game [5,17], though they require man ygames toget
there and donot learn within the ﬁrst 500 games.
Within our analysis, tamer agen tslearn muchmore
quic kly than allpreviously reported agen tsand reac h
aﬁnal performance that ishigher than allother incre-
8Weshould note that Ramon et. al.rejected aform oftheir
algorithm that reac hed about 42lines cleared onthe third
game. They deemed itunsatisfactory because itunlearned
bythe ﬁfth game and neverimpro vedagain, eventually per-
forming worse than randomly .Ramon etal.’s agen tisthe
only one wefound that approac hes the performance ofour
system after 3games.
9This statemen tissupp orted bytraining byone ofthe au-
thors but not necessarily bythe sub jects, since the trainer
sub jects usually stopp ed training after the tamer agen t
reac hed satisfactory performance.
10Most trainers stopp edgiving feedbac kbythe end ofthe
ﬁfth game, stating that they did not think they could train
the agen ttoplayanybetter. Therefore most agen tsare op-
erating with astatic policy bythe sixth game. Score varia-
tions come from the stochasticit yinherit inTetris, including
the highest scoring game ofalltrainers (809 lines cleared),
whic hnoticeably brings the average score ofgame 9above
that ofthe other games.Table 1:Results ofvarious Tetris agen ts.
Metho d Mean Lines Cleared Games
atGame 3 atPeak forPeak
tamer 65.89 65.89 3
RRL-KBR [15] 5 50 120
Policy Iteration [2]∼0(no learning 3183 1500
untilgame 100)
Genetic Algorithm [5]∼0(no learning 586,103 3000
untilgame 500)
CE+RL [17]∼0(no learning 348,895 5000
untilgame 100)
 0 50 100 150 200 250 300
 1  2  3  4  5  6  7  8  9  10Environmental Reward (Lines Cleared)
Game NumberMean Reward By Group in Tetris
All trainers
AI background
No technical background
Random (uses initial weights)
Figure 4:Themean numberoflines cleared pergame byexperimen-
talgroup.
men tally updating algorithms.
5.2 Mountain Car
The Tetris results demonstrate TAMER’s eﬀectiv eness
inadomain with relativ ely infrequen tactions. Con-
versely ,our exp erimen tsinMoun tain Car test itsper-
formance inadomain with frequen tactions. The au-
tonomous algorithm used forcomparison wasSarsa( λ)[16]
with the same function appro ximator (alinear model
overGaussian RBF features, using gradien tdescen tup-
dates), analgorithm that iskno wn toperform wellon
Moun tain Car. TwoSarsa( λ)agen tswere used: one
tuned fortotal cum ulativ eenvironmen talreward across
all previous episo des (Figure 6)after 3episo des and
one tuned for after 20episo des (whic hwewill refer
toasSarsa-3 and Sarsa-20, resp ectiv ely). Wetuned
via ahill-clim bing algorithm that varied one parame-
ter (α,λ,or/epsilon1inthe standard notation ofSarsa [16])
atatime, testing the agen t’sperformance under each
value for that parameter, taking the best performing
value, and then repeating (for ﬁftyormore iterations).
The speciﬁc numberofepiso des (3and 20) were chosen
toexhibit diﬀeren temphases onthe trade-oﬀ between
learning quic kly and reac hing the best asymptotic per-
formance.
The tamer agen tswere shap edforthree runs oftwenty
episo des byeachtrainer. Weconsider the ﬁrst run
apractice run for the trainer and presen tthe com-
bined data from second and third runs. Results are
shown inFigures 5and 6.Figure 5(a) showsthat the
TAMER agen ts,onaverage, consisten tlyoutp erformed
the Sarsa-3 agen tand outp erformed the Sarsa-20 forthe
ﬁrst ﬁveepiso des, after whic hSarsa-20 agen tsshowed
comparable performance. Under the guidance ofthe
best trainers (Figure 5(b)), tamer agen tsconsisten tly
-350-300-250-200-150-100
 0 2 4 6 8 10 12 14 16 18 20Env. Reward (Time to Goal)
Episode NumberMean Reward in Mountain Car
TAMER, Over 2nd and 3rd Runs
Sarsa-3
Sarsa-20
(a)-350-300-250-200-150-100
 0 2 4 6 8 10 12 14 16 18 20Env. Reward (Time to Goal)
Episode NumberMean Reward (Best and Worst Trainers) in M.C.
Best 5 Trainers
Worst 5 Trainers
Sarsa-3
Sarsa-20
(b)
Figure 5:Error barsshowa95% conﬁdence interv al(with aGaussian
assumption). (a) Themean environmen talreward (-1 pertime step)
receiv edfor the Moun tain Car task for the second and third agen ts
(runs) shap edbyeachtrainer under tamer and forautonomous agen ts
using Sarsa( λ),using parameters tuned for best cum ulativ ereward
after 3and20episo des. (b) The average amoun tofenvironmen tal
reward receiv edbyagen tsshap edbythe best ﬁveand worst ﬁve
trainers, asdetermined overallthree runs.
-4000-3500-3000-2500-2000-1500-1000-500 0
 0  2  4  6  8  10  12  14  16  18  20Cumulative Environmental Reward
(Time to Goal)
Episode NumberMean Cumulative Reward in Mountain Car
TAMER, Over 2nd and 3rd Runs
Sarsa-3
Sarsa-20
Figure 6:Mean cum ulativ eenvironmen tal reward receiv edfor the
Moun tain Car task.
outp erform anySarsa agen t,and, under the worst train-
ers, they perform somewhat worse than the Sarsa-20
agen t.Most imp ortan tly,tamer agen ts, onaverage,
also outp erformed eachSarsa agen tinmean cum ula-
tiveenvironmen talrew ard through the length ofarun
(Figure 6). Since eachtime step incurred a-1environ-
men tal reward, Figure 6isalso ameasure ofsample
complexit y.The four trainers who did not haveacom-
puter science background achiev edperformance asgood
ormarginally better than the ﬁfteen who did. Overall,
the results, though less dramatic than those for Tetris,
supp ort our claim that tamer can reduce sample com-
plexit yoverautonomous algorithms.
6. CONCLUSION AND FUTURE WORK
The tamer framew ork, whic hallowshuman trainers to
shap eagen tsvia positiv eand negativ ereinforcemen t,
provides aneasy-to-implemen ttechnique that:
1.works inthe absence ofanenvironmen tal reward
function,
2.reduces sample complexit y,and
3.isaccessible topeople who lackkno wledge ofcom-
puter science.
Our exp erimen taldata suggests that tamer agen tsout-
perform autonomous learning agen tsinthe short-term,
arriving ata“good”policy after very few learning tri-
als. Italso suggests that well-tuned autonomous agen ts
are better atmaximizing ﬁnal, peak performance after
man ymore trials.
Giv enthis diﬀerence instrengths, weaim toexplore how
best touse both human reinforcemen t,H,and, whenavailable, environmen talreward, R,relying onthe for-
mer more heavily forearly learning and onthe latter for
ﬁne-tuning toachiev ebetter results than either metho d
can achiev einisolation. Wewould also liketoimple-
men tTAMER intasks that are curren tlyintractable for
autonomous learning algorithms. Further, wewish to
investigate howtamer migh tneed tobemodiﬁed for
tasks inwhic hatrainer cannot evaluate behavior within
asmall temp oral windo w(e.g., the behavior ishidden
attimes orthe results ofthe behavior are dela yed).
7. ACKNO WLEDGMENTS
Wethank Ian Fasel for his contributions toprevious work
inthis line ofresearc h.This researc hissupp orted inpart
byanNSF Graduate Researc hFellowship and the DARPA
IPTO Bootstrap Learning program.
8. REFERENCES
[1] P.Abb eeland A.Ng.Appren ticeship learning via inverse
reinforcemen tlearning. ACMInternational Confer ence
Proceeding Series ,2004.
[2] D.Bertsek asand J.Tsitsiklis. Neur o-Dynamic Programming .
Athena Scien tiﬁc, 1996.
[3] A.Billard, S.Calinon, R.Dillmann, and S.Schaal. Robot
programming bydemonstration. InB.Siciliano and O.Khatib,
editors, Handb ookofRobotics .Springer, 2008.
[4] B.Blum berg, M.Downie, Y.Ivanov,M.Berlin, M.Johnson,
and B.Tomlinson. Integrated learning forinteractiv esynthetic
characters. Proc.ofthe29th annual conferenceonComputer
graphics and inter active techniques ,2002.
[5] N.Bohm, G.Kok ai,and S.Mandl. Evolving aheuristic
function forthe game ofTetris. Proc.Lernen,
Wissensentde ckung und Adaptivitat LWA,2004.
[6] M.Bouton. Learning and Behavior: AContemp orary
Synthesis .Sinauer Asso ciates, 2007.
[7] W.E.Hockley .Analysis ofresp onse time distributions inthe
study ofcognitiv eprocesses. Journal ofexperimental
psycholo gy.Learning, memory, and cognition ,10,1984.
[8] C.Isbell, M.Kearns, S.Singh, C.Shelton, P.Stone, and
D.Kormann. Cob otinLam bdaMOO: An Adaptiv eSocial
Statistics Agen t.AAMAS ,2006.
[9] F.Kaplan, P.Oudey er,E.Kubin yi,and A.Mikl´ osi. Rob otic
clickertraining. Robotics and Autonomous Systems ,38(3-4),
2002.
[10] W. B.Kno x,I.Fasel, and P.Stone. Design principles for
creating human-shapable agen ts.InAAAI Spring 2009
Symp osium onAgents that Learn fromHuman Teachers ,
Marc h2009.
[11] G.Kuhlmann, P.Stone, R.Mooney ,andJ.Sha vlik. Guiding a
reinforcemen tlearner with natural language advice: Initial
results inRob oCup soccer. InThe AAAI-2004 Workshop on
Supervisory Contr olofLearning and Adaptive Systems ,July
2004.
[12] R.Maclin and J.Sha vlik. Creating advice-taking reinforcemen t
learners. Machine Learning ,22(1):251–281, 1996.
[13] D.Moreno, C.Regueiro, R.Iglesias, and S.Barro. Using prior
knowledge toimpro vereinforcemen tlearning inmobile
robotics. Proc.Towar dsAutonomous Robotics Systems. Univ.
ofEssex, UK,2004.
[14] A.Ng, D.Harada, and S.Russell. Policy invariance under
reward transformations: Theory and application toreward
shaping. ICML ,1999.
[15] J.Ramon andK.Driessens. On the numeric stabilit yof
gaussian processes regression forrelational reinforcemen t
learning. ICML-2004 Workshop onRelational Reinfor cement
Learning ,pages10–14,2004.
[16] R.Sutton andA.Barto. Reinfor cement Learning: An
Introduction .MIT Press, 1998.
[17] I.Szita and A.Lorincz. Learning Tetris Using the Noisy
Cross-En trop yMetho d.Neur alComputation ,18(12), 2006.
[18] A.Thomaz and C.Breazeal. Reinforcemen tLearning with
Human Teachers: Evidence ofFeedbac kand Guidance with
Implications forLearning Performance. AAAI ,2006.


Policy invariance under reward transformations: Theory and application to reward shaping

 !
"#$&%'()*"$	+,.-
 /#0
11*"2"34!5$-
16798&:&;=<?>A@CBDE&F&GIHKJMLON&J	PHQ;=HK:3HRFTSVUW3HQ;XUZY[W3LLO<]\M\^`_bacKdKegfhiKjklfmnjfoklp]klqgkr_smt
mKklpbfhuqgkvexw[_'yT^{z'|lkvy}_shgmKk~zKQ{fhubf|rfw{fhgsf|lfw9^{'Q'bXbQgQ?OR??b?]KOn?3nss K]ss¡`bs¢¤£*¥b¦Q§K¨©¦ª	«
k~q¬cz'cfh¬klm]pbfqegklsz'egfOq®j_bm¯Qkveukr_smnq®dKm¯Qfh°
«
k~j«
a_]¯Kkv±jz=eukr_smnqeg_²e«
f³huf°
z'h´¯µy}dKmj¶egkl_bm_by·z¹¸³zbhgs_=p¯Qfjklqgkr_smcnhg_QjfOqgq#cnhgfOqfhgpsfe«
f _bcQºeukra»z'|¼c_s|rk~jws½¿¾Àe9klq9q«
_°
mÁe«
z=eOÂfOqk~¯QfqZe«
fc_sqgkveukrpsf|rklmKfzbhÃegh´z'mnqy}_bhua»z=egkl_bmÄyMz'akl|rk~z'hÃy}hu_badQeukr|lkrexwe«
f_shgwsb_smKfjzbmZzb¯K¯»zhgf°
zbhu¯y}_bhTegh´z'mQºqgkveukr_smnqÂfe°
ffmÄqeuz'egfq/e«
z=ek~qfÅQcKhufquqklÂK|lf9zbqe«
f/¯QkrÆfhgfmnjfklmp=z'|ldKf_'yz'mÈÇ'ÉÊËÍÌÉgÇ=É¶Î¹c_'eufmQºeuklzb|sy}dnmnj¶eukr_smÃzbcKcK|lkrfO¯eg_¼e«
_sqgf{qeuz'egfq½#ÏKdKhge«
fhºa_bhufb`e«
klq»k~qZq«
_°
mÐeg_µÂfCzÑmKfjfquqgzbhgwÄj_bmQº¯Qkregkl_bmÁy}_sh»krm]p=z'huklzbmnjfsTklmÁe«
fCqgfmqfIe«
z=e[z'm]w_be«
fh[euhuzbmnqxy}_shga»z'egkl_bm¤a»zXwÁw]klf|~¯ÒqgdKÂ_scQegkla»z'|c_b|lkljkrfOq{dKmK|lfquq{y}dKhe«
fh zsqgqgdKacQeukr_smnq	z'hufa»zb¯QfzbÂ_sdQeÈe«
fÓdKmn¯Kfhu|rw]klmKÔ¸³o*Õ#½ª	«
fOqfÓhgfOqdK|reuqq«
f¯|lkr«
eC_bme«
fÈcKh´zbj¶eukljfµ_'yÉgÖ×TÇ=ÉgØÚÙÛnÇuÜ·ÝËÍÞ]ß' z²afe«
_Q¯Údnqgf¯ÁklmÚhgfkrmQy}_shujfafm?eZ|lfzbhgmQºklmK°
«
fhgfÂ?wàzb¯K¯Qkregkl_bmz'|euhuzbkrmKklmKhuf°
z'h´¯Kq¹z'hufdnqgf¯¿eu_ÁsdKk~¯Qf²e«
fµ|lfzbhgmnkrmKÚzbbfm?eO½¾ámâcnzbhºeukljdK|lzbh*q_saf°
f|l|vºÀ]mK_°
m¬ãgÂKdKsquäµklm¿huf°
z'h´¯q«
z'cnkrmKCcKhg_Qjf¯QdKhufq/z'hufZq«
_°
mÑeu_Cz'huklqgfy}hu_bamK_smQºÀc_begfm?euklzb|vºÀÂnzbqgf¯.huf°
z'h´¯KqIzbmn¯¬afe«
_Q¯Kqzbhgf*bklpbfm[y}_sh j_bmqxeuhgdnjegklmK»q«
z'cKklmKÃc_begfm?euklzb|lqj_bhuhgfOqc_bmn¯KkrmK{eu_¼¯Qk~qxe´z'mnjfºÀÂnzbqgf¯¹zbmn¯qdnÂKb_?z'|rºÂnzsqfO¯«
fdKhuklqegk~jq½)åÑf¤q«
_°
e«
z=eÈqgdnj«
c_'ºeufm?egk~z'|~q jzbm9|lfzs¯9eg_»qdnÂnqxe´z'm?egk~z'|hgfO¯Qdnj¶eukr_smnq	klm|lfzbhgmKklmKÃegklafb½æ çsè
¦Q§Kéêë©¦Kìéè¾ámÚqfOí?dKfm?egk~z'| ¯Kfjk~qgkr_smÐcKhu_bÂn|rfaq	qdnj«
zsq»z'hufqegd¯QkrfO¯klme«
f¯Qw]mnzbaklj{cKhu_bshuzbaÃaklmK¹z'mn¯ÃhufklmQy}_bh´jfaÃfm?e#|lfzbhgmKºklmK|rkregfhuz'egdKhufq=e«
f»ãxe´zbqg]ä*k~q#hgfcKhufqgfm?egfO¯/Â]we«
fÉgÖ×TÇ=ÉgØî´ïQÞðÌËMñ=Þn½Cò*klpbfm²e«
f[hgf°
zbhu¯Cy}dKmnjegkl_bmÄzbmn¯µza_Q¯Qf|`_'ye«
f*¯Q_sa»z'klmV?e«
f*_bcQeukra»zb|c_b|lk~jwklq	¯QfegfhgaklmKf¯R½m[f|lfºafm?euzbhgwÃe«
f_shgfegk~jz'|Rí?dKfOqxeukr_smZe«
z=ez'huklqgfq`klq`e«
klqóå«
z'e
y}huff¯K_ba®¯Q_°
f«
zXpbf klm9qgcfjkry}w]krmn/e«
f¼huf°
z'h´¯y}dKmnjegkl_bmVqgdnj«
e«
z=ee«
f_bcKegklazb|Vc_b|lkljwZhgfa»z'klmnq	dKmnj«
zbmKbfO¯nô¾ámõe«
f®±nf|~¯ö_byÁdQeukr|lkrexw÷e«
f_shgws°
«
klj«
qegd¯QkrfOqÔcKhukrºa»z'hukr|lwÈqgkrmKs|rfºÀqegfcÐ¯QfOjk~qkl_bmqTe«
fj_shghufqgc_bmn¯QklmKÑísdnfqºegkl_bmøy}_bh²e«
fÚdQeukr|lkvexwÔy}dKmnj¶eukr_sm®jz'm!Âf¤zbmnq°
fhgfO¯ÔpbfhgwqgkracK|lwb½ Ïn_bhùqgkrmnb|lfºáqxeufcú¯QfOjk~qkl_bmnq°
kve«
_bdQe!dKmnjfhgºeuzbkrm?exws¿z'm]wõa_bmK_beg_smKklj¬egh´z'mnqy}_bhua»z=eukr_smú_bmúdKegkl|rkregklfq|lfzXpbfOq9e«
fÈ_scQegkla»z'|¯QfOjk~qkl_bmâdKmj«
z'mKsf¯Rû°
kve«
dKmnjfhgºeuzbkrm?exwsI_smK|lw!c_sqgkregklpbfÒ|lkrmnfz'hÐegh´z'mqxy}_shga»z=eukr_smnqÈzbhgfz'|rº|l_°
fO¯Òü ps_bmý fdKa»z'mKmz'm¯9¸_shgsfmnqegfhgmþO'sXÿ½ª	«
fqgfhufqgdK|ve´q«
zXpsf{klac_she´z'm?eTklacK|rk~jz'egkl_bmnqy}_bhT¯QfOqklbmnkrmKfp=z'|rºdnz'egkl_bm¤y}dKmnjegkl_bmnq[krmÚ?z'afq	f|lkljkveukrmnÈdQegkl|lkvexwÁy}dKmnjegkl_bmnqy}hu_ba«
dKa»z'mqnzbmn¯9azbm]wZ_'e«
fh zbhgfOzbq½ª
__bdKhV]mK_°
|lf¯Kbfbe«
fTí?dKfqegkl_bm¹_'yQc_b|lkljw¼klm]p=z'huklzbmnjf#dKmQº¯Qfh3huf°
z'h´¯*y}dKmnjegkl_bmegh´z'mnqy}_bhua»z=eukr_smnq«
zsq3mK_'e#Âffmy}dK|l|rwfÅQcK|l_bhuf¯Óy}_bhàqfOísdnfm?egk~z'|/¯Kfjk~qgkr_smcnhg_sÂK|rfa»q½ Õ#_b|lk~jw?ºcKhufqgfhup]krmKCegh´z'mnqy}_bhua»z=eukr_smnqzbhgfIklaÃc_bhgeuzbm?e»z'eÃ|lfzsqxe»klme«
fqgfz'hufzbqó
ª	«
fõe´zbqg)_'y Ù¶Ì É¶ïnðÌ ïQÉuÇ¿ÖÙ¶Ì Ë»Ç=ÌËMñ=Þ)_by!¸³o*ÕTqü dnqeVþObbXÿklm]pb_s|rpsfq³hufj_=pbfhgklmK¤e«
fÐa_Q¯Qf|»zbmn¯huf°
z'h´¯y}dKmj¶egkl_bm!y}hu_ba _sÂnqgfhupbf¯ù_bcQeukra»zb|»Âf«
zXp?ºkl_bhO½  iQff¿zb|lqg_âe«
f¯Qk~qujdnquqkl_bm®_byµËÍÞ
	=ÖÉ´ÙÖ¿ÉgÖËÍÞnÝîñ'Égð´Ö»ÖÞÌlÖuÇ'É¶ÞnËÍÞ]ßÒklmAü  dnqgqgf|l|Vþs
OÿÀ½  Õ#_b|lk~jw?ºcKhufqgfhup]krmnegh´z'mnqy}_bhua»z=egkl_bmq¯QfeufhuakrmKf[e«
f9fÅ]egfm?eeu_°
«
klj«
zÃhgf°
zbhu¯Zy}dKmnjegkl_bmjzbmIÂfhufj_=psfhuf¯R½
ª	«
f9cKh´zbj¶eukljf[_byÉgÖ×TÇ'ÉgØàÙgÛKÇ´ÜËÍÞ?ßIklmÈhufklmQy}_bh´jfafm?e|lfzbhgmKklmKIj_bmqk~qxe´q_by`qgdKcKcK|lw]krmKzb¯n¯Qkveukr_smnz'|3huf°
z'h´¯Kqeu_Iz9|rfOz'humKklmK9zbbfm?e¼eg_bdKk~¯QfÃkve´q*|rfOz'humKkrmn[cKhu_QjfquqÂfws_bmn¯Ãe«
_sqgfqgdKcKcK|lkrfO¯»Â]w/e«
f dKm¯Qfhu|rw]klmK/¸³o*Õ#½?¾Àek~qTklac_she´z'm?eeu_/dKmn¯Kfh´qxe´z'mn¯e«
fklacnzbje`_byq«
zbcKklmK_sm[e«
f|rfOz'humKf¯9c_b|lkljwb½! "$#%'&()+*,(.-/%0#21!1!+3456#"4(72#%45#%()894:-;<*=#%>!6#1?<"!@?>A@BA#%"62C"DEF"G!H()<*I1?2 458JB
K 72#2 )L4'*,4:"G!45(NMDOPCQ6RTS
U2-"G!A%4V4(W#% 1!"46#% X1? :Y48<B[Z 4()VG$#2()>]\F<E^Q+__%`WE!a.45  45#2b#2()>$c=#%4:>
EdQ__%`fehg
ª	«
k~q3cnz'cfh&y}_Qjdnqgfq3cKhukra»zbhgkl|rw¹_smhgf°
zbhu¯/q«
zbcKklmKn°
«
k~j««
zsqe«
fÃc_begfm?euklzb|&eu_9Âf»z[psfhuwc_°
fhgy}dK|&egfj«
mKklí?dKfÃy}_shqujz'|lklmKâdKcøhufklmQy}_shujfafm?eµ|rfOz'humKklmK¿afe«
_Q¯Kq²eu_«
z'mKº¯Q|lfCj_sacK|rfÅÐcKhu_bÂn|rfaqàü o¼_bhuklb_z'mn¯^`_b|l_baÂfeegkVþOb'Q¸³z'euz'huk~j'Rþs'Qiz'mn¯Q|/j'p9z'mn¯I |~qxeuh<jbaVþs
OÿÀ½A i]klakr|~z'hk~¯Qfzsq«
zXpsfÄz'huk~qfmkrmùe«
fÁz'mKkla»z'|egh´z'klmKkrmnÓ|lkveufh´z=egdnhgfsûqgffÈü iQz'Qqgkl¯KzÃfez'|½rRþOb?ÿ¼y}_bh[zµ¯Kklqujdnquqgkr_smV½ lkyÍegfmVzpbfhgwqgkracK|lf¼cz=eeufhumZ_'yVfÅ]egh´zhgf°
zbhu¯nqTqgd?mZjfqTeg_hgfmn¯Qfhqegh´z'kl«
ey}_sh°
z'h´¯âz'mÔ_be«
fh°
k~qfÁj_saÃcn|rfegf|lwklmseuhuzsj¶e´z'ÂK|lfcKhu_bÂK|lfa½ª
_Zqff°
«
w[c_b|lkljw9klm]pXzbhgk~z'mjf¹klq krac_bhgeuz'm?eklm³q«
z'cKklmKj_smnqgkl¯Qfh#e«
f y}_b|l|r_°
klmK¹fÅKz'acK|lfq#_byÂndKsq#e«
z=eTjz'm»z'huk~qfsóü  z'mn¯K|njbp[z'mn¯I |~qeghj'aVþs
Oÿ`¯KfqujhukrÂfqzqgwQqxeufa e«
z'e|lfz'humnq/eg_Ñhgk~¯QfIzàqgkradK|~z=egfO¯ÈÂKk~jwQj|rf9eg_ÑzCcnzbheukljdK|~z'hÃ|r_bºjz'egkl_bmV½ª
_qcffO¯ÑdKcµ|rfOz'humKklmKnRe«
fwàcKhu_=p?k~¯QfO¯³c_sqgkveukrpsfhuf°
z'h´¯Kq°
«
fmnfpbfhRe«
f`z'sfm?eVa»zb¯QfTcKhg_sbhufquqeg_°
z'h´¯Kq·e«
fb_?z'|½ª	«
fz'sfm?eT|lfz'humKfO¯Ãeg_hukl¯Kfklmegklm?w»jkrh´j|lfqTmKfzbhe«
fqeuz'hge qxe´z=eufÂfOjz'dqf¹mK_cfmnzb|vexw°
zsq`klmnjdKhghuf¯[y}_bhhgk~¯QklmKz°
zXwy}hg_sa e«
fb_?z'|½`Ôqgkrakl|lzbhcKhu_bÂK|lfa _]jjdKhuhuf¯°
kve«zIq_Qjjfhgºcn|lzXw]klmKZhu_bÂ_'e¼ÂfklmKIegh´z'klmKfO¯³Â]w³o¼zXp]k~¯Cmn¯Qhufz'm¯à¼qxeuhg_ª
f|r|lfh0Mcfhuqg_bmz'|j_saÃadKmKk~jz'egkl_bmo½{fjz'dnqgfc_squqfOqgqgkr_smkrmCqg_Qjjfh k~qklac_she´z'm?ee«
fwIcKhg_=p]k~¯Qf¯zZhgfº°
z'h´¯/y}_sheg_sdnj«
klmKe«
f Ânzb|r|½ª	«
fz'sfm?e|lfz'humKfO¯»z¹c_s|rk~jw°
«
fhgfÂ?wIkre¼hufa»z'klmKf¯mnfÅ]eeu_»e«
fÃÂnzb|r|3z'mn¯Úãgp]krÂKh´z=euf¯R äeg_sdnj«
klmK*e«
fÂz'|l|Kzbq3y}hufí?dKfm?eu|rw/zsq3c_?qgqgkrÂn|rfs½ª	«
fOqf	c_b|lkvºjklfqzbhgfj|rfOz'hu|rw»mK_be _bcQeukra»zb|·y}_bh	e«
f_bhukrskrmnzb|R¸³o¼Õ#½ª	«
fOqf`fÅKz'acK|lfq&qdKsbfOqxeVe«
z=e&e«
f	q«
z'cKklmK huf°
z'h´¯KqRadnqe_bÂfwjfhe´z'klmj_bm¯Qkveukr_smnqkrye«
fwIzbhgfmn_'eeg_[aÃk~qg|rfOzb¯Ie«
fz'sfm?eàklm?eg_¿|rfOz'humKkrmn¿qgdKÂ_scQegkla»z'|c_b|lkljkrfOq½ª	«
f¤¯QkpmÃºjdn|vexw°
kve«
c_sqgkveukrpsfºÀhgf°
zbhu¯jwQj|lfqT|rfOzb¯Kq_smKfeu_/j_smnqk~¯Qfhhuf°
z'h´¯Kq»¯Qfhuklpbf¯Áy}hu_ba zÈj_bmnqgfhup=z=egklpbfc_'eufm?egk~z'|5q¤e«
z'ek~q·e«
fhgf°
zbhu¯Iy}_sh*fÅQfjdKegklmKIzZegh´z'mnqgkregkl_bmCÂfe°
ffm³e°
_qeuz=eufqCklqr}fquqgfm?egk~z'|l|rw^9e«
fÐ¯QkrÆ·fhufmnjfÈkrmÔe«
fÈp=zb|rdKfÄ_'yz²c_'egfm?egk~z'| y}dnmnj¶eukr_smÒz'cKcK|lklf¯Áeg_µfOzbj«
qeuz=eufb½¾ÀeZegdnhgmnq_bdKeàe«
z=e²mK_'e²_bmn|rwk~qàe«
k~q²z¿qgd?mZjklfm?eµj_bm¯Qkveukr_smÔy}_shbdz'h´z'm?egffklmK/c_b|lkljw[klm]pXzbhgk~z'mjf*dKmn¯Qfhhgf°
zbhu¯Zegh´z'mnqy}_bhgºa»z=eukr_smnq¹ÂKdKeIe«
z=eOzbquqdnaÃklmKÁmK_¤cKhukr_shI]mK_°
|lf¯KbfÑ_'ye«
fÑ¸³o*Õ#{e«
k~q9klqz'|~qg_ÄzÒÞ·Öuð´ÖÙ´ÙÇ'É¶ÎCj_smn¯Qkregkl_bmÚy}_sh9ÂfºklmKÈz'ÂK|lfIeg_µa»z'sfqdnj«
zÑbdz'h´z'm?egffb½Òi]fjegkl_bmÒàbklpbfOqe«
fÄ¯Qf±mKkveukr_smnq³mnff¯Kf¯eg_Òqeuz'egf²e«
klqàj|~z'kla4cnhgfOjk~qf|rwsz'm¯Ñi]fOj¶eukr_smsqxe´z=eufq¹z'mn¯àcKhu_=pbfOqe«
fZj|~z'klaI½i]fjegkl_bmàq«
_°
q«
_°
eu_j_bmnqeghudnjeq«
zbcKkrmn9c_'eufm?egk~z'|~q¹_'y{pXzbhgkl_bdq]krm¯Kqz'm¯µ¯Qfa_bmqxeuhuz'egfq*e«
fklhf%m»jzbjwCklmÈqcffO¯QkrmndKc|lfz'humKklmK_bmµqg_baf[qgkracK|lf[¯Q_sa»z'klmnq½Ï3krmz'|l|rwsiQfj¶eukr_smutj_smKmKfOj¶euqC_bdKhàhgfOqdn|ve´qeg_¿fÅQk~qxeukrmK¿z'|lb_bhukre«
a»q³qgdnj«
zsq¯KpXzbm?euz'sf|rfOz'humKkrmnÁü 	z'klhu¯RRþObbXÿ¼z'mn¯wvKºc_b|lkljwÄkregfhuz'ºegkl_bmøü {fhgeuqgf=zsq{zbmn¯ª
qkreuqgkl?|lk~qRþObxOÿÀzbmn¯Òj|l_sqgfq°
kve«¯Qk~qgjdnquqkl_bmz'mn¯[y}dQeudKhgf°
_bhu·½
y z
§^{i|gì}.ìè
¨#§nì{¥~
Bh GI<)383J}UOJh8&L¾ámÃe«
k~qqgfjegkl_bmV°
fcKhu_=p]kl¯Qf	qg_baf	_'ye«
f ¯Qf±nmKkregkl_bmnqdnqfO¯e«
hg_sdK«
_sdQee«
f[cnzbcfh&y}_Qjdnqgkrmn³_bmÑe«
f9jzsqf»_by	±nmKkregfºqeuz=eufZ¸³z'hub_=p³¯Kfjk~qgkr_smàcKhu_QjfOqgqgfq ¸³o¼Õ`q<½9i«
zbcKklmKklq_'yklm?egfhgfOqxe¼eg_IdnqklmCÂ_'e«
±nmKkregfºÀqeuz'egfz'mn¯³krmQ±mKkveufºáqxe´z=eufcKhu_bÂK|lfa»qÂndQe²e«
f¤dKmn¯Kfhu|rw]klmK¸Co¼Õ e«
f_bhuwây}_bhÑe«
fklmQ±nmKkregfºÀqeuz'egf9jzbqgf[k~qqgklbmKkr±jzbm?eg|lwàa_bhuf9¯Kk5mZjdn|veOfpsfmklm¤e«
fCzbÂnqgfmnjfI_byq«
zbcKklmKn½Óýfpsfhge«
f|rfOqgq	_bdKhZzbmnz'|lw?ºqgklq»zbmn¯ÁaÃfe«
_Q¯Q_s|r_sbwÈzbhgffzsqkl|lwÈbfmKfh´z'|lkpf¯Èy}hg_saúe«
f±nmKkregfeg_/e«
f¼krmQ±mKkveufºáqxe´z=euf¼qgcnzbjfjzsqf_smnjfe«
f¼dKmn¯Qfhg|lw?ºklmK[¸³o*Õ¤e«
f_bhuwIklq|lzbkl¯_sdQez'mn¯°
f°
kl|r|3afm?egkl_bme«
klqz'?z'klmÄ|~z=eufhOû`ÂKdKe»y}_bhmK_°
|lfe»dnq»qeuzbhe_bdKh»¯Qf±nmKkregkl_bmnq°
kre«
fÅQcK|lkljkveu|rwIj_smnqk~¯QfhgklmKÃ_bmK|lw»±nmKkregfºÀqeuz'egf¯Q_sa»z'klmnq½}±nmKkregfºáqeuz=euf;HK;dWÚ:&<WbJ LJ8w3;b<]LOLL
G
k~qzeudKcK|lf]=<6J=J¶°
«
fhufbó;Áklq¼zÃ±nmKkregfÃqfe_'yLUHQUO<]L
û

2%%2
?sk~q z»qgfe_by ¢¡ÒH^bUJ83L
û£
¤=¥¦§:%¨ ©«ª=
ªN
z'hufZe«
fmKfÅ]egºÀqeuz'egfU;=HK8¬LOJ}UJ8N3;fo#H?3J \MJMUJM<]L
°
kve«
¤¥¦®©¯/¼skrp]klmK[e«
f»cKhu_bÂnz'ºÂKkl|rkrexwZ_byegh´z'mqkregkl_bmKklmKeg_»qeuz=euf6©2¯VdKc_sm[e´z']klmK»zbj¶eukr_sm
klmÈqxe´z=egf©bû=Nª° ^þÿ`k~q¹e«
f:&J L2!W38·UA±ÀH^bU2;
û{z'mn¯.qgcfOjkr±nfqe«
f9hgf°
zbhu¯Ñ¯Qk~qxeuhgklÂKdQeukr_smnq½³Ïn_bh/qgklaÃcn|rk~jkrexwb°
f°
kl|r|zbquqgdKafZhgf°
zbhu¯nqz'huf9¯QfegfhgaklmKklqegk~j'klm°
«
klj«
jzbqgf¿klq{zÂ_bdKmn¯Kf¯»hufzb|Ky}dKmnjegkl_bm[jz'|l|rfO¯Ãe«
f;=<s>HQ;=:²±ÀW&8F¬UJ8
½T¾áme«
f|lkveufh´z=eudKhgfsKhuf°
z'h´¯»y}dKmnjegkl_bmnq¼z'huf¹exw]cKkljz'|l|rw°
hukvegegfm¢ó!u³'µ´ ¶¸·{°
kre«
H©!
TÂfkrmKÃe«
f¹hgf°
zbhu¯hufjfkrpsf¯dnc_smCe´z']krmnIzsj¶egkl_bm
krm²qeuz=eufH©s½ª	«
_bdK«°
f°
kl|r|&_'yÍegfm°
hgkregfhuf°
z'h´¯[y}dKmnj¶eukr_smnq klmIe«
k~qy}_shga°
f°
kl|r|z'|~qg_Izb|r|l_°
zIaÃ_shgfÃbfmKfh´z'|&y}_bhuaó¹º³A»³¼µ´ ¶½·{°
kre«
H®©
<©¯nTÂfklmK/e«
f¹hgf°
zbhu¯hufjfklpbfO¯»dKc_bmZe´z']klmKzbjegkl_bm
klmIqeuz'egf©zbmn¯9egh´z'mnqgkregkl_bmKklmKÃeg_»qeuz=euf6©2¯½ò*klpbfmÒzÑ±nÅ]fO¯Úqfe[_'yzbjegkl_bmnq' zb\MJ¾
_=pbfhZzµqfe_'yqeuz'egfqøklq9z'm]wÄy}dnmnj¶eukr_smÀ¿ óÁ£´ ¶Â½Ôý_'egf³e«
z'ec_b|lkljkrfOq»z'huf¯Kf±nmKfO¯Á_=pbfhqeuz=eufqZzbmn¯ÐmK_'e»_=pbfh»¸³o¼Õ`qqg_µe«
f²quz'af³c_b|lkljwÁa»zXwÁÂfÑz'cKcn|rklf¯¤eu_Èe°
_Ä¯QkrÆfhgfm?e¸³o*ÕTqqg_|r_smKzbq*e«
fe°
_¸³o*ÕTqdqfÃe«
fZqgzbaÃfZqeuz'egfOqz'm¯»zbj¶eukr_smnq½#ò*klpbfmZz'm]w/c_b|lkljw]¿_=psfhTqeuz'egfqÃ²z'mn¯»zbm?w¸³o*ÕÄ l®=JF<ÅdnqklmKe«
fqgzbaÃf9qxe´z=egfOq/zbmn¯zbjegkl_bmnq°
fa»zXwZe«
fm³¯Qf±nmKfe«
fKHK\ W&<±ÀW&8F'UOJh8rÆ6ÇÈ
°
«
k~j«
fpXzb|rdz=egfO¯³z=ez'm]wqeuz=eufH©ÃskrpsfqÆ6ÇÈ
©fÁÄÉ¼ü ÊbË¹Ê2ÌË

ÌÊÍË
%%RûJ¿ <©ÿÀ°
«
fhufÁÊÎ&k~q`e«
f¹huf°
z'h´¯hgfOjfkrpsf¯_bm[e«
fÅÏe«
qegfc9_'y3fÅQfjdQegklmKe«
f¹c_b|lkljw0¿y}hu_ba qeuz'egf©bz'm¯e«
f¼fÅQcfj¶e´z=eukr_sm»klq`_=pbfhe«
f*qxe´z=eufºegh´z'mnqgkveukr_smnqe´z'sfmdKc_bmZfÅQfjdKegklmK¿½#åÑfe«
fm[¯Kf±nmKfe«
f
3UJÐÈHK\FKHK\ W&<±ÀW&8F'UOJh8
eu_ÂfÆ$ÑÈ
©f,ÓqdKcÇ
ÆÇÈ
©f¶Xe«
fÓÒ¬<±ÀW&8F'UOJh8
fp=zb|rdnz'egfO¯9z'ez'm]w[©ªÔ=
ª¢zbqÒ
ÇÈ
©!
bºÉÃ¥Õ/Ö×)ØÙfÚÜÛ Ý·ü H©!
©
¯Ë
Æ
ÇÈ
©
¯Àÿ xþf°
«
fhuf`e«
fmK_beuz=eukr_smH©
¯oÞ
¤=¥¦§:#afzbmnq&e«
z'eÃ©
¯klq¯Qh´z°
mzbjj_shu¯QklmKeg_*e«
f ¯KklqeghukrÂndQegkl_bm
¤F¥¦§:sz'm¯e«
f
3UJÐÈHK\
Ò¬±W38F'UJ8
zbq,ÒÑÈ
©!
bÒqdncÇ
ÒÇÈ
©!
¶½#Ï3krmz'|l|rws°
f¯Qf±nmKf{e«
fo&UOJhÐÈHn\oÃ\ Jh¾
y}_bhz'mÃ¸³o¼Õ« zsqF¿ÑÈ
©f,z'hua»z=Å
¦ßàÒÑÈ
©!
¶½ª	«
f_bcQeukra»z'|c_s|rk~jwZa»zXwZmn_'e ÂfdKmKk~í?dKfb=zbmn¯°
fTaÃ_shgfsfmKfhuzb|r|lw¼quzXw¼z c_s|rk~jwX¿klq_bcQeukra»z'|klmN kvy]¿,©f.ª®z'hua»z=Å
¦ßàÒÑÈ
©!
y}_sh9zb|r|©ºª	½á
zsqxeu|rws°
«
fmÑe«
f9j_bm?eufÅ]e/¸³o*Õklq/j|lfz'hO°
f»a»zXwàzb|lqg_¯Qhu_bce«
f]¿ºáqdKÂqgjhgklcQez'mn¯°
hukregfÆÇ
huz'e«
fhe«
zbmÆ6ÇÈ
fe´j'½åÑfzb|lqg_[mnff¯Ñqg_baf[M|lzbhgsf|lw³qxe´z'mn¯Kzbhu¯ohufbdn|lzbhgkrexwj_bmKº¯Qkregkl_bmnq¼q_9zbqeu_Za»z'sfqdKhuf/zb|r|&_'y#e«
fÃz'Â_=pbf¯Qf±nmKkregkl_bmnqa»z'sfqgfmnqgfb½¹ÏK_sh¼dKm¯Qklquj_sdKm?egfO¯²«.þ¼¸³o*ÕTq°
fÃzbqºqgdKafZe«
z=e0j_smse´z'klmnq/zà¯Qk~qxeukrmKsdKk~q«
fO¯Èqeuz=euf©2â9jz'|l|rfO¯z'mH?#Ln;f#JM83ELUOHQU<
sqg_¹e«
z=ee«
f¸³o*Õâãgqeg_scnqgä¹z'yÍegfh`zegh´z'mqkregkl_bmklmseu_'©â
°
kve«
mK_y}dKhge«
fhhuf°
z'h´¯Kq{e«
fhgfOz=yÍeufhO½¸_shgf_=pbfhO&zbszbkrm²y}_sh/dKmn¯Kklquj_sdKm?egf¯Ä¸³o¼Õ`q°
f9zsqgqgdKafz'|l|*c_s|rk~jklfqIz'huf3;foT<?;
 afzbmKkrmnÈe«
z=e9dnc_smÒfÅQfjdQeºklmKÑzbm]wµc_s|rk~jwÈqeuzbheukrmKCy}hg_saúz'm]wÈqeuz=eufb°
f°
kr|l|°
kve«cKhu_bÂnzbÂKkl|rkrexwàþfpsfm?egdnzb|r|lwIegh´z'mnqgkregkl_bm³klm?eg_©â
½¹i]klmnjf/e«
klqk~qhufzb|r|lw²zàj_bmn¯Qkregkl_bmÄ_bm²/°
f°
kr|l|	krmÄe«
k~qÃcnz'cfhqgzXwe«
f9ÌÉgÇ=ÞnÙ¶ËÍÌ ËMñ'Þ[Ü·ÉgñbÊ´ÇsÊË ËÍÌËMÖÙãÔz'huf/cnhg_scfhkrye«
k~q¹j_smn¯Qkrºegkl_bm«
_b|~¯Kq½/o¼klquj_sdKm?egf¯Ñ¸³o*ÕTq«
zXpsf/mK_j_shghufqgc_smn¯QklmKz'Âq_shgÂKklmK³qeuz'egf[zbmn¯Èz'hufZz'|°
zXw]qkrmQ±mKkveufº«
_bhukn_bmûmK_'eufe«
fhufy}_bhuf*e«
z'ey}_bh	e«
fa°
fjz'm°
hukveufAä
©2â
N	½ª	«
fÚz'Â_=pbf°
fhgfÐe«
f¤qeuzbmn¯Kzbhu¯ùhgfbdK|~z'hukvexwj_bmn¯Qkregkl_bmqmKff¯QfO¯ y}_bh ¸³o*ÕTq°
kre«æå
ÞnËÍÌáÖ qxe´z=egfõqgcnzbjfqlMqgffbfb½ n½Qü i]dQegeg_bmzbmn¯9	zbheu_nRþs
OÿÜ¶°
«
klj«
klq&e«
f{jzbqgf°
«
k~j«°
f«
zs¯³fÅQcK|lk~jkreg|lwquz'k~¯°
f°
_bdn|l¯y}_Qjdnq*_bmV½¹ÏK_sh*¸³o*ÕTq°
kre«
klmQ±nmnkveufÃqeuz'egfqcnzsjfOqa_bhuf°
_bdK|~¯ÂfmKff¯QfO¯Róy}_shfÅKzbaÃcn|rfsQkrmIe«
f*dnmn¯Qk~qgj_bdKm?egfO¯9jzbqgfb]e«
ffÅQcfj¶e´z=eukr_sm9klm_bdnh`¯Qf±nmKkregkl_bm»_'yÆ6ÇÈ
©f,µÉü Ê
Ë
¹ÊÌË

ÌÊÍË
%2Rû¿ ©ÿa»zXw²mK_'efpsfmÄfÅQklqe½
Ìª	«
fqgf9k~quqdKfOqÃmKffO¯µeg_ÑÂf9cKhu_bcKºfhu|lwàzb¯n¯QhgfOqgqgf¯àÂfy}_bhuf°
fZjz'm²fpsfmµ¯Qf±nmKfe«
krmK?qqgdnj«zbq_bcQeukra»zb|sc_b|lkljkrfOq'zbmn¯¹fÅKjf|l|rfmse#q_sdKh´jfqVy}_bhe«
k~qa»z'egfºhuklzb|klmnj|rdn¯Kfü {fhgeuqgf=zbq·þs!t?^ç fhgmÁè z'mn¯Qf2ºá
fhga»zK·þO
b]{fhgeuqgf=zbq	zbmn¯i«
hufpbfsRþ]
OÿÀ½!{dQeIdKmQy}_bhgegdnmnz=euf|lwbfÅ]ºcK|~z'klmKklmK³e«
f9klmQ±nmnkveuf.¨ ¨&jzbqgf[klmÈy}dn|r|	bfmKfh´z'|lkrexw°
_bdK|~¯hufí?dKklhgf#a_bhufafzbqgdKhuf#e«
f_bhuwe«
z'm°
f°
klq«
eg_ ¯Qf|rpsfkrm?eu_«
fhgfssz'm¯°
f_bmn|rwÃj_baafm?ee«
z=eO°
kre«
e«
fz'cncKhg_scKhuklz'egfbfmKfh´z'|lknz=eukr_smnqV_byKe«
f`hufí?dKklhgfO¯hufbdn|lzbhgkrexw*j_bmn¯Kkveukr_smnq_sme«
f{¸³o*Õ#=z'|l|]_'yn_sdKh&hufqgdK|reuq3z'hufTfOzbqgkr|lw¹bfmnfh´z'|lkpf¯*eg_¼e«
fklmQ±nmKkregf'¨ ¨Qjzbqgfb½ª	«
hu_bdK«
_bdQe{e«
klqcz'cfhO°
f°
kl|r|«
_°
ºfpsfhj_sm?egklm?dz'|l|rw[¯Khuz°
|lklmKQq	eg_«
_°
e«
fhufqgdK|reuq	a»zXwZÂfcKhu_=pbfO¯Zy}_sh krmK±nmKkregf[¨ ¨vQe«
_sdK«°
f¯Qfy}fh e«
fa_bhufbfmKºfh´z'|&cKhu_?_byMqy}_sh¼klmQ±nmKkregf¢¨ Á¨Qeu_Ze«
fy}dK|l|#cnz'cfhO½ÏK_sh¼mn_°
°
f mK_'euf _bmn|rwe«
z'ey}_bhTe«
fkrmQ±mKkveufºáqxe´z=euf¼jzsqfsszbm»krac_bhgºeuzbm?e[z'mn¯¤dnqgfy}dn|j_smn¯Qkregkl_bmÚk~qe«
z'e»e«
fChgfkrmQy}_shujfafm?e´qz'huf¹Â_sdKmn¯QfO¯Ikrmz'Âq_s|rdQeufp=z'|ldKfbûQe«
k~q°
kr|l|VÂfafm?egkl_bmKfO¯z'?z'klm[|~z=eufhklm9e«
fcnzbcfh½é<ê=G!4Á4ë4(¢#]454 5#%<()Å"]"G!ì+#2(!í$2- #Hî=#2!8JGB>!4C"4@)"4(H(!2",<34C"4()V!g
~
B~
SN3H?3J 8&EµY9<?>/HQ;=:&L¾ám¿e«
k~qqfOj¶eukr_smV°
f²krm?euhg_Q¯Qdnjfà_sdKhIy}_bhua»z'|*y}huzbaf°
_bhu_'yCq«
zbcKkrmnùhuf°
z'h´¯Kq½2¾ám?eudKkveukrpsf|lwb°
fÓz'hufÒeuhgw]klmKeu_|lfz'humzÚc_b|lkljwÓy}_shàqg_bafÈ¸³o*Õï ð J6J=Jz'm¯°
f°
k~q«
eu_«
f|lcÚ_bdnhZ|rfOz'humKklmKµz'|lb_bhukre«
aÂ?wÐbklp?ºklmK³kreÃzb¯n¯Qkveukr_smnz'|/ãgq«
z'cKklmK?ähgf°
zbhu¯Kq°
«
klj«°
kl|r|«
_bcfºy}dK|l|rw/bdKk~¯Qf{kve#eg_°
z'h´¯Kq&|rfOz'humKklmK¼z¼b_]_Q¯'M_bh#_bcQeukra»zb|/c_s|vºk~jw³yMzsqxeufhO½ª
_y}_bhuazb|rknfÃe«
klq°
fZzsqgqgdKafÃe«
z'e3huz'e«
fhe«
z'mIhgdKmnmKkrmn_sdKh`hufklmQy}_shujfafm?e	|rfOz'humKkrmnz'|lb_shgkre«
a!_sm ñ J6J=J°
f°
kr|l|hudKmâkve³_smqg_baf¤Ì ÉuÇ=ÞKÙ¶Ýîñ=É+»Ö´Ø¹¸³o*Õw¯dI®=JF<X¯n¶°
«
fhgfÅÅ¯oµË.òk~q¹e«
f[hgf°
zbhu¯³y}dKmnj¶eukr_smµkrm²e«
f»euhuzbmnqxy}_shgafO¯à¸³o*Õ#zbmn¯ò
ób³¼ó³u´ ¶ô· klqzàÂ_sdKmn¯QfO¯Ähufzb|vºÀp=z'|ldKf¯µy}dKmnj¶ºegkl_bmjzb|r|lf¯¤e«
fLN3H?#JM83Eø;=<?>/HQ;=:õ±ÀW38öbUJ8
½õ i]klakvº|~z'heu_L&e«
f[¯Q_ba»zbkrm²_'yò
y}_bhe«
fZdKm¯Qklquj_sdKm?egfO¯Ñjzbqgfq«
_bdK|~¯Ñqxeuhgk~j¶eu|rw³Âf0«ä
©2â
³A»³¼	VÂKdQe°
f°
kr|l|mK_beÂf*_=psfhu|rwcf¯Kzbm?egk~j¼zbÂ_sdQe	e«
k~q	c_skrm?e	y}_bh	mK_°
½:Ãi]_nQkry&klme«
fZ_bhukrskrmz'|¸³o*ÕI°
f°
_sdK|~¯«
zXpbfhgfOjfkrpsf¯àhgf°
zbhu¯H©!
©2¯/y}_bhegh´z'mnqgkveukr_smKklmK³y}hu_ba÷©9eg_.©¯`_smÐzsj¶egkl_bm
e«
fmÚklmÚe«
fCmKf°
¸³o*Õó¯°
f°
_bdK|~¯ÁhgfOjfkrpsfhgf°
zbhu¯H©!
©
¯Ë²ò
©!
©
¯{_bm9e«
fqgzbaÃf¹fpbfm?eO½ÏK_shz'm]w±KÅQfO¯à¸³o*Õz'm¯àzsqgqgdKakrmnzs¯K¯QkregklpbfsVafa_bhuw?º|lfquqq«
z'cKklmK»huf°
z'h´¯9y}dKmnj¶eukr_smnqne«
klqX
¯øËrò
k~qe«
fa_sqe3sfmKfhuzb|sc_squqgkrÂK|lf`y}_bhua_'yq«
zbcKkrmnhuf°
z'h´¯Kq½
Í¸_shgfº_=pbfhe«
fw³j_=psfh*zZyMzbkrhu|rw|~z'hubfhuzbmKbf_byTc_squqklÂK|lfÃq«
z'cKºklmKàhuf°
z'h´¯Kq_bmKfakl«
ej_bafdKc°
kve«
½ÐÏn_bhfÅKz'acK|lfbeg_fmnj_sdKh´z'bf	a_=p]krmn¼eu_°
zbhu¯nqz¹b_szb| szq«
zbcKklmK'ºÀhgf°
zbhu¯y}dKmnjegkl_bmÃe«
z=e_bmnf{akl«
eTj«
_]_sqgf{klqò
®©!
<©¯/,ÀÊ°
«
fmQºfpsfh;©2¯k~q`j|r_?qfhÁ}klm°
«
z'egfpbfh`z'cKcKhu_bcnhgk~z=euf qfmnqff#eu_e«
fb_?z'|&e«
zbm.©sVzbmn¯ò
®©
<©¯nXø[_be«
fh°
k~qfs°
«
fhuf$ÊZklqqg_bafc_?qkregklpbf/hgf°
zbhu¯R½k¼hOQeg_Zfmnj_sdKh´z'bf*e´z']krmn[zbjegkl_bm
klmq_safTqfe_'ynqeuz'egfOqi¹â?O_smKfakl«
e&qgfeò
®©
<©¯nÃwÊ°
«
fmKfpsfh

<©6ª¢ùâbbzbmn¯ò
®©!
<©¯/bÚ*_'e«
fh°
k~qfs½k¼mKf»f|rfaÃfm?euz'huwCÂndQe/klaÃc_bhgeuzbm?ecKhg_scfhexwC_'y	e«
k~q¹y}_shga_'y*hgf°
zbhu¯Èegh´z'mnqy}_bhua»z=eukr_smÈklq»e«
z'ekre[jzbmÐsfmKfhuzb|r|lwµÂfËÜdrÖ%ÖÞnÌáÖ´Ø=ó¾áma»zbm?w hgfkrmKy}_bh´jfafmseÚ|lfzbhgmKklmK®z'cKºcK|lkljz=eukr_smnq°
fÁzbhgfÐmK_'e²fÅQcK|lk~jkreg|lwÔbklpbfmÄ zbq²zÓegdQºcK|lfA J6J=J&ÂKdKez'hufZz'|l|r_°
f¯Ceg_C|rfOz'hum²zbÂ_sdQe]_bmn|rwàe«
hu_bdn«
euzb]krmK³zsj¶eukr_smnqklm²e«
fI¸³o*ÕÔz'mn¯ÈÂ?wÑ_bÂKºqgfhup?klmK¤e«
fÄhufqgdK|regklmKÓqeuz=eufµegh´z'mqkregkl_bmnqàz'mn¯âhuf°
z'h´¯Kq½ò*klpbfmµqgdnj«
zbjjfquq*eg_L!°
f»jzbm²qgkla/dK|~z=euf«
zXp?klmKIe«
fquz'af[exw]cf_'y*zsjjfquqeu_«¯qgklaÃcn|rwµÂ]wµe´z']krmnÑzsj¶egkl_bmqú<û(;"G)ù-/!  1W#21?J+E+*=*,45  82()45>!Jù#2(7(;2iV<()<#% ÜE()%"()<8#%4 :BA#>!>!4:"47fEi-/%H9$üÃýPhþÿ?ÿþýnUÅP2ÿþfÿ^ÿþ+ýnU-/%b#%@!4"#B	XE)#%(W>*,G)<
«üXPhþÿ?ÿþ
ýUi45="G!Ã<*=#%>]*F*=2) 5>¢GW#+7f8<47f+>¢4(¢"G)624V245()#2
ÀgÃ(W>!JÁ"G!#21!1!21!45#%"82(W>!4:"4(!EW4:"b"!(),2!","GW#"+E)4:-*=#"V247f1"46#2 4:"®BLV)#%#%("<445 5#"0"G)2$*=]*,45  bV47fHG!<E"G)<($"G)ë2() :BH#2>)>4"42(W#2
-n>!I"G)4,V47f<Ã!b4(8G)245(!VGW#%1)4()V]<* #>!4ã4:"Á#2  +*,X!"8+#% 5<*=#%>ã@B[#2(B
3+>1?4:"47f;-#%8<"2+g=4(!8Ó"G)4b>!<b(!2"Ã#>!>$#%(B4("<C"Y4()VX48G)(! "XE!*F>!<-n<,"G)4=! ","X"G!;-/! 5
1W#%1?<+g
klmó!9z'm¯ùe«
fmöãgcKhufegfmn¯QklmKsä°
fÒ_bÂqfhgpsf¯øhgf°
zbhu¯H©!
©2¯/Ërò
®©
<©¯n°
«
fmKfpbfh°
fzbj¶eudnz'|l|lw9_sÂnqfhgpsf¯huf°
z'h´¯À$®©!
<©¯/Zklm!½®ý¼z=egdnhuzb|r|lwbe«
f²qklacK|lfàhufz'ºqg_bm²e«
z=e/e«
k~q°
_bhu]qklqe«
z'eH z'mn¯u¯`dnqgf»e«
f9quz'afzbjegkl_bmnq9qeuz'egfOqÈz'mn¯!euhuzbmnqgkveukr_sm!cnhg_sÂnz'ÂKkl|lkveukrfOq½ª	«
dnq_bmn|rklmKf=_»klmKf	aÃ_Q¯Qf|vºÀÂnzsqfO¯Xa_Q¯Qf|rºy}hgff{zb|rs_bhukve«
a»q3e«
z'ea»zXw¹Âfz'cncK|rklf¯eg_ a»zXwkrmbfmnfh´z'|?Âf{hgfOzb¯Qkl|lw¹zbcKcK|lkrfO¯eg_0
¯klmIe«
fquz'af°
zXwb½i]klmnjf°
fz'hufT|lfz'humKklmK¹z¼c_b|lk~jw¹y}_bhb¯?klmÃe«
f«
_scf	_'ydnqxºklmKkve`krm'!=e«
fí?dKfOqxeukr_smz'e«
z'm¯klq#e«
dnq#e«
fy}_b|l|r_°
klmKóÏK_sh°
«
z=e	y}_bhua»q{_'y3q«
zbcKkrmn'ºÀhgf°
zbhu¯y}dKmnjegkl_bmnqò
jzbm°
fbdz'h´z'm?egffe«
z=e=¿ÑÈ
Õ
'e«
f{_bcQeukra»zb|Qc_s|rk~jwklmH¯ °
kl|r|Kzb|lqg_Âf_bcKegklazb|nklm'ôª	«
fmKfÅ]e`qfOj¶eukr_sm°
kr|l|zbmnq°
fhe«
k~qeu_zyMz'klh ¯Qfshgff¹_'y&sfmKfhuzb|rkrexwb½
¨ìè
§^{¥]ëã|á¦K¥¾ám/cnhuzsj¶egk~jzb|?zbcKcK|lkljz=eukr_smnq°
f{_'yÍegfm¯Q_*mK_be3fÅQzsj¶eu|rw]mK_°Úz¹cKhukr_shgki z'mn¯a»zXw_bha»zXw/mn_'eT]mK_°
H©!
<©¯/½,k¼dnhb_?z'|3klqe«
fhufy}_bhufb·skrpsfmÓz'mn¯  z'mn¯Cc_?qgqgkrÂn|rwÔ¶·eu_j_safdKc°
kre«
zZq«
zbcKkrmn'ºÀhgf°
zbhu¯Zy}dKmnj¶eukr_smò
ó?w³Ôø³«´ ¶¸·Ðe«
z'e`k~qãgb_]_Q¯Käz'mn¯»qg_e«
z=eb¿ÑÈ
Õ°
kl|r|Âf_bcQeukra»z'|klm¢!½¾ám9e«
k~q qgfjegkl_bmV°
f°
kl|l|Vbklpbf¹zy}_bhua.y}_bhò
dnmn¯Qfh°
«
k~j«°
fjzbmsdnz'h´z'm?egffb¿öÑÈ
Õ°
kl|l|KÂf	_scQegkla»z'|Qklm!½3åÑfz'|~qg_µcKhg_=p]k~¯Qfz°
fOz'Áj_bm]psfh´qfq«
_°
krmK²e«
z=eO°
kve«
_bdKey}dKhge«
fh	?mn_°
|rfO¯Qbf¼_'yö¿z'mn¯[]e«
k~q	klq`e«
f*_smK|lwexw?cf*_'yq«
z'cKklmKÃy}dKmnjegkl_bmIe«
z=ejzbmIzb|°
zXwQq{bklpbf¼e«
k~qbdz'h´z'm?egffb½Ï3krh´qey}_Qjdnqgkrmn_bme«
f dKmn¯Qk~quj_bdnmseuf¯»jzbqgf6¢þ¶?|lfeTdnqeghuw¼eg_ ?z'klm¹qg_bafklm?egdKkregkl_bm/zbÂ_sdQe°
«
z'eò
akr«
e&bklpbfhuklqgfeg_e«
f¹q«
zbcKkrmnàãgÂKdKsä/c_skrm?euf¯9_bdQeklmZe«
f*¾ám?euhg_Q¯Qdnjegkl_bmV½k¼m zbmn¯Q|/j'pz'm¯ |reghj'a qÂKk~jwQj|rfTeuzbqg·°
«
fme«
f	z'sfm?e°
zbqhuf°
z'h´¯Qf¯y}_shThukl¯QklmKeu_°
zbhu¯nq#e«
fb_szb|KÂKdQe	mK_beTcKdKmQºk~q«
fO¯¹y}_bh&hgk~¯QklmK¼z°
zXw y}hu_bakre=kre3|lfzbhgmKfO¯*eg_¼hgk~¯Qfklm/zeukrm]wjklh´j|lf/z'mn¯Ie«
fhufÂ]w9_bÂQe´z'klmc_?qkregklpbfhuf°
z'h´¯°
«
fmKfpbfhkre«
zbcKcfmKf¯Ceg_ÂfaÃ_=p]klmK9eg_°
z'h´¯Kq e«
fb_?z'|½/¸_bhufÃbfmKºfh´z'|l|lwb]kvy&e«
fhgf¹k~q	q_saÃf¹qgfí?dKfmjf*_'y3qeuz'egfOq;©
<©Ì2%%2<© qgdnj«
e«
z'ee«
f¼z'bfm?eTjz'meuhuzXpsf|]e«
hg_sdK«
e«
fa!krmZzjwQj|lf©
¶ ©ÌÅ¶ §2§%§^¶ © '¶ ©
¶ §2§%§:¶·z'mn¯I?z'klmImnfe¼c_sqgkrºegklpbfÃq«
z'cKklmK'ºÀhuf°
z'h´¯[Â]w¯Q_bklmK[qg_Aò
®©

<©Ì2Ë
§%§2§Ëò
© "!

#!
© ^ËÀò
© ù
¹<©
$®!e«
fmµkreÃqgffa»qe«
z=e¹e«
f»zbbfm?e¹a»zXwÂf²ãg¯Kklqegh´zbjegf¯nä»y}hu_ba°
«
z'egfpsfh¹krehufz'|l|lwÁq«
_bdn|l¯ÚÂf³euhgw]klmKµeu_Ð¯Q_°Mqgdnj«
zbq[hgk~¯QfCeg_°
z'h´¯Kqe«
fb_szb| :{z'mn¯9klmnqxeufzs¯Zeuhgw[eg_hgfcfOz=euf¯Q|lwZb_hu_bdKm¯Ze«
klqjwQj|rfs½ª
_*zs¯K¯QhufquqVe«
klq#¯QkpmZjdK|rexw°
kre«
jwQj|lfq'zy}_shgaây}_shò
e«
z'eklaÃafO¯Qklz'egf|rw[j_bafq{eu_aklmn¯Iklq{eu_|rfeò
ÂfzØ=Ë %¼ÖÉgÖÞð´Öñxî/Ünñ=ÌáÖÞÌ ËMÇ Ù¶óò
©!
©2¯/['&X©¯ëä(&X®©°
«
fhuf)&®klqqg_baf[y}dnmnj¶eukr_smÄ_=pbfhÃqeuz'egfOq½ª	«
k~q°
zXwsò
©

©2ÌË§%§2§Ëøò
®©*#!

"!
<© ^Ëò
© ù
¹<©
¼ n¼zbmn¯°
f«
zXpsff|lkraklmnz=euf¯e«
f/cKhu_bÂn|rfa _byTjwQj|lfq e«
z'e9ãg¯Kklqegh´zbjeuäe«
fIzbbfm?eO½à hufZe«
fhuf[_be«
fh°
zXwQqeu_àj«
_]_sqgfò
ô m¯
zbqgk~¯QfÃy}hg_sa÷jwQj|rfOqzbhgfÃe«
fhuf»z'm]w³_'e«
fh/cKhg_sÂK|lfa»q°
kve«q«
z'cKklmKe«
z=e°
fZmnff¯Ñeu_Czb¯n¯QhgfOqgq´ô!¾ÀeeudKhgmq/_sdQee«
z=e°
kre«
_sdQeÃa_shgf9cKhgkl_bhÃ]mK_°
|lf¯Qsf[z'Â_bdQeH zbmn¯²Ãqgdnj«c_'egfm?egk~z'|rºÂnzsqfO¯Úq«
z'cKklmKµy}dKmj¶egkl_bmqò
z'hufe«
fC_smK|rwòe«
z=e°
kl|l|bbdz'h´z'm?egffj_bmqk~qxeufmnjw°
kre«
e«
fT_scQegkla»z'|sc_s|rk~jwklm'!½#¸³_bhuf_=pbfhXe«
k~qegdnhgmnqT_bdQeTeg_Âffquqgfm?egk~z'|l|rwÃz'|l|°
fmKff¯µklmÄ_shu¯Qfheg_àa»z'bfZe«
k~qÃbdnzbhuzbmseuffs½ª	«
klqk~q/a»zb¯Kfy}_bhua»z'|RklmIe«
f¹y}_s|r|l_°
krmn/e«
f_shgfaIó+
N3<Wn;=<WÐñ-,
ÖÌ#Ç=ÞnÎ
.d.^/.3Ç=Þ·Ø/Ç'ÞnÎ*ÙgÛKÇuÜ·ËÍÞ?ßÉgÖ×TÇ=ÉgØî´ïQÞðÌËMñ=Þò
óëø³uó³² ´¶ ·öÊ´ÖIß'Ë	=ÖÞ1032Ö³ÙÇ=ÎòË~ÙÇbU<]8·UJ HK\¬<3HKLO<]:
ÙgÛnÇuÜËÍÞ]ß¹î´ïQÞðÌ ËMñ'ÞàË î/ÌÍÛnÖÉuÖÃÖ54sË~Ù¶ÌMÙÇÄÉgÖ´Ç Ý	=Ç ïnÖ´ØIî´ïQÞðÌËMñ=Þ&úóã ´¶ ·õÙ¶ïnð´ÛÚÌ}ÛKÇ'Ì{îñ'ÉÑÇ©ªÔ¼ä
©â

ª¢6©2¯ªÔ6.ò
©!
©
¯bw7&X®©
¯=ä8&X©f+  !9×&ÛKÖÉgÖ²»ä
©â
  Ë î;: þ=<>0@?·ÛnÖÞA.²Ì}ÛKÇ'ÌòË~ÙÄÇÑÜnñ=ÌáÖÞÌ ËMÇ ÝÊ´ÇXÙÖ´ØÚÙgÛnÇuÜËÍÞ]ß²î´ïQÞðÌ ËMñ'ÞÔË~ÙÄÇÒÞ·Ö´ðuÖÙ´ÙÇ=É¶ÎÇ=Þ·ØÙ¶ïCBÁðËMÖÞnÌÑð´ñ'ÞØ=ËÍÌËMñ=Þ¿îñ=ÉÚËÍÌCÌáñÚßbïnÇ=ÉgÇ'ÞnÌáÖuÖ¿ð´ñ=ÞKÙ¶Ë~Ù¶ÝÌáÖÞ·ðÎ×ËÍÌ}Û®Ì}ÛKÖâñ´ÜÌËÇÜnñ ËMðÎ
9×&ÛnÖÞ rÖ´Ç=É¶ÞËÍÞ?ßÐî´Égñ¯º  J6J=JËò
ÓÉgÇ'ÌÍÛnÖÉÒÌ}ÛKÇ'Þî´Éuñ   J6J=J< .ËÍÞàÌÍÛnÖTîñrñ=×ËÍÞ]ß9ÙÖÞKÙÖ D
9FEïCB¤ðËMÖÞ·ðÎG<-HMîò
Ë~Ù¿Ç¤Ünñ=ÌáÖÞnÌËMÇ ÝÊ´ÇXÙÖ´ØÔÙÛnÇuÜ·ËÍÞ?ßî´ïQÞ·ðÌËMñ=ÞA.VÌÍÛnÖÞ9Ö%	=ÖÉ¶ÎñuÜ·Ì Ë»Ç=ÜKñ ËMðÎËÍÞ$¯R×ËÇ ÙñÊ´ÖÃÇ=ÞÑñ´ÜÌËÇQÜnñ ËMðÎZËÍÞ¢
9Ç'ÞØ'	ËMð´Ö	XÖÉ´ÙÇG<>0
9JIÖ´ð´ÖÙuÙ¶ËÍÌÎG<KH}îò
Ë~ÙÁÞñ=Ì³ÇÄÜnñ=ÌáÖÞÌ ËMÇ ÝÊ´ÇXÙÖ´Ø¿ÙgÛKÇuÜ·ÝËÍÞ]ß²î´ïQÞðÌ ËMñ'Þ
9Ö 0 ßL0 ÞñÒÙ´ïnð´ÛM&)Ö54sË~Ù´Ì ÙÈÙÇ=ÌË~Ù}î´Î=ËÍÞ?ßNO
ïnÇ=ÌËMñ=Þ
9P<G<.Ì}ÛKÖÞ[Ì}ÛKÖÉgÖ¼Ö54sË~Ù¶Ì
9ÜÉuñuÜnÖÉ<ÌÉgÇ=ÞnÙ¶ËÍÌ ËMñ'Þî´ïQÞ·ðÌËMñ=ÞnÙFÇ=Þ·ØÇÉgÖ×TÇ=ÉuØî´ïQÞðÌ ËMñ'Þó!³µ´ ¶ ·Q.Ù¶ïnð´ÛÌ}ÛKÇ'ÌRÞñ¹ñuÜ·Ì Ë»ÇOÜnñvËMðÎ*ËÍÞ6¯·Ë~Ù{ñuÜ·Ì Ë»ÇQËÍÞ6R0 |~qg_[mK_begfe«
fy}_b|l|r_°
klmKó ÏK_bh¼e«
fklmQ±nmKkregfºÀqeuz'egfjzsqfskry_bmnf°
fhgfeu_Cj«
_]_sqgfZqg_bafS&Ôeg_Cj_bmnqeghudnjezc_begfmseuklzb|vºÂnzsqfO¯¿q«
z'cnkrmKÁy}dKmj¶egkl_bm¹e«
fm¿y}_she«
f²y}_bhuazb|hgfOqdn|ve´qeg_Ðb_Äe«
hu_bdK«
°
fàhufzb|r|lw¤q«
_sdK|~¯Ó¯Qfazbmn¯¤e«
z=eT&ÂfÂ_bdKmn¯Kf¯Rq_Ôe«
z'eÄe«
f¿q«
z'cKklmKÔhuf°
z'h´¯Kqò
zbhgfÒzb|lqg_Â_bdKmn¯Kf¯L qklakr|~z'h`eg_e«
f¹j_bm¯Qkveukr_sme«
z'e;¿Âf¼Â_bdKm¯Qf¯Rklm[i]fjegkl_bm9K½rþf¶ûse«
k~qk~qgqgdKf°
kl|l|nÂf¼¯Qk~qgjdnquqfO¯»z'szbkrm|~z=eufhO½ý _begfIe«
z=ey}_bhe«
fI±nmKkregfºáqeuz=eufjzbqgfbe«
k~q»k~qÃzÑp=zbjdK_bdqj_smn¯Qkregkl_bmCqgklmnjf&°
_bdn|l¯R«
zXp]klmK[zZh´z'mnbf_'y±mKkveufjz'hgº¯Qklmnz'|lkrexwb#z'dQeu_ba»z=eukljz'|l|rwàÂfZÂ_bdnmn¯Qf¯V½I|lqg_n&e«
f[mKfjfqºqgkvexw»z'm¯[qgd?mZjklfmnjwj_bmn¯Kkveukr_smnq`zbÂ_=psf akr«
e	qgffa.z/|rkreºeg|lf»a_bhuf»j_bacK|lk~jz=euf¯Ce«
zbmàdqdnzb| 3z'mn¯Ce«
klqk~q¹Âfjzbdnqgfe«
fhuf¹jz'mÂfa/dK|regklcK|lf_bcQeukra»z'|c_s|rk~jklfq	klmL _bhklmL¯M½ý fpbfhe«
f|lfquq	kreIq«
_bdn|l¯¤Âfàj|rfOz'h»e«
z=e9e«
fCí?dnzbm?egkr±jz'ºegkl_bmqTdnqgf¯a»zbbfe«
klqTe«
f¼qeghu_bmKsfqeTc_?qgqgklÂK|rf e«
f_bhufa!_'ye«
klq y}_bhuaI½ª	«
fÃqd^m»jkrfmnjwIj_smn¯Qkregkl_bm³quzXw]qe«
z'e¼q_[|r_smKzbq°
f dqf zc_'eufm?egk~z'|rºÀÂnzbqgf¯ò
'e«
fm°
f zbhgfsdnz'h´z'm?egff¯z'm]wÔ¿öÑÈ
Õ°
fakl«
eÂfÃeghuw]krmnZeg_|lfzbhgm°
kl|l|z'|~q_9Âf_bcKºegkla»z'|{krmT!½ª	«
f[mnfjfOqgqgkrexwàj_smn¯Qkregkl_bmÄquzXwQq¹e«
z=ekry°
f
«
zXpsfmK_I?mn_°
|rfO¯Qbf_bybÔz'm¯AÃe«
fm°
fadnqxe¹j«
_]_sqgfzZc_'eufm?egk~z'|rºÀÂnzbqgf¯ò
y}_sh|lfz'humKklmKZklm¼¯ kry°
f°
z'm?e eu_bdz'h´z'm?egffj_bmnqgk~qxeufmnjw°
kre«
|lfz'humKklmKe«
fZ_bcQeukra»zb|c_s|vºk~jwIkrm!½]}¾Ày°
f/¯Q_«
zXpsfklmseukra»z'egf]mK_°
|lf¯Qsf_'y,<Ãe«
fme«
fmKfOjfquqgkvexwÃj_bm¯Qkveukr_sm»¯Q_]fqmn_'e`quzXwa/dj«
?zbmn¯krek~q`c_squqgkrÂK|lf¼e«
z'e°
f*aÃkl«
e	ÂfzbÂK|rf¼eg_Ãdnqgf¼_be«
fhq«
zbcKklmKy}dKmnjegkl_bmnq½:ª	«
fcnhg_]_'yQ_by]mnfjfOqgqgkrexwk~qVbklpbfm¹krmcKcfmn¯QkrÅ½fç fhufb°
f_bmn|rwIcKhu_=pbf*e«
z'eãÉTí?dnz'egkl_bm« !	k~qz»qgd?mZjklfm?e¼j_smn¯Qkregkl_bmVóe«
z=ekvyò
klqÃklmn¯Qff¯È_by e«
f9y}_shgaöklmNe«
fm°
fIazXwbdz'h´z'm?egff9e«
z=e9fpbfhgwÄ_bcQeukra»z'|c_s|rk~jwÁkrmµ¯°
kr|l|¼zb|lqg_Âf³_bcQeukra»zb| krmº!½ szbkrm°
fcKhu_=pbfIe«
k~qZhufqgdK|re»y}dK|l|rwhukrs_bhu_bdnqg|rw[_bmK|lw[y}_she«
fjzsqf_by#±nmKkregf¨ ¨vûne«
f/cnhg_]_'y#y}_shklmQ±nmKkregf¨ Á¨k~qmnfz'hu|lw¼k~¯Qfm?eukljz'|=ÂKdQe&hufí?dKklhufqz|rkreeu|rf{a_bhufjzbhgf¹klmVUxdqxeukvy}w]klmK»e«
f*dqf_bye«
f`f|r|la»z'mÔÉTí?dnz'egkl_bmnq½¹;d±Ho±LOWXWubJM<]8F¾öZYá
feò
Âf_bye«
f/y}_shga skrpsfmklm¶½3¾Ày
Ôâþb'e«
fmÃqgkrmjf{hgfcK|lzsjklmK[&X®©f°
kve«
&Ó¯®©f,&X©fä]¹y}_shz'm]wj_bmqxe´z'm?e °
_sdK|~¯mK_be#j«
zbmKbfTe«
fq«
z'cKºklmKàhuf°
z'h´¯Kqò
°
«
k~j«
klqzÑ¯QkvÆ·fhufmjfI_'y e«
fqgf9c_'eufmQºegk~z'|~q¶°
fa»zXwbQÂ]w[hufcK|~zbjkrmn&X®©f°
kve«
&X®©f ä8&X©2â	krymKfOjfququz'huwbzsqgqgdKaf°
kve«
_bdQe|r_?qgq¹_by{sfmKfhuzb|rkrexwe«
z'ee«
f&ÒdnqfO¯Zeu_»fÅQcKhufquqò
p]k~z	quz=euklq±nfq\&X®©%â,ÓK½ÏK_shÃe«
f_bhuklbklmnz'|	¸³o*Õ!°
fI]mK_°
e«
z=e»kve´qÃ_bcQeukra»z'|Ò*ºy}dKmnjegkl_bmÀÒÑÈ
quz=egk~q±nfqÃe«
f³{f|l|ra»z'mwÉTí?dnz'egkl_bmnq¢ qfffb½ n½Vü i]dQegeg_smzbmn¯	z'hgeg_n·þOb
OÿÜÒ
ÑÈ
©!
,µÉÃ¥
ÕÖ×
ØÙÚÜÛ Ý^]H®©
<©
¯Ë
a»z'Å¦Õ
ßà
Ò
ÑÈ
©
¯

¯_i]_saÃfqgkracK|lfz'|lbfÂKh´z'k~ja»z'mnkrcKdn|lz'egkl_bm9e«
fmIskrpsfq	dnqÒ
ÑÈ
©!
=ä8&X©fbºÉ
¥Õ]/H®©
<©
¯Ë
`&X©
¯=äa&X®©fË
¼az'Å¦+Õhßà
®Ò
ÑÈ
©
¯

¯ äa&X®©
¯_¾Ày°
fCmK_°
¯Kf±nmKfcb ÒÈ
Õ®©!
ad  ÒÑÈ
©!
Xäe&X®©f[zbmn¯qgdKÂnqegkregdQeuf[e«
z=e»z'mn¯ò
©!
<©
¯ 7&X®©
¯;äf&X©f/Ânzbj´klmseu_Ãe«
fcKhufp]kl_bdnq	fOí?dnz=eukr_smV°
f¹sfeb ÒÈ
Õ®©!
µÉ
¥Õ]H®©
<©
¯Ë²ò
©!
©
¯Ë
¼a»z'Å¦Õ
ßà
bÒÈ
Õ®©
¯

¯_µÉ
¥Õ]

¯©!
©
¯Ë
a»z=Å¦
Õßà
b ÒÈ
Õ©
¯

¯_{dQee«
k~qk~qfÅKzbj¶eu|rw9e«
f{f|l|ra»zbm³fí?dnz'egkl_bmy}_bhÅ¯M½*ÏK_she«
f`dKm¯Qklquj_sdKm?egfO¯/jzsqfs°
f`aÃ_shgf_=pbfh«
zXpsfgb ÒÈ
Õ®©%â!
,ÒÑÈ
©2â)
äh&X©2âf,ÓäàÒn½iQ_nib ÒÈ
Õ®©
`quz=euklq±nfOqe«
fC{f|l|ra»z'mÚfOí?dnz=eukr_smnqy}_bh¢
¯zbmn¯¤a/dnqe[klmÚyMzbje[Âfe«
fàdKmnklí?dKfÑ_bcQeukra»zb|ÅÒ*ºy}dKmnjegkl_bmV½ª	«
dnqXÒÑÈ
Õ
®©!
A
b ÒÈ
Õ®©!
»ÒÑÈ
®©!
,äj&X®©f¶Vz'm¯e«
f_scQegkla»z'|3c_s|rk~jwy}_bhÁ¯Re«
fhgfy}_bhuf*quz=euklq±nfq¿
ÑÈ
Õ
®©Âª z'hua»z=Å¦ßà
Ò
ÑÈ
Õ
®©
 z'hua»z=Å¦ßà
Ò
ÑÈ
©!
=ä8&X©f z'hua»z=Å¦ßà
Ò
ÑÈ
©!
z'm¯Ðk~q»e«
fhufy}_shgfzb|lqg_µ_bcQeukra»zb| krmµ!½ª
_Èq«
_°
fpbfhgw_bcKegklazb|Rc_s|rk~jw»klmÔ klqzb|lqg_Ã_bcQeukra»z'|·klm¢¯ KqgkracK|lw[z'cKºcK|lwZe«
f/quz'af¹cKhg_]_by°
kre«
e«
fhg_s|rfOq	_'y, zbmn¯Ô¯Rkrm?eufhgºj«
zbmKbfO¯T z'mn¯ÈdnqklmKe«
f9q«
zbcKkrmny}dKmnjegkl_bmºäò
½ª	«
klqj_sacK|rfegfOq{e«
fcKhu_]_'yx½ kl]n;\M\ HQ;f¾~nm
Þ·ØbÖÉTÌ}ÛKÖ ð´ñ=Þ·Ø=ËÍÌËMñ=ÞnÙ	ñxî?·ÛKÖ´ñ=ÉuÖpo.·Ù¶ïOÜÝÜnñXÙÖÑÌÍÛnÇ=Ìò
ØsñÖÙCËÍÞ·ØbÖ´Ö´ØÁÌÀÇ qbÖÑÌ}ÛKÖÃîñ=É+ò
©!
<©¯/A`&X©¯när&X®©f=0
EïOÜbÜnñXÙÖVî´ïQÉ¶Ì}ÛKÖÉ`Ì}ÛKÇ'Ìs&X©â
bÚË îöÔþt0?·ÛKÖÞîñ'ÉÇ¹©ªÔ
.
ª¢.Ò
ÑÈ
Õ
®©
 Ò
ÑÈ
©!
 äa&X®© hs)Æ
ÑÈ
Õ
®©fÆ
ÑÈ
©f=ä8&X©f+ }W¹;d±uY
s!°
zbq	cKhu_=pbfO¯ZklmIe«
f/qgd?mZjklfmjw[cnhg_]_'y#z'Â_=pbfsû}Wy}_s|r|l_°
qklaÃafO¯Qklz'egf|rw¿y}hg_sa e«
fÈk~¯QfmseukvexwÆ$Ñ
®©wa»z=Å
¦ßàÒÑ
®©!
¶½ kY[<WÐÈHK;NrxY[
#W3LUO8&<]LOLÃHn8&:Ò\ <]HQ;=8&J 8&EùY
|ve«
_bdK«°
f«
zXpsfmK_'e3cKhu_=pbf¯*kre«
fhufbOe«
f`kl¯Kfm?egkregklfq3klmÃ^`_bhu_b|l|lzbhgw*zbjegdnzb|r|lw«
_s|l¯/y}_bhz'huÂKkregh´z'huw¹c_s|rk~jklfqF¿bmn_'e1Uxdnqe3e«
f	_bcQeukvºa»z'|Vc_b|lk~jw·óÆ6ÇÈ
Õ
©fbÆ6ÇÈ
©f=ä8&X©f z'mn¯qgkrakl|lzbhg|lw»y}_shÒ*ºy}dKmnjegkl_bmnq¶½{j_bmqfOísdnfmnjf*_by&e«
k~q k~q	e«
z'e c_begfmseuklzb|vºÂnzsqfO¯[q«
z'cKklmKÃklq*ÉgñsÊï]Ù¶Ì·klm[e«
fqfmnqf¼e«
z=emKfzbhºÀ_bcKegklazb|c_b|lkljkrfOq*zbhgfÃz'|~q_IcKhufqgfhupbfO¯Rûe«
z'e¹k~qRkvy°
fÃ|rfOz'humCz9mKfzbh_bcKegklazb|]c_s|rk~jw6¿9krmH¯ùMquzXwb
¨Æ6ÇÈ
Õ
©fäÆ$ÑÈ
Õ
©f%¨#:gvdnqxºklmKZc_'egfm?egk~z'|rºÂnzsqfO¯q«
zbcKklmKnne«
fmL¿°
kr|l|&zb|lqg_ZÂfmKfOz'hgº_bcKegklazb|	krmÀ J¨Æ6ÇÈ
©fëäÆÑÈ
®©f%¨6:wv¶½µª
_µqgff9e«
k~qz'cncK|rwÄe«
fk~¯Qfmseukvexw°
fxUxdqxeZc_bklm?egf¯Á_sdQe»eg_µc_b|lkljkrfOq¿z'm¯Zeu_¿ÑÈ
º¿ÑÈ
Õ
nz'mn¯IqgdKÂQegh´zbje½:Y[<WÐÈHK;~
7[\ \b\MJhsJM<]L«o&UOJhÐÈHK\»W383:&<?;
&Yª
_Âfeeufh{dKmn¯Qfhuqeuzbmn¯°
«
wc_'eufm?egk~z'|rºÀÂnzbqgf¯ò
cKhufqgfhupbf_bcKºegkla»z'|c_b|lk~jklfqkreklq°
_bhge«
mK_begklmKkvy°
f«
zXpbfz'mµ¸³o*Õ e«
z'e«
zbq/zc_begfm?euklzb|vºÀÂnzbqgf¯ÁÉgÖËÍÞîñ'Égð´Ö»ÖÞnÌ#î´ïQÞðÌ ËMñ'ÞH©!
©
¯,À7&X®©
¯öäy&X©f]e«
fm²Ç'ÞnÎc_b|lk~jwZk~q{_bcQeukra»z'|klm!½ª	«
dnq]c_'eufm?egk~z'|rºÂzbqgf¯Zq«
z'cnkrmKy}dKmnjegkl_bmnq	zbhgf¼krmQº¯QkrÆfhgfm?eeg_[c_s|rk~jklfqklme«
fqgfmnqgfe«
z=e¼e«
fw9skrpsfdnq mn_hufzbqg_bmZeg_ÃcKhgfy}fh	z'm]wc_s|rk~jw»_=psfh	z'm]w_'e«
fhOûQz=ez'm[klm?egdQºkregklpbf|rfpbf| e«
k~qzbjj_sdKm?euq¼y}_bh°
«
we«
fwÑ¯Q_mK_'eskrpsfdnqz'm]w³hufzsq_sm³eu_Icnhgfy}fhz'm]w³c_b|lk~jw³_be«
fhe«
zbm¿ÑÈ
°
«
fm°
f¹q°
kreuj«
y}hg_sa  eu_'¯M½ª	«
fª	«
f_shgfaqdKsbfOqxe´qVe«
z=e°
f	j«
_]_sqgf`q«
zbcKkrmnhuf°
z'h´¯Kq_'yQe«
fy}_shgaò
®©
<©¯nÃT`&X©¯när&X©f½3¾ámz'cKcK|lk~jz=eukr_smnq
0 50 100 150 200 250 300 350 400 450 500010020030040050060010x10 Grid world
Trial numberSteps taken to reach goal
0 50 100 150 200 250 300 350 400 450 50000.511.522.5x 104 50x50 Grid world
Trial numberSteps taken to reach goalMz) }ÂoÏ3krsdKhufþbó;Mz!,ÉÅQcfhgklafm?e°
kre«
þO=ÅRþ*bhuk~¯]º°
_bhu|~¯R½&ÕT|r_be_'yVqxeufcnqeuzbbfmeu_b_?z'|QpQq½&euhgk~z'|nm?dna/ÂfhO½#o¼_'e`klqTmK_/q«
z'cKklmK¯Q_beºá¯Kzbq«
k~q^&TÒ
 tt&Ãâ'nqg_b|lkl¯9klqz&r-&Ãâb½}ÂdëÉÅQcfhukrafm?e°
kre«
t''Å^tb/shgk~¯]º°
_shg|~¯R½&q«
_bdK|~¯³_byTj_sdKhuqgfÂfÃj«
_sqgfmdnqgkrmn[fÅQcfhge¼]mK_°
|lf¯Qsfz'Â_bdKe`e«
f¯Q_ba»z'klmV½#q{eu_«
_°
_smKf*azXw»¯Q_e«
klq^`_bhu_b|rº|~z'huwÄCqgdKbsfqeuqÃzÑcnz'hgegk~jdK|~z'hu|lwÑmnkljf9y}_shgaõy}_sh{&¼Tkry°
f]mK_°
fmK_bdK«
zbÂ_sdQe¹e«
fZ¯Q_sa»z'klm³eg_IeuhgwCj«
_?_?qklmK9kve/zsqqgdnj«
½Iåàf[qgffe«
z=ekvy|&X©fÆ]ÑÈ
®©f°
kre«
&X®©â
¬klmCe«
f»dKmn¯Qk~quj_bdnmseuf¯Ñjzsqff¶Re«
fm.ÉTí?dnz'egkl_bmw})euf|l|lq¹dnqe«
z=eÃe«
f9pXzb|rdnfZy}dKmnjegkl_bmÐkrmw
¯k~qÆ
ÑÈ
Õ
®©i}Ôq2zbmn¯e«
klqTklq`zcnz'hgegk~jdn|lzbhg|lw/fOzbqgw/p=z'|ldKfy}dKmj¶egkl_bmeu_|lfz'humVû?fpsfm|~zbj´]krmnàz³a_Q¯Qf|{_by e«
f°
_bhu|l¯Vz'|l|`e«
z=e°
_bdK|~¯Èhufa»z'klmeg_CÂf[¯K_bmKf°
_bdn|l¯ÑÂfZeg_|lfzbhgm²e«
fZmK_smQºCfhu_LÒ*ºÀpXzb|rdnfq½ª	«
_sdK«
eg_ÚzXpb_skl¯Óaklquj_smnjfcQegkl_bmV°
f²zb|lqg_¤qxeuhgfOqgqIe«
klqk~q[mK_be9e«
fÑ_bmK|lw°
zXwÁ_'y/j«
_]_?qklmKÈdnqgfy}dn|r&¼*z'mn¯Úe«
z'eqgdnj«
q«
z'cKklmKÚhuf°
z'h´¯Kqjzbm«
f|lcùqgkrsmKkv±jz'm?eu|rw.Ö	=ÖÞøË î&Ë~ÙîÇ'É¼î´ÉgñÆ]ÑÈ
MquzXwCklmµe«
f9qgdKcQºÀmK_bhua0¶#qdj«
zbqÂ]wbdnkl¯QklmKZfÅQcK|r_shuz'egkl_bmVfe´j'½lzbmn¯°
f°
kr|l|3qgff/fÅKz'acK|lfq_'ye«
klq/krm²e«
fZmKfÅ?e/qgfjegkl_bmV½{dQeklmÈz'm]wàjzbqgfb&qg_|l_bmK³zsq°
f9j«
_]_sqgfZc_'eufm?egk~z'|rºÂzbqgf¯ò
°
f«
zXpbf»e«
f9sdnz'h´z'm?euffe«
z=eTz'm]w0MmKfzbhºá_scQegkla»z'|]c_s|rk~jw°
f{|lfz'humÃkrm¯°
kl|r|Kzb|lqg_ÂfL}mKfOz'hgºx_bcQeukra»z'|klm²!½á
fe/dnqmn_°
egdnhgm²_bdnhz=eeufmQºegkl_bm[eg_q_saf¼qgazb|r|fÅQcfhukrafmse´q`e«
z'e	¯Qfa_bmnqegh´z=euf«
_°c_'egfm?egk~z'|rºÂnzsqfO¯9q«
zbcKkrmnakl«
e ÂfzbcKcK|lkrfO¯IkrmIcKh´zbj¶eukljfb½~
{V§ì}{è
¦K¥¸dnj«
facKklhgk~jzb|°
_shg Âfy}_bhuf dnq«
zbq j_sm]p?klmnjºklmKb|lwUxdnqxeukv±f¯öe«
f dqf_'yÓq«
z'cnkrmKü ¸³z'euzbhgk~j'RþOb'Q zbmn¯Q|/j'p[zbmn¯9|lqeghj'aVþOb!
Oÿ{z'mn¯°
f°
kl|r|mK_beÃÂ_'e«
fheg_ÈeghuwÄeu_Èy}dKhge«
fhrUxdnqegkry}w¤kve´q[dqfs½Iç fhufb_sdKhZs_sz'|krmQºqegfzs¯Zk~q`eg_q«
_°
«
_°
c_begfm?euklzb|vºáqxexw]|lf¼q«
z'cnkrmKy}dKmnjegkl_bmnq±Ke#krm?eu_ e«
fTcKk~j¶eudKhgfs'z'mn¯eg_¼¯Kfa_bmnqegh´z=euf«
_°
qgdnj«
q«
z'cKºklmKÃy}dKmnj¶eukr_smnqakl«
e Âf¯Kfhukrpsf¯9krmIcKh´zbj¶eukljfb½
ª
_°
zbhu¯Kqe«
fqgfTs_szb|lq°
fTj«
_?qfy}_shqgkracK|lkljkvexwzbmn¯j|~z'hukvexweg_/dnqfpsfhuw/qgkracK|lf bhuk~¯]º°
_bhu|~¯¯Q_sa»z'klmnq#eg_q«
_°
jzsqf{e«
fklmseufhufqegklmK¤zbqgcfOj¶euq9_'yÃc_begfmseuklzb|vºÀÂnzsqfO¯Òq«
z'cKklmKn½ª	«
f±nh´qxe[¯K_ba»z'klm°
zbq»z²q«
_bhgegfOqxegºcz=e«
º eu_'ºÀb_szb|þO=ÅRþàbhukl¯Qº°
_bhu|l¯°
kre«
qeuz'hgezbmn¯²b_?z'|qeuz=eufq/krmµ_scKc_?qkregf9j_bhumKfhuqmK_¯Qk~qgj_bdKm?eukrmKVz'mn¯Ñz9ºuþÃcfhgºÀqegfcàhufklmQy}_bh´jfafm?e½Ãj¶ºegkl_bmq`zbhgfe«
fÃj_saÃczbquqT¯QklhgfOj¶eukr_smnq]z'm¯»a_=pbfþ*qxeufcZklme«
fkrm?eufmn¯QfO¯¯QklhgfOj¶egkl_bmÔ
st!_by3e«
feukraf/zbmn¯zhuzbmn¯Q_ba¯QklhgfOj¶eukr_sm»bt_'ye«
feukrafs?zbmn¯»z'sfm?eqxe´zXw]qkrme«
fquz'afcK|~zbjf»kvykreeghukrfOqeu_°
zb|rà_'ÆÄe«
fZshgk~¯R½[å«
z=e/akl«
e/Âfzb_]_Q¯²q«
z'cKklmKc_'egfm?egk~z'|
&X®©gôøåàf«
zb¯Ñc_skrm?euf¯²_bdKefzbhg|lklfh&e«
z'e=É`ísdz=egkl_bm}WqgdKbsfqeuq&X©fbÆ]ÑÈ
©f&akr«
eÂfzb_]_Q¯q«
z'cKklmKc_'egfm?egk~z'|½iQ_|rfe`dnqTmK_°
s_¹e«
hu_bdn«e«
f¹exw?cf_'y3hgfOzbqg_bmKklmKe«
z'eaÃkl«
eqgdKbsfqe	zjhgdn¯Kf*fOqxeukvºa»z=euf`_byÆ
ÑÈ
ûbÂ?w¯K_bklmK*qg_n°
f«
_bcfTeu_*¯Kfa_bmnqegh´z=euf«
_°
°
kre«
z*|rkreeu|rffÅQcfhe]mK_°
|lf¯Kbf	z'Â_bdQeT¯Qk~qeuz'mjfqTz'mn¯e«
f|l_]jz=eukr_sm»_'yRe«
fb_?z'|sqgkrakl|lzbhThufzsq_smKkrmnazXwÃcfh«
z'cnq`Âfdnqgf¯[eg_Zqgkrakl|lzbhg|lwZ¯QfhgklpbfV&Áy}_sh_'e«
fhaklmKkradKaÃºÀj_sqeºeg_bºb_?z'|·cKhu_bÂK|lfa»q½t
c_bmeuhgw]klmK»eu_euz'sf/z»qegfceu_°
zbhu¯nq{e«
fb_szb| °
f«
zXpsfz'mÔ
st®j«
z'mnjf_'y3euz']klmKe«
f/¯QfqgklhgfO¯Iqegfceg_°
z'h´¯Kq{e«
fb_?z'|z'mn¯ÚzÄbt j«
z'mjf_'y/zÈhuzbmn¯Q_ba zbjegkl_bmV½¾Ày°
feuzbbfÃz[h´z'mn¯K_ba zbjegkl_bmVe«
fmCdnmK|rfOqgq°
fÃzbhgfÃz=e*e«
fÃÂ_bhgº¯Qfh_'ye«
f¹bhukl¯°
_shg|~¯R°
fz'huf¹zbq	|lkrsf|lweg_a_=pbf¼eg_°
z'h´¯Kqzbq/z°
zXwy}hu_ba e«
f[b_szb| ½ç fmnjfs&y}hu_ba÷a_sqeqxe´z=eufq°
f°
_bdK|~¯¤fÅQcfj¶e[e«
f³_scQegkla»z'|¼c_s|rk~jwÐeg_Èa»zbbf³z'Â_bdKe9K½
qegfcq9_by'M¸³zbm«
z'eeuzbm¿¯Qk~qeuz'mjf[cKhu_bshgfOqgqeu_°
zbhu¯nq»e«
fb_?z'|¹cfhegklafqegfcVz'mn¯zÐjhgd¯Qf²fqegklaz'egf²_'ye«
fµfÅ]ºcfj¶euf¯[m?dna/ÂfhT_byqxeufcnq`mKff¯QfO¯»eg_bfeeg_/e«
f*b_?z'|Qy}hu_ba©°
_bdn|l¯ÐÂfT^`7#1t` ®©/1`)u=

n½ª	«
dnq°
fqfe
S32
1
4G
0 100 200 300 400 500 600 700 8000501001502002503003504004505x5 Grid world with 5 flags/subgoals
Trial numberSteps taken to reach goalMz) }ÂoÏ3krsdKhufQó6 z!Át=Å
tbhukl¯Qº°
_bhu|l¯°
kre«
t[qdnÂKb_?z'|~q6}klmnj|ldn¯QklmK9b_szb|&qeuz'egf°
«
klj«
a/dnqe¼Âf/p]k~qkregfO¯krmC_bh´¯Qfhþu^Js

5½}ÂdÉÅ]cfhuklaÃfm?e°
kre«
t=Å
tbhuk~¯]º°
_bhu|~¯°
kre«
qdnÂKb_?z'|~q½ÑÕ|l_'e_byqxeufcnq/euzbbfmµeg_Ñb_?z'|TpQq½Ceuhgk~z'|{m]dKa/ÂfhO½²o_beÃk~q/mn_q«
z'cKklmKK¯Q_'egºÀ¯Kzsq«
k~q^&re&Ãâbnqg_b|lk~¯[k~q^&re&
½_bdnhfqegkla»z=euf/_'ye«
f/p=zb|rdKf/y}dKmnjegkl_bmàzbmn¯e«
fhufy}_shgf&X®©feg_ÑÂf)&Ãâ)©fbÆÈ
©f0¸ä^^`7#1t` ©!X`!5=

n½ª	«
k~q¹klq°
«
z'e°
fdnqgf¯àzbq¹_bdnh*sdKfquq*_'y`zÁãgb_]_Q¯Kä[q«
z'cKºklmK²y}dKmnjegkl_bmV½ |~q_	zsqZzµq«
z'cKklmKbºhuf°
z'h´¯²e«
z=e°
_bdK|~¯ÂfIí?dKkregf[yMz'h¢}klmµe«
fqgdKcQºÀmK_bhua0*y}hg_saÆ
ÑÈ
Õ
©f¶°
f9zb|lqg_eghuklf¯àdnqklmKT&X©f!^:tL&â
©f¶½ª	«
f»hufqgdK|reuq*_'y{e«
klq*±nh´qefÅQcfhukrafmse5¼zbhgfq«
_°
mZklm[Ï3klbdKhufþzn½ëM |l|fÅQcfhgklafm?euqhufc_bhgegf¯Iklme«
k~qqgfjegkl_bm³zbhgfzXpsfh´z'bfOq{_=pbfh?krm¯QfcfmQº¯Qfmse9hgdKmq½: ¼q[jz'mÚÂf³hgfOzb¯Qkl|lwÐqgffmdnqklmKµfkve«
fh9_'ye«
fqgfq«
zbcKkrmn[y}dKmj¶egkl_bmqqklbmnkv±jz'm?eg|lw«
f|rcf¯²qgcff¯àdKc|lfz'humKklmKn½¸_shgf_=pbfhkve¹klq°
_bhge«
hufºáqeghufquqklmKe«
z=efpsfme«
_bdK«
^:tL&Ãâk~qTí?dKkregf yMzbhy}hu_baÆ]ÑÈ
?kve	qxeukr|l|qgkrsmKkr±jzbmseu|rw«
f|rcf¯/e«
f	klmKkregk~z'|Qqxe´z'sfq&_'yn|lfzbhgmKklmK½&ÏK_shz|~z'hubfhFt''Å
t'bhuk~¯]º°
_bhu|~¯R{e«
fÑhgfOqdn|ve´q[Âfj_bafCfpbfmÓaÃ_shgfÑ¯Qh´z'a»z=eukljbóÏ3krsdKhuf/þÂIq«
_°
qe«
f¼hufqgdK|re{_byVe«
f¹quz'af¼fÅQcfhgklafm?e	hgfºcfz=euf¯9_bm[e«
f|~z'hubfh`shgk~¯R½ª	«
f¹cK|r_beuq{y}_sh\&Ãâzbmn¯9
 tt&Ãâz'huf/qg_Z|l_°
klme«
fshuzbc«
e«
z'e¼e«
fwjz'mÂnzbhgf|rwIÂfÃqffmVû|lfz'humKklmK°
kve«
_bdKeTq«
zbcKklmK¹klqj|rfOz'hu|rw|l_sqgklmK«
_bcf|lfquq|lweu_e«
fc_begfm?euklzb|vºÀÂnzbqgf¯Iq«
z'cKklmKz'|lb_shgkre«
a½fkveufh´z=egklmKne«
fs_szb|_'ye«
fqgf/fÅQcfhgklafm?euq°
zsq mK_beeu_eghuwZeu_Uxdnqegkry}wIq«
zbcKkrmn$q4e«
z=e«
zsqÂffm³¯K_bmKfyMzbh a_bhufj_sm]p?klmnjkrmKs|rw/Â?w/_'e«
fh´q½&¾ámnqxeufzs¯R°
«
z'e°
f«
zXpsf{¯Kfa_bmQºqegh´z=egfO¯«
fhuf»klqzqxexw]|lfZ_'yqg_bafZpbfhgwàqklacK|rf[hgfOzbqg_bmKklmKe«
z=eÂ]w9cKdKeegklmK»eu_bsfe«
fhz»¯Qk~qeuz'mjfºeg_bºs_szb|«
fdKhuklqegk~j'«
zsq fmz'ÂK|lf¯dnqeg_[cKklj´zZqgfmqklÂK|lf&Óe«
z'e¼¯Qh´z'a»z=eukljz'|l|rwÃ4()V6#%#HZ ""(H#%(W>]c=#%"EdQ+_2_ eÜEtgQYV%+>BH<3Y1) %#%"4(
E 5#%()4(!V0#"V> fMg31?<4("ë*,4:"GA!##!P?U#2 [Vf#+7fH#%(W#% 52V2)Å! "G!+*,45(!V[G)#21!45(!V4V(!4
8+#%(" :B1?+>4()VÅ!1$ +#%(!4()V!g
qgcfO¯9dncI|lfzbhgmnkrmK½ý fÅ]e z'mn_'e«
fh9j|lzsqgqZ_'ycKhu_bÂK|lfa»qy}_sh°
«
k~j«
zÈqgkrakl|lzbhqexw?|lf	_'yhgfOzbqg_bmKklmK*aÃkl«
e°
_bhuklq¯Q_sa»z'klmnq°
«
fhuf°
f	jzbmzbquqgkrsmZqdKÂnb_szb|lq½^`_bmnqgkl¯Kfhe«
f*bhukl¯]º°
_bhu|l¯Ãklm[Ï3krsdKhgf¹'zn°
«
fhgf°
f qxe´z'hgekrme«
f |l_°
fhºÀ|rfyÍe«
z'mn¯j_bhumKfhObz'm¯adnqecKk~j´dncZzqfeT_'yãz'?qgä*klmZqfOísdnfmnjfÂfy}_bhuf b_skrmK¹eu_e«
f±nmnzb|s_sz'|&qxe´z=egfs½ jegkl_bmnq¼z'mn¯huf°
z'h´¯Kqzbhgf¹e«
fquz'af/zsqklmCe«
fcKhgfp]kr_sdnq*bhukl¯]º°
_bhu|l¯RRzbmn¯³e«
f»qxe´z=eufºáqcnzsjfÃklq¹fÅ]ºcnzbmn¯Qf¯»eg_/sffc»egh´zbj´Ã_'yRe«
f*j_b|l|rfOj¶euf¯zbsq½iQkrmnjffOzbj«zbklq`zqgdKÂKb_?z'|'kre`k~qeufacQegklmKeu_j«
_]_?qfò
qg_e«
z'e°
fz'hufhuf°
z'h´¯QfO¯y}_bh	p]klqgkregklmK/e«
fqdnÂKb_?z'|~q½á
fe{dnq	mn_°
qff«
_°
z/c_begfm?euklzb|vºy}dKmnjegkl_bm9qexw]|rf¼_'yhufzsq_smKklmK/jzbmZklmn¯QffO¯|lfzb¯Ädnqeg_Ñj«
_]_sqgf9qgdnj«
zbmò
Tzbmn¯«
_°
É`í?dnz=eukr_smµ}Wy}dKhge«
fhqgdKbsfqeuq{a»z'bmnkveudn¯Qfq{y}_sh	e«
fqgdKÂKs_szb|huf°
z'h´¯Kq½åÚkre«
]mK_°
|lf¯Qsf#_'y]e«
f{qdnÂKb_?z'|=|l_]jz=eukr_smnqVz'm¯¹dnqklmKhufz'ºqg_bmKklmK¹z'mnzb|r_sb_bdqVeu_¼e«
z=eTqgdKsbfqegfO¯fOz'hu|rklfh;Mn½
¹qegfcq#_'ycKhu_bshgfOqgqcfh¼egklafqegfcV·feujb½ °
fa»zXw9fOqxeukra»z=eufe«
ffÅ]ºcfj¶euf¯Úm]dKaÂfh[_byegklaÃfOqxeufcnqquzXwg¶mKff¯QfO¯¤eg_ÄhgfOzbj«e«
f³b_?z'|½¿¾Ày°
f³kra»zbbklmKfe«
z'e9fOzbj«
qdnÂKb_?z'|k~q[z'Â_bdKefí?dnzb|r|lw«
zbhu¯/eg_hufzsj«
y}hu_baÔe«
f cnhgfp?kl_bdq&_smKfbbe«
fm«
zXp?ºklmKhufzsj«
f¯¼e«
f
·ºe«
qgdKÂKs_sz'|°
f°
_sdK|l¯qegkl|l|«
zXpsfTz'Â_bdKe®tä i5t!Tqegfcnq`eg_Ãb_n½#âq|lkl«
eg|lwaÃ_shgf*huf±nmKfO¯[zbhgsdQºafm?e/j«
zbmKbfOq*e«
k~q¹eg_uJt]ä8äÐ^:t5t!¼qegfcnq°
«
fhufK½:t	j_safqRy}hu_baÓe«
fãxexw]cKk~jzb|bjzsqfOä°
«
fhuf°
f`z'huf«
z'|ry°
zXwÂfe°
ffmÑe«
f·ºe«
z'm¯¡Ë
þºÀqe/qdnÂKb_?z'|~q<&z'm¯Ñqg__bdnh±nh´qxej«
_bk~jf9_'yz&X®©/k~qx&Ãâ)©flä6®t$äj
¥ä¤^:tut¢¶°
«
fhgf8
¥¯QfmK_begfOqe«
fÄm]dKa/ÂfhÑ_'y[qgdKÂKs_szb|lq°
f«
zXpsfzbj«
krfpbf¯°
«
fm°
f{zbhgf	z=eb©s½t
qklmK*e«
klq3y}_shga_byq«
z'cKklmKbºhuf°
z'h´¯y}dKmnjegkl_bmV°
fZqgffÃe«
z=e&X®©£&Ãâ)©fQUxdKacnqÂ]w
ut°
«
fmKfpbfh°
fhufzbj«
zbm?wÁqdnÂKb_?z'|6}_be«
fh»e«
z'm¤e«
f±nmnzb|b_?z'|qeuz'egf3zbmn¯µqg_9e«
f[q«
z'cnkrmKhuf°
z'h´¯Cy}dKmnj¶eukr_smò
©!
©¯n,e&X©¯/%ä&X©fklqbklp?klmK\ut	huf°
z'h´¯y}_shhufzsj«
ºklmK¼fOzbj«
_'yKe«
fqgf{qgdKÂKs_szb|lq½ª	«
k~q#klq3fÅQzsj¶eu|rw°
«
z'e#_sdKh3krmQºegdnkveukr_sm«
zs¯/qgdKbsfqegf¯¹akl«
e#Âf	z s_?_Q¯q«
zbcKkrmnhuf°
z'h´¯R½ÏK_shÃj_bacnzbhgk~q_smV°
fz'|~qg_àjzbhghuklf¯²_bdKee«
klqÃfÅ]cfhuklaÃfm?ednqgkrmn¤zÐa_bhufÑ±nmKfºegdnmKf¯q«
z'cKklmK¤hgf°
zbhu¯Úe«
z'eqklakvº|~z'h¹eg_Ie«
f»cKhufp]kr_sdnq¹bhuk~¯]º°
_bhu|~¯fÅQcfhukrafm?e´q&fÅQcK|lkljkveu|rwfqegkla»z=euf¯Óe«
f²hufa»z'klmKklmKÄeukrafº eu_'ºÀb_szb|y}_shfzbj«
qxe´z=eufz'm¯»j_bmqxeuhgdnjegfO¯/e«
fj_bhuhgfOqc_bmn¯KkrmKV&
©f,cbÆÈ
©f#c_'ºegfm?egk~z'|y}dKmj¶egkl_bm½ª	«
f»hufqgdK|ve_by`e«
fqgf»fÅQcfhukrafmse´qz'hufq«
_°
mÄklmÁÏ3klbdKhuf=ÂV{z'mn¯°
fqgffIe«
z'e»dnqklmKÑ_bdKhÃ±nh´qejhudn¯Qfq«
z'cnkrmKày}dKmnjegkl_bm(&Ãâ«
zsq»z'|l|r_°
f¯µdqeg_µqgkrsmKkv±nºjzbm?eg|lwqgcff¯³dKc³|lfz'humKklmKZ_=psfhmn_'e¼dqklmK9q«
zbcKkrmnAMzbmn¯e«
f ±nmKfº eudKmKfO¯&
dKmqdKhucKhuklqgklmKb|lw/?zXpbffpsfm»Âfegegfh`cfhºy}_bhua»z'mnjf½å«
fmhufcfz=eukrmne«
k~qfÅQcfhukrafmse_smI|~z'hubfh¯Q_sazbkrmq_bh°
kre«
a_bhufqgdKÂKs_szb|lqe«
fhgfOqdn|ve´qÔ}mK_beZhgfºc_bhgegf¯«
fhgff{ÂfOj_baf¹fpsfmIa_bhuf¹¯Khuzbaz'egk~j'½¤ ¥
ì¥]©Rë¥]¥]ìxéè
¨è
ê3¦àéè
©ù|gë¥]ìxéè
¥åÑf«
zXpsf¼q«
_°
m[mnfjfOqgquz'huwz'mn¯9qd?mZjkrfmsej_smn¯Qkregkl_bmnq`y}_shz[q«
zbcKkrmnZy}dKmnjegkl_bmò
eg_I|rfOzXpbf_scQegkla»z'|3c_s|rk~jklfq¼krm]p=z'hukrºz'm?eO½ çfhufàzbhgfe°
_ÄfOzbqgw¤bfmnfh´z'|lkpOz=egkl_bmq°
_she«
afmQºegkl_bmnkrmKóÓqgk~¯Qf²y}hg_sa sdnz'h´z'm?egffklmK¤j_bmqk~qxeufmnjw°
«
kl|lfeghuw]krmn¬eg_|lfz'hum e«
f._bcQeukra»z'|²c_b|lkljwbÈkreâklqfzsqweu_q«
_°
}Â]wÚz'mÒz'hubdnaÃfm?e9qklakr|~z'h[eg_² fa»z'hu¿þCkrm¿iQfj¶ºegkl_bmNs!e«
z=ec_'egfm?egk~z'|rºÂnzsqfO¯ò
z'|~q_°
_shg°
«
fmÓeghuw?ºklmKÁeg_¤|rfOz'hum¿zÐb_]_Q¯Òc_b|lkljwÚy}hu_ba°
kre«
klmâz¿ÉuÖ¶Ù¶ÌÉ¶ËMðÌáÖ´Øj|~zbquq³_byZc_s|rk~jklfqqgdnj«
zsq³klme«
fÄy}h´z'af°
_bhu¿qegd¯QkrfO¯klmü §¹fzbhgmnq{fez'|½rRþObb=ÿAMz'm¯°
«
k~j«
y}_shµfÅKz'acK|lfÁkrmQºj|ldn¯QfOqe«
f e´zbqg_byR±nmn¯QklmKe«
fÂfqe°
fkl«
euqy}_bh{zmKfdnhuzb|mKfe°
_shgÈa»zbcKcKklmKµy}hu_baqeuz'egfOqeg_Äzsj¶eukr_smnq<½¿|lqg_n	y}_shi]faÃkrºá¸³z'hub_=pÄ¯Kfjk~qgkr_smÚcKhg_QjfquqfOqÔ iQ¸³o*ÕTq°
«
fhufCzbjºegkl_bmq#euzbbf	p=z'huw?klmK¹z'a_bdnmse´q#_'yegklaf	eg_j_sacK|rfegfsÉTí?dnz'ºegkl_bmødKmnqgdKhucKhgk~qgkrmKs|rwÄbfmnfh´z'|lkpfq/eg_ò
©!
©¯hu¨
[©
!AªL«&X©
¯Fäh&X©f¶°
«
fhuf|¨Iklq{e«
f*egklaf¹e«
fzbjegkl_bm9eg_]_beg_»j_bacK|lfegfsnz'm¯¬²k~q{e«
f¯Kklquj_sdKm?e	h´z=eufb½Ï3krmz'|l|rwse«
f/ã7&X®©¯p2ä[&X®©ä	y}_bhuaâz'|~q_ qgffaq_bm¹e«
f`qgdKhgºyMzbjf`hufaklmKk~qgjfm?e&_byKegfhga»q&krmÃq_saÃf`_'yKe«
f{fí?dnz=eukr_smnq&dnqfO¯klmI¼¯Qp=z'm?euzbbf*|rfOz'humKkrmnIü 	z'klhu¯R·þOb'Xÿzbmn¯¢vQºÀc_s|rk~jw[kveufhgºz=eukr_sm³ü {fhe´qfXzsq	z'mn¯ª
qgkve´qkl]|rk~qRþOb!xOÿ½{e`zpsfhuwjhgdn¯Kf|lfpbf| Rkre¹egdKhumnq*_bdKe*e«
z=efOzbj«
_byTe«
faAa»zXwÂfÃe«
_bdK«
e_'yzbq»eghuw?klmKÑeg_µa_Q¯Qkry}w& qg_²eg_µszbkrm¤qg_baf³j_saÃcndQeuz'ºegkl_bmz'|_bh»hufcnhgfOqfmse´z=eukr_smnz'|	zb¯Qp=zbmse´z'sfb½¤¾Ày°
fj_smnqk~¯Qfhe«
f³cKhu_bÂK|lfa_'ya_]¯Kkvy}w]klmKK&¼{e«
fm¤eghuw]krmK²eu_È|lfz'hum¤zhu_bdK«
q«
z'cKklmK9y}dKmnjegkl_bmµqffa»q*eg_|lfzs¯àí?dKkregfmnz'egdKh´z'|l|rweg_z'mz'|lb_shgkre«
aøy}_sh	a/dK|regkrºÀqujzb|rf*p=z'|ldKfºy}dKmnjegkl_bmIz'cKcKhu_XÅ]ºklaz'egkl_bmVû	z'm¯Äz'|re«
_bdK«
kvea»zXwµkrmnkveuklzb|r|lwÈqffaõdKm]dnqgdnz'|eg_euhgw9eg_ÔlÖuÇ'É¶Þ[z»q«
z'cKklmKy}dKmj¶egkl_bmnkreklq e«
fadK|regk~qgjz'|lfãhu_bdn«
p]q½±nmnfä»z'cncKhg_XÅQkla»z=egkl_bmIzbqgcfOj¶e e«
z=ee«
k~q |lfzb¯nq
eg_°
«
klj«
a»zbbfq kve*c_?qgqgkrÂn|rwIc_°
fhgy}dK|û ®e«
k~q°
kl|l|Âfe«
fqgdKÂ#Uxfje _'y&y}dQegdKhuf°
_bhu½¾ámÔe«
k~qCcnzbcfh°
f«
zXpsfÄq«
_°
mâe«
z'eÑc_'eufm?egk~z'|rºÂzbqgf¯q«
z'cKklmK¤huf°
z'h´¯Kq0`&X©¯/Åä¯&X©fI|rfOzXpbfw}mKfOz'hgºx_bcQeukra»zb|c_b|lkljkrfOq¹dKmnj«
zbmKbfO¯R½»¸_bhuf_=psfhOe«
k~q°
zsq*cKhg_=psf¯³eg_Âfe«
f»_bmK|lwexw]cf»_by{q«
z'cKklmKIe«
z=ejzbmàsdnz'h´z'm?egffqgdnj«
krmQºp=z'huklzbmnjfàdKmK|lfquq°
fÑazbbfy}dnhe«
fhzbquqgdKacQegkl_bmnqIz'Â_bdKee«
f¸³o*Õ#½s{dQe7Uxdnqe{zsqqg_bafcKh´zbj¶eukveukr_smKfh´q#dnqgf¯Qk~qgj_bdKm?egºklmKÈfpsfm¤_bmÚdnmn¯Qk~qgj_bdKm?egfO¯¤cKhu_bÂK|lfa»qLMcfh«
zbcnq»eu_ÈklaºcKhu_=pbfÃj_bm]psfhubfmjf_by{z'|lb_shgkre«
aq¶°
fÃÂf|rklfpsfe«
z=e¹y}dQºegdnhgffÅQcfhukrfmnjf°
kre«
c_'eufm?egk~z'|rºÀqexw]|rfq«
z'cKklmK²huf°
z'h´¯Kqa»zXwZz'|~q_|rfOzb¯9_bmKf¼eg_Ã_Qjjzbqgkr_smnz'|l|rwÃeuhgwZq«
z'cnkrmKÃhuf°
z'h´¯Kqe«
z=ez'huf krmqcKklhuf¯»Â]wc_begfmseuklzb|lq]ÂKdQe°
«
k~j«
z'hufcfh«
z'cqmK_beqeghuk~j¶eg|lw³_by`e«
fÃy}_bhua°
f«
zXpsfskrpsfmV½»Ïn_bh¹fÅKz'acK|lfbklm³z'mnzb|r_sbweg_»dqklmKZ¯Qk~qgj_bdKm?eukrmKfpbfm_smIdnmn¯Qk~qgj_bdKm?egfO¯cKhu_bÂK|lfa»qVkre¹k~qj_smnjfkrp=zbÂK|rfÃe«
z=e¹y}_bhjfhgeuzbkrmÑcKhu_bÂK|lfa»qkreTa»zXwÂffzbqgklfh#y}_bh`z'mfÅQcfhgeeg_cKhu_bc_sqgfz¹c_begfmseuklzb|s&y}_bh*z'mÒãgdKmn¯Qk~quj_bdnmseuf¯Kä[q«
zbcKklmK»y}dKmnjegkl_bmy&X®©¯/,äg&X©f¶fpsfm°
«
fm-°  þs½]ÉTpbfm³e«
_sdK«
_bdKh*e«
f_shgfa azXwmn_|l_bmKsfh sdnz'h´z'm?egff¹_bcQeukra»z'|lkrexw[klme«
k~qjzbqgfbqgdnj«
zZq«
z'cKºklmKÃy}dKmnj¶eukr_sma»zXwbQcKdKhuf|lw»y}hu_ba zbmfmKskrmKffhukrmnc_bklm?e _'yp]krf°
=qxeukr|l|sÂf°
_bhge«
euhgw]klmK;qwUxd¯Qkljkr_sdnqg|rwz'm¯°
kve«
jzbhgfs½¾ám²e«
f[qgzbafZqcKklhukveO°
«
fhgfOzbq_sdKhhufsdK|lzbhgkrexwCj_bmn¯Qkregkl_bmq«
zs¯Ä¯Qfazbmn¯QfO¯ÄdnqgklmKàÂ_bdnmn¯Qf¯&¼Tkre»klqzb|lqg_CcK|~z'dqklÂK|lfe«
z=eqg_baf¹cKhuzsj¶eukveukr_smKfh´q	akr«
e°
z'm?e	eg_ÃeuhgwIjfhgeuzbkrmIdKmQºÂ_bdKmn¯Kf¯T&¼½ýz'egdKh´z'|l|rwsnkryfÅQcfhe¼?mn_°
|rfO¯QbfzbÂ_sdQe e«
f¯Q_sazbkrmCklq¹zXp=z'kl|lzbÂK|lfbe«
fmCmK_smQºc_'eufm?egk~z'|#q«
z'cKklmK[y}dKmnj¶ºegkl_bmq	aÃkl«
ezb|lqg_Âf¹y}dK|l|rwz'cncKhg_scKhuklz'egfb½qbdKk~¯Qf|rklmKfqy}_bhj«
_]_?qklmKZq«
z'cKklmKy}dKmj¶egkl_bmq°
f«
zXpsfqgdKbsfqegf¯ÓzÁ¯KklqeuzbmnjfºÂnzsqfO¯«
fdKhgk~qegk~jàzbmn¯¿zÁqgdKÂKb_?z'|rºÂnzsqfO¯«
fdKhuk~qxeuklj*y}_sh¼j«
_]_sqgklmK»c_begfmseuklzb|lqûnÂfjz'dnqgf/q«
z'cKºklmKàk~qÃ_'yÍeufmÐjhgdjk~z'|`eg_Ñazb]krmKà|rfOz'humKklmK³euhuzsj¶euzbÂK|lfb°
fÂf|lkrfpbf¼e«
f*euzsq_byV±nmn¯QklmKÃb_]_Q¯[q«
z'cnkrmK/y}dKmnjegkl_bmnq°
kl|r|Âfzcnhg_sÂK|rfa¬_'y#klmnjhufzsqklmKÃkrac_bhgeuz'mjfb½¢Ú©²±è
é`³r|{ê\´Ã}Ä{è
¦K¥½{ý µk~q[qgdKcKc_bhgegfO¯ÐÂ]w¤zµ{fhubf|lfwÐÏKf|l|l_°
q«
klcV½ª	«
klq°
_bhu°
zsqÁzb|lqg_øqgdKcKc_bhgegfO¯ klm cnz'hge¤Â?w Ák ¸t
¾bh´z'm?eo ãçb'ºáx=º´þºásbnþsãk*ýÁ÷bh´z'm?e³ýbsKþ'ºá?Oº´þºbbnþs]zbmn¯Iý¼iQÏÄbh´z'm?eÉ{^	i?ºá
]s'QX½µ
{7¶{V§^{è
©¹{¥ü {zbkrh´¯RRþOb'bÿ*	z'klh´¯Rá
½^¼½¼xþs'W¶½  fklmQy}_bh´jfaÃfm?eá
fzbhgmnkrmK[krm²j_smseukrm]dK_sdnqeukrafsó*¼¯Qp=z'm?euzbbf/dnc¯Kz'egklmKn½·<ê=G!40#2 ¼ 5#%"<0"L"G)[@!<7#"4(r"G)#%"2<"G)4(!V 4&fº# +#%(!+> GW#%1)4()V°J* #%> <«"ø@?À1?<#"45(!V1)CB8G)2 52V48+#% 5 :BZ¸¼g V!gE="G)]8+#21"!2-Ó#'1!48$4(L8G)Å21!Y<#"X#%Á#$<* #>7f('"G)2)VG["G)Å!(W>!J B4()VirG)#2<*=#%>!b(! :B-n2Ã8JG!8&6#"g
¾ámf¹	Égñð´Ö´Ö´Ø=ËÍÞ]ß=Ù³ñxîÌ}ÛKÖSH¶ÞÌÀÖÉ¶ÞÇ'Ì ËMñ'ÞÇºñ'ÞîÖÉgÖÞ·ð´Öàñ'ÞIÖïQÉgÇ
IÖÌ×Tñ=Éuq=Ù¶½ü `fhe´qf=zbqþOb)tXÿ`fhe´qf=zbq o/½Õ#½Åxþs!t!¶½¼»/Î=ÞÇËMð¹{Égñuß'ÉgÇ]ÃËÍÞ?ß Ç=Þ·Ø¾½&ÜÌË»Çfºñ=ÞÌ Éuñ¿.cÀñvï?Ö¼HH¶½{e«
fmnz»iQjklfm?egkr±jb½ü `fhe´qf=zbq	z'mn¯i«
hgfpbfbRþO?
=ÿ*{fhe´qfXzsq¬o½ÓÕ#½Òzbmn¯i«
hufpbfsTi·½,É ½ëþ?
!½
EÌÀñOð´ÛKÇXÙ¶ÌËMða½&ÜÌË»Ç[ºñ=ÞnÌÉgñÁD?ÛnÖ|»/Ë~ÙðÉuÖÌáÖS?Ë»ÖºÇ=ÙÖ½T¼jzs¯Qfak~j*ÕThgfOqgq½ü `fhe´qf=zbq	z'mn¯ª
qgkve´qkl]|rk~qRþb!x=ÿ{fhgeuqgf=zbqo½	Õ#½zbmn¯ª
qkreuqgkr]|lklq6Ân½ý/½ãxþOb!x!¶½
IÖïQÉgñ'ÝØ=Î=Þ·ÇÃËMðS¹{ÉuñgßbÉgÇÃÝËÍÞ]ß'½{e«
fmnz»iQjkrfm?egkr±j'½ü o_shgklb_z'mn¯³^`_s|r_sa/Âfeeuk RþObb'ÿo_shgklb_ ¸È½ zbmn¯^`_b|l_ba/Âfegegk{¸È½xþs'W¶½w_bÂ_'e[q«
z'cnkrmKóofpbf|l_bcKºkrmK9z'dQeu_bmK_saÃ_sdnqzbbfmse´q	e«
hg_sdK«
|lfzbhgmKklmK½|Ã¼É¶Ì Ëå
ðËMÇH¶ÞnÌáÖ% ËvßsÖÞð´Ö]þ) !¶ó ssQþCÄWs]=K½ü ç fhumè zbmn¯Qf%ºá
fhua»zK·þ!
b=ÿÅç fhgmè zbmn¯Qf2ºá
fhua»zKÃk½ëxþO
s!¶½jÃ¼ØbÇ´ÜÌË	XÖ ÅCÇ=Éuqsñ	gºñ'ÞnÌÉgñQ¹{Éuñ=ÝðuÖÙ´ÙÖ¶Ù¶½Ti]cKhuklmKbfhºÆfhu|lzbn½ü §¹fz'humnq{fezb| ½lRþsb'ÿ[§¹fOz'humnq¸µ½rÃ¸³z'mq_sdKhrÇ½lÃzbmn¯ý n`½/Ç½ëxþObs!¶½µcKcKhu_XÅ]kla»z=euf[cK|~z'mKmnkrmKàkrmÁ|~z'hubfÕÓk*¸³o*ÕTq{p]klzÃhufdnquz'Ân|rf*egh´zZUxfOj¶eu_bhukrfOq½
9¹{ÉgÖáÜÉ¶ËÍÞnÌÁ<=½ü ¸³z=e´z'hukljbRþbb'ÿ¹¸³z=e´z'hukljbV¸µ½7Ân½,þs')½f°
zbhu¯³y}dKmnj¶ºegkl_bmnq³y}_bh²zbjjf|lfh´z=euf¯|rfOz'humKklmKn½ú¾ámp¹{ÉgñOðuÖ´Ö´Ø=ËÍÞ]ß=ÙÐñîÌÍÛnÖN
lÖ	=ÖÞÌÍÛTH¶ÞÌÀÖÉ¶ÞÇ'Ì ËMñ'ÞÇ^ºñ'ÞîÖÉgÖÞð´Öñ=Þ8ÅCÇsðuÛQËÍÞÖ,
Ö´Ç=É¶ÞËÍÞ?ßb½n¸_shg?z'm§zbdQy}a»z'mKm½ü  z'mn¯K|njbp[z'mn¯I |~qeghj'aVþs
'ÿX zbmn¯Q|/j'p·Â½âz'mn¯ |vºqxeuh<jbaIÕ#½Fþs
!½á
fOz'humKkrmn»eg_[¯Khgklpbf/z[ÂKk~jwQj|lfdnqgkrmnhgfkrmQy}_shujfafm?e|lfz'humKklmKZz'm¯Iq«
zbcKkrmnn½T¾ámT¹{Éuñð´Ö´Ö´Ø=ËÍÞ?ß'ÙñxîàÌÍÛnÖTÈTË î´ÌÀÖ´ÖÞnÌ}ÛH¶ÞnÌáÖÉ¶ÞÇ=ÌËMñ=Þ·Çºñ=ÞîÖÉgÖÞð´ÖÄñ'ÞeÅ³Ç'ÝðuÛQËÍÞÖ,
Ö´Ç=É¶ÞËÍÞ?ßb½n¸_shg?z'm§zbdQy}a»z'mKm½ü dnquqf|r|Vþs
'ÿXdnquqgf|l| [i·½0þs
!½á
fOz'humKklmK¿zbbfm?e´qy}_bh dKmjfhgeuzbkrmfm?p]klhg_smKafm?euqMfÅ]egfmn¯QfO¯z'Ânqegh´zbje¶½`¾ám¹{ÉgñOðuÖ´Ö´Ø=ËÍÞ]ß=Ù¤ñxîÐÌÍÛnÖN
rÖ%	XÖÞnÌ}ÛÉÃ¼ÞnÞïnÇ[Ã	º/Å 2ñ=Éuq'ÝÙÛnñuÜ.ñ'Þ£ºñÜïQÌáÇ=ÌËMñ=Þ·Ç,
Ö´Ç=É¶ÞnËÍÞ]ß¼?·ÛKÖ´ñ'É¶Î
9º6½,
?ÝÊ>Ë
<=n¸Czb¯Qk~q_smVQåÚk~qgj_bmnqgkrm½K^{¸ ÕThgfOqgq½ü dnqeVþOb'bÿÅ dnqxeOÂn½Xxþs'W¶½Óo¼_Ècf_bcn|rfÂf«
zXpsf³zbjºj_bh´¯QklmKeg_9`f|r|la»z'm7 q cnhgklmnjklcnzb|_'y_scQegkla»z'|lkvexwKô.i]dKÂQºaÃkreeuf¯9eg_Âs_bdnhgmnzb|·_'yFÉ`j_smK_bak~j*Õ#fhuqgcfj¶eukrpsfq½ü iQz'Qqgkl¯KzÃfezb| ½lVþOb?Xÿ¹iQzbQqk~¯KzKá
½r zXw]a_bm¯R	iR½r zbmn¯ª
_sdKhgfeJ?wsZo/½[xþs?¶½ i«
zbcKkrmnhg_sÂ_be²Âf«
zXp?kl_bdnhdnqklmKcKhukrmnjkrcn|rfOqy}hg_sa krmnqeghudKafm?e´z'|Èj_smn¯Qkregkl_bmKklmK½Ì
ñbÊ´ñ'Ì ËMðÙÓÇ=ÞØMÃ¼ïQÌáñ=Þññ'ï]Ù
EÎXÙ¶ÌÀÖ%/Ù¶b
hsÄ])ó sKþÄ=sn½
ü i]krmn«
z'mn¯{ÇffsVþs'bÿi]klmK«
¼i·½	z'mn¯KÇffbÁ½Xþbb)½ mdKcKcfh¤Â_sdKmn¯ _bm e«
fâ|l_squqÐy}hg_sa z'cKcKhu_XÅQkra»z'egf_bcQeukra»z'|rºÀpXzb|rdnfTy}dKmj¶egkl_bmq½`Å³ÇsðuÛQËÍÞÖ,
ÖuÇ'É¶ÞnËÍÞ]ß'þ2xKó b?*Ässn½ü i]dQeeu_bmz'm¯I	zbheu_n·þs
=ÿi]dQegeg_smV¼½Ii·½Iz'mn¯ 	z'hgeg_½òÃ½ëxþs
)¶½Ì
ÖËÍÞîñ=ÉuðuÖ%»ÖÞÌ,
Ö´Ç=É¶ÞËÍÞ?ßtDSÃ¼Þ8HÞnÌÉgñ=ÝØ=ïnðÌ ËMñ'Þn½¸¾ª
Õhufquq½ü pb_bmý fdKa»z'mKmzbmn¯9¸³_bhubfmqxeufhumV·þ's'ÿ¹pb_sm ýfdQºazbmKmV
Ân½zbmn¯¤¸_bhubfmnqxeufhumVÓk½þ's)½p?·ÛKÖ´ñ=É¶ÎÄñîÍÇÖÙÇ=Þ·ØN
ð´ñ=ÞñËMðrÎ¼Ö´ÛKÇ	OËMñ=É¶½*Õhukrmjfeu_bmt
mnkrpsfhgºqkrexw[Õhufquq]ÕThgklmnjfeg_smVKý f°
Âsfh´qgfwb?±nh´qe	fO¯Qkregkl_bmV½ü åÚkr|l|rk~z'a»q zbmn¯I{zbkrh´¯RRþOb'bÿ*åÚkl|l|rk~z'a»qf½ Â½zbmn¯	z'klhu¯Vá
½·^¼½FxþOb'W¶½ª
kr«
e cfhy}_shga»z'mjfÂ_bdKmn¯nq _bmshgff¯Qwc_s|rk~jklfq`Ânzbqgf¯Z_bm[klacfhy}fOj¶e	p=z'|ldKfy}dnmnj¶eukr_smnq½&¾ám¹{Éuñ=ÝðuÖ´Ö´Ø=ËÍÞ]ß=Ù³ñxîIÌ}ÛKÖK?nÖÞnÌ}Û¼ÏÇlÖ2ñ=Éuq=ÙÛnñuÜÔñ=ÞKÃ¼ØbÇ´ÜÌË	XÖÇ=ÞØ,
Ö´Ç'É¶ÞnËÍÞ?ß
EÎXÙ¶ÌáÖÙ¶½¢^
{è
êì
¢MÐz
§Ké	éÑ¶[éÑ¶è
{©¹{¥]¥]ìá¦Ò¾ámÁe«
k~qZcKcfmn¯QkrÅR°
fqsfe´j«
e«
fcKhg_]_by¼_'y*e«
fmKfjfqºqgkvexw»cnzbhe_byª	«
f_shgfa þb½ÏK_sh	ÂKhgfp]kvexws°
f*bklpbf¼e«
f¹cKhu_]_'y_bmn|rwy}_she«
fÃjzbqgf_byX¨ ]¨oøQû·e«
fbfmKfh´z'|lkpOz=eukr_smklq*_bÂKºp]kr_sdnq¼ÂKdQea_bhufegfO¯Qkl_bdnq½åàfÂfskrm°
kve«
e«
f/y}_s|r|l_°
klmKá
faazn½Ó
<WÐuÐÄHTÔ
H}îÌ}ÛKÖÉgÖàÖ54sË~Ù¶ÌMÙ0©Tªwä
©2â
©¯]ª Ç=ÞØ
¯ùªÔ®Ù¶ïnð´ÛÌ}ÛKÇ'Ìò
®©
<©¯n[° ò
®©
¯h©2¯/G.`ÌÍÛnÖÞàÌÍÛnÖÉgÖÖ54bË~Ù¶Ì Ù
9ÜÉuñuÜnÖÉ<¤Ì ÉuÇ=ÞKÙ¶ËÍÌËMñ=ÞÄî´ïQÞ·ðÌËMñ=ÞnÙ[)Ç=Þ·ØÚÇ¤ÉgÖ×TÇ=ÉgØî´ïQÞðÌËMñ=Þ« Ù¶ïnðuÛÄÌ}ÛKÇ=ÌÞ·ñàñ´ÜÌËÇ&ÜKñ ËMðÎ²ËÍÞu¯*Ë~ÙIñuÜ·ÝÌËÇVËÍÞR0¹;d±
i]bfeuj«
á
faa»zus!óµquqdKaf°
kre«
_bdQeI|r_?qgq[_'ybfmKfh´z'|lkrexw³e«
z=eò
®©
<©¯ni$ò
®©!
¯h©¯nz'mn¯²|lfeÕ¸ò
©!
©¯nöäò
®©
¯h©2¯/6$ÁK½#¾ám[e«
f¹dKmn¯Qk~quj_bdnmseuf¯Ijzsqfsz'|~qg_[zbquqdnaÃf/y}_bh*qklacK|rk~jkrexwe«
z=eÅ© ° I©¯ ½$Må«
fmA¼.þbe«
fÈcKhu_?_byy}_bh©ø ©
¯k~q³mKfOz'hu|rwÓe«
fÐqgzbaÃfsÂKdQe«
zXp?ºklmK»eg_[fmnqgdKhufcKhu_bcfhumKfquq/Uxdnqe¼a»z'sfqkre¼adnj«
a_bhuf¹egfº¯Qkl_bdnq½:åÑf`e«
fmj_smnqeghudnj¶e, zbq&y}_s|r|l_°
qóá
fe
¤=¥¦®©2¯/,¤=¥¦Õ®©¯/,Ôþ Ksz'mn¯|lfeÃH®©
<©¯pbÒz'm¯$H®©
¯h©2¯/,Õi'K½I^`|rfOz'hu|rwA¿ÑÈ
©f6
¯ ½¢k¼mÑe«
fZ_'e«
fh«
z'mn¯R3qklmnjfÅ¯0 Ëøò
°
f«
zXpsfÅ¯®©!
<©¯/¼ò
®©!
<©¯/[zbmn¯Å¯®©!
¯h©¯nëMÕi'ËTò
®©!
¯h©¯n;ò
©!
©¯n,äaÕibi:Å¯®©!
<©¯/¶nzbmn¯«
fmnjfÅ¿ÑÈ
Õ
®©f,
½ kåÑfz'huf*mn_°
hufzb¯Kweg_Zq«
_°
e«
fazbkrmmKfOjfOqgqgkvexw[hgfOqdK|re½¹;d±d±Z83<Wb<]LLOJ}U¾ö=B
quqgdKafò
klqmn_'e»c_begfmseuklzb|vºÂnzsqfO¯R½ZåÑf»mKffO¯Ceu_³q«
_°ù°
fZjz'mµj_bmnqeghudnj¶e<¬qgdnj«e«
z=emn_C_bcKegklazb|`c_b|lk~jw.¿ÑÈ
Õ
klmr¯`klqÃz'|~qg_³_bcQeukra»zb|`klm!½Z{wá
faa»zsnkryò
®©!
<©
¯¯Qfcfm¯Kq_bm
°
f»z'huf¯Q_smKfbû«
fmnjf°
fmKffO¯²_bmK|lwCj_smnqgkl¯Qfhq«
z'cKklmKy}dKmnjegkl_bmnq
ÖA×
ÖØ#ÙØ>Ú
ÛØLÜÏ3krsdKhuf¼snóª	«
f²dKmn|lzbÂf|rfO¯Òe«
klj´Óf¯Qsfqj_shghufqgc_smn¯¤eu_Â_'e«
zbjegkl_bmnq½¼|r|#f¯Kbfq«
zXpbf/cKhg_sÂnz'Ânkr|lkvexw²þs½ª	«
fÃf¯Qsf©

b
©2âZjz'huhukrfOq»z²hgf°
zbhu¯KÕxbQz'm¯Ázb|r|_'e«
fhZfO¯QbfOq«
zXpsfãfhu_Ãhuf°
z'h´¯R½_'ye«
fy}_bhuaò
©!
©¯nbò
©!<©¯;°
«
klj«
¯Q_mK_beT¯Qfcfm¯_bm
¶½¾ÀyëÀ þs3|lfeb
©â
 ©â
ÂfZe«
fI¯QklqegklmKsdKklq«
fO¯Èz'Ânqg_bhuÂKklmKqeuz=eufbû*_'e«
fh°
klqgf³|lfeb
©%âàÂfÑq_saf±KÅQf¯Òqeuz'egfs½Ôý _'eukrmne«
z=e»j_bmqxe´z'm?e/_'ÆRqgfeuq_'ye«
fIhgf°
zbhu¯²¯Q_àmK_'ez=Æ·fj¶eÃe«
f_bcKegklazb|c_s|rk~jw°
«
fm:¬þb°
fa»zXwbVÂ]w³hufcn|lzsjklmKIzb|r|ò
©!©2¯/°
kre«ò
©!<©¯/Ãäò
b
©2âb
©2â*kry{mKfOjfOqgquz'huwbVzsqgqgdKaf°
kre«
_sdQe|r_?qgq{_'y3bfmKfh´z'|lkvexwÃe«
z=eò
b
©2â)b
©2â,ÓK½ý_°
¯Qfº±nmKf&X®©f6£äò
©!b
©%âf*y}_bhzb|r|Ó©b½I{wàzbquqgdKacQegkl_bm²_'yòmK_beÂfkrmK¹c_'eufm?egk~z'|rºÂzbqgf¯R=e«
fhuf	fÅQk~qxe´qb©
)©Ìqgdnj«
e«
z'e`&X©Ì2Wä{&X®©
|° ò
®©
<©Ì;}|lfednqzsqgqgdKaf©
<©Ìb
©2â z'huf¯Qk~qxeukrmj¶eûe«
f/_be«
fh¼jzsqfOqzbhgffkve«
fh¼krac_squqklÂK|lf/_bh«
z'mKº¯Q|lf¯ÈqgklaÃkl|~z'hu|rw^¶½Cåàf[e«
fmÄj_smnqxeuhgdj¶eH klmµe«
fZy}_b|l|l_°
ºklmK°
zXwT qxeukr|l|{zsqgqgdKaklmKr¨ ]¨= !¶½CÏnhg_sa qeuz'egf[©
|rfe¤¥Ý¦b
©â
Ó
¤¥ÝJ¦+Õ©Ì
;øþ Kz'mn¯Iy}hu_baqxe´z=egfOqã©Ì
zbmn¯b
©â|lfe`Â_'e«
zbjegkl_bmnq
zbmn¯
¯n|lfzs¯Ãeg_b
©â°
kve«
cKhu_bÂnzbÂKkr|lkrexw[þb½ |~qg_¯Qf±nmKfÕÄò
®©
©Ì
Ë
ò
®©Ì
b
©â
¹äò
®©
b
©â
zbmn¯|lfe$®©

b
©%âfÞÕxbQ,H§n%§n%§: ³f|lqgf°
«
fhufb½ª	«
klqa_Q¯Qf|Vk~q	kl|r|ldnqegh´z=egfO¯IkrmÏ3krsdKhgfsn½ª	«
fm°
f«
zXpbfÒ
ÑÈ
©

ÕÒ
ÑÈ
©

W¯ Ò
ÑÈ
Õ
©

Õ
Ëuò
®©
b
©2âò
©
<©Ì
Ë
ò
©Ì
b
©â
=ä
ÕÒ
ÑÈ
Õ
©

¯ò
©
<©Ì2Ë
ò
©Ìb
©%âf+°
«
fhgf°
f«
zXpbfÒhuf|lkrfO¯®_bm¬e«
fÓyMzbj¶eÁe«
z=eÆ$ÑÈ
b
©â
ÄÆ]ÑÈ
Õ
b
©â
,ÒÃÂ]w9j_bmnqeghudnj¶eukr_smV½ ç fmnjf¿
ÑÈ
©
 ß
kryÕ£$Án¯ö_be«
fh°
k~qf¿
ÑÈ
Õ
©

ß
¯ökryÕ£$Án_be«
fh°
k~qfk