{"id":258164,"date":"2026-07-13T16:05:57","date_gmt":"2026-07-13T13:05:57","guid":{"rendered":"https:\/\/1kitap1.com\/en\/decision-making-under-uncertainty-kochenderfer\/"},"modified":"2026-07-13T16:05:57","modified_gmt":"2026-07-13T13:05:57","slug":"decision-making-under-uncertainty-kochenderfer","status":"publish","type":"post","link":"https:\/\/1kitap1.com\/en\/decision-making-under-uncertainty-kochenderfer\/","title":{"rendered":"Decision Making Under Uncertainty – Kochenderfer"},"content":{"rendered":"
\"<\/figure>\n

Duff discusses the formulation of model-based reinforcement learning as a Bayes- adaptive Markov decision process [12]. In general, solving such a belief-state formulation exactly is intractable. Strens applies the concept of Thompson sampling [14] to model- based reinforcement learning [13]. Variants of the online planning algorithms presented in the previous chapter have been extended to Bayesian model-based reinforcement learning, including sparse sampling [15] and Monte Carlo tree search [16], [17]. Model-free reinforcement learning algorithms are often used for situations in which it is not feasible to build an explicit representation of the transition and reward models.<\/p>\n

Q-learning and Sarsa are two commonly used model-free techniques. Eligibility traces were proposed in the context of temporal difference learning by Sutton [18], and they were extended to Sarsa(\u03bb) [19] and Q (\u03bb) [20], [21]. Much of the ongoing work in the \ufb01eld of reinforcement learning is concerned with generalizing from limited experience. The recent book Reinforcement Learning and Dynamic Programming Using Function Approximators by Busoniu et al. surveys a variety of different local and global function approximation methods [22].<\/p>\n

Several different abstraction methods have been proposed over the years [23]\u2013[26]. Although not discussed in this chapter, there has been some work on Bayesian approaches to model-free reinforcement learning. One approach is to maintain a dis- tribution over state-action values [27], [28]. There are also Bayesian policy gradient methods that have been used with some success [29]. Multiagent reinforcement learning was also not discussed in this chapter, but Busoniu, Babuska, and De Schutter survey recent research in the area [30]. 1. J.C. Gittins, \u201cBandit Processes and Dynamic Allocation Indices,\u201d Journal of the Royal Statistical Society.<\/p>\n

Series B (Methodological), vol. 41, no. 2, pp. 148\u2013177, 1979. 2. R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998. 3. M. Wiering and M. van Otterlo, eds., Reinforcement Learning: State of the Art. New York: Springer, 2012. 4. T.<\/p>\n

Kovacs and R. Egginton, \u201cOn the Analysis and Design of Software for Rein- forcement Learning, with a Survey of Existing Systems,\u201d Machine Learning, vol. 84, no. 1-2, pp. 7\u201349, 2011. doi: 10.1007\/s10994-011-5237-8. 5. J. Gittins, K. Glazebrook, and R. Weber, Multi-Armed Bandit Allocation Indices, 2nd ed. Hoboken, NJ: Wiley, 2011. \u0012\u0014\u0012 6. J. Ni\u00f1o-Mora, \u201cA (2\/3)n Fast-Pivoting Algorithm for the Gittins Index and Optimal Stopping of a Markov Chain,\u201d INFORMS Journal on Computing, vol. 19, no. 4, pp. 596\u2013606, 2007.<\/p>\n

doi: 10.1287\/ijoc.1060.0206. 7. I.M. Sonin, \u201cA Generalized Gittins Index for a Markov Chain and Its Recursive Calculation,\u201d Statistics and Probability Letters, vol. 78, no. 12, pp. 1526\u20131533, 2008. doi: 10.1016\/j.spl.2008.01.049. 8. P.R.<\/p>\n

\n

Chaw-Bing Chang and Keh-Ping Dunn MIT Lincoln Laboratory is a federally funded research and development center that applies advanced technology to problems of national security. The books in the MIT Lincoln Laboratory Series cover a broad range of technology areas in which Lincoln Laboratory has made leading contributions. The books listed above and future volumes in this series renew the knowledge-sharing tradition established by the seminal MIT Radiation Laboratory Series published between 1947 and 1953. %FDJTJPO .BLJOH 6OEFS 6ODFSUBJOUZ 5IFPSZ BOE “QQMJDBUJPO .ZLFM +\u000f ,PDIFOEFSGFS XJUI DPOUSJCVUJPOT GSPN $ISJTUPQIFS “NBUP (JSJTI $IPXEIBSZ +POBUIBO 1\u000f )PX )BZMFZ +\u000f %BWJTPO 3FZOPMET +BTPO 3\u000f 5IPSOUPO 1FESP “\u000f 5PSSFT\u000e$BSSBTRVJMMP \/\u000f ,FNBM \u00c3SF +PIO 7JBO \u0268F .*5 1SFTT $BNCSJEHF .BTTBDIVTFUUT -POEPO &OHMBOE \u00aa \u0013\u0011\u0012\u0016 .BTTBDIVTFUUT *OTUJUVUF PG 5FDIOPMPHZ “MM SJHIUT SFTFSWFE\u000f \/P QBSU PG UIJT CPPL NBZ CF SFQSPEVDFE JO BOZ GPSN CZ BOZ FMFDUSPOJD PS NFDIBOJDBM NFBOT JODMVEJOH QIPUPDPQZJOH SFDPSEJOH PS JOGPSNBUJPO TUPSBHF BOE SFUSJFWBM XJUIPVU QFSNJTTJPO JO XSJUJOH GSPN UIF QVCMJTIFS\u000f .*5 1SFTT CPPLT NBZ CF QVSDIBTFE BU TQFDJBM RVBOUJUZ EJTDPVOUT GPS CVTJOFTT PS TBMFT QSPNPUJPOBM VTF\u000f ‘PS JOGPSNBUJPO QMFBTF FNBJM TQFDJBM@TBMFT!NJUQSFTT\u000fNJU\u000fFEV\u000f \u0268JT CPPL XBT TFU JO “EPCF (BSBNPOE 1SP CZ UIF BVUIPS JO -“5&9\u000f 1SJOUFE BOE CPVOE JO UIF 6OJUFE 4UBUFT PG “NFSJDB\u000f -JCSBSZ PG $POHSFTT $BUBMPHJOH\u000eJO\u000e1VCMJDBUJPO %BUB ,PDIFOEFSGFS .ZLFM +\u000f \u0012\u001a\u0019\u0011o %FDJTJPO NBLJOH VOEFS VODFSUBJOUZ \u001b UIFPSZ BOE BQQMJDBUJPO \u0010 .ZLFM +\u000f ,PDIFOEFSGFS XJUI $ISJTUPQIFS “NBUP (JSJTI $IPXEIBSZ +POBUIBO 1\u000f )PX )BZMFZ +\u000f %BWJTPO 3FZOPMET +BTPO 3\u000f \u0268PSOUPO 1FESP “\u000f 5PSSFT\u000e$BSSBTRVJMMP \/\u000f ,FNBM \u00c3SF BOE +PIO 7JBO\u000f Q\u000f DN \u0089 -JODPMO -BCPSBUPSZ TFSJFT *ODMVEFT CJCMJPHSBQIJDBM SFGFSFODFT BOE JOEFY\u000f *4#\/ \u001a\u0018\u0019\u000e\u0011\u000e\u0013\u0017\u0013\u000e\u0011\u0013\u001a\u0013\u0016\u000e\u0015 IBSEDPWFS \u001b BML\u000f QBQFS \u0012\u000f *OUFMMJHFOU DPOUSPM TZTUFNT\u000f \u0013\u000f “VUPNBUJD NBDIJOFSZ\u000f \u0014\u000f %FDJTJPO NBLJOH\u0089.BUIFNBUJDBM NPEFMT\u000f *\u000f 5JUMF\u000f 5+\u0013\u0012\u0018\u000f\u0016\u000f,\u0017\u0014 \u0013\u0011\u0012\u0016 \u0011\u0011\u0014h\u000f\u0016\u0017\u0089ED\u0013\u0014 \u0013\u0011\u0012\u0015\u0011\u0015\u0019\u0012\u0013\u0018 \u0012\u0011 \u001a \u0019 \u0018 \u0017 \u0016 \u0015 \u0014 \u0013 \u0012 %FEJDBUJPO To my family.<\/p>\n<\/blockquote>\n

This is a short excerpt from the opening of “” by Unknown, quoted for review and introduction purposes. All rights belong to the copyright holders.<\/em><\/p>\n

\n
\n

Table of Contents<\/p>\nToggle<\/span><\/path><\/svg><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n