Onesimo Hernandez-lerma's Books

Markov Chains and Invariant Probabilities

By: Onesimo Hernandez-Lerma,Jean B. Lasserre

Write a review Read reviews Take a Quiz Solve Book Puzzle

Markov Chains and Invariant Probabilities

By: Onesimo Hernandez-Lerma,Jean B. Lasserre

This book is about discrete-time, time-homogeneous, Markov chains (Mes) and their ergodic behavior. To this end, most of the material is in fact about stable Mes, by which we mean Mes that admit an invariant probability measure. To state this more precisely and give an overview of the questions we shall be dealing with, we will first introduce some notation and terminology. Let (X,B) be a measurable space, and consider a X-valued Markov chain ~. = {~k' k = 0, 1, ... } with transition probability function (t.pJ.) P(x, B), i.e., P(x, B) := Prob (~k+1 E B I ~k = x) for each x E X, B E B, and k = 0,1, .... The Me ~. is said to be stable if there exists a probability measure (p.m.) /.l on B such that (*) VB EB. /.l(B) = Ix /.l(dx) P(x, B) If (*) holds then /.l is called an invariant p.m. for the Me ~. (or the t.p.f. P).

Average ratings

N/A

Write a review

Solve jigsaw puzzle

Age

Page Count

234

Publisher

Springer Science & Business Media

Published Date

ISBN 10

3764370009

ISBN 13

9783764370008

Discrete-Time Markov Control Processes

Basic Optimality Criteria

By: Onesimo Hernandez-Lerma,Jean B. Lasserre

Write a review Read reviews Take a Quiz Solve Book Puzzle

Discrete-Time Markov Control Processes

Basic Optimality Criteria

By: Onesimo Hernandez-Lerma,Jean B. Lasserre

This book presents the first part of a planned two-volume series devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes (MCPs). Interest is mainly confined to MCPs with Borel state and control (or action) spaces, and possibly unbounded costs and noncompact control constraint sets. MCPs are a class of stochastic control problems, also known as Markov decision processes, controlled Markov processes, or stochastic dynamic pro grams; sometimes, particularly when the state space is a countable set, they are also called Markov decision (or controlled Markov) chains. Regardless of the name used, MCPs appear in many fields, for example, engineering, economics, operations research, statistics, renewable and nonrenewable re source management, (control of) epidemics, etc. However, most of the lit erature (say, at least 90%) is concentrated on MCPs for which (a) the state space is a countable set, and/or (b) the costs-per-stage are bounded, and/or (c) the control constraint sets are compact. But curiously enough, the most widely used control model in engineering and economics--namely the LQ (Linear system/Quadratic cost) model-satisfies none of these conditions. Moreover, when dealing with "partially observable" systems) a standard approach is to transform them into equivalent "completely observable" sys tems in a larger state space (in fact, a space of probability measures), which is uncountable even if the original state process is finite-valued.

Average ratings

N/A

Write a review

Solve jigsaw puzzle

Age

Page Count

223

Publisher

Springer Science & Business Media

Published Date

ISBN 10

1461207290

ISBN 13

9781461207290

Further Topics on Discrete-Time Markov Control Processes

By: Onesimo Hernandez-Lerma,Jean B. Lasserre

Write a review Read reviews Take a Quiz Solve Book Puzzle

Further Topics on Discrete-Time Markov Control Processes

By: Onesimo Hernandez-Lerma,Jean B. Lasserre

Devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes, the text is mainly confined to MCPs with Borel state and control spaces. Although the book follows on from the author's earlier work, an important feature of this volume is that it is self-contained and can thus be read independently of the first. The control model studied is sufficiently general to include virtually all the usual discrete-time stochastic control models that appear in applications to engineering, economics, mathematical population processes, operations research, and management science.

Average ratings

N/A

Write a review

Solve jigsaw puzzle

Age

Page Count

286

Publisher

Springer Science & Business Media

Published Date

ISBN 10

1461205611

ISBN 13

9781461205616

Selected Topics On Continuous-time Controlled Markov Chains And Markov Games

By: Tomas Prieto-rumeau,Onesimo Hernandez-lerma

Write a review Read reviews Take a Quiz Solve Book Puzzle

Selected Topics On Continuous-time Controlled Markov Chains And Markov Games

By: Tomas Prieto-rumeau,Onesimo Hernandez-lerma

This book concerns continuous-time controlled Markov chains, also known as continuous-time Markov decision processes. They form a class of stochastic control problems in which a single decision-maker wishes to optimize a given objective function. This book is also concerned with Markov games, where two decision-makers (or players) try to optimize their own objective function. Both decision-making processes appear in a large number of applications in economics, operations research, engineering, and computer science, among other areas.An extensive, self-contained, up-to-date analysis of basic optimality criteria (such as discounted and average reward), and advanced optimality criteria (e.g., bias, overtaking, sensitive discount, and Blackwell optimality) is presented. A particular emphasis is made on the application of the results herein: algorithmic and computational issues are discussed, and applications to population models and epidemic processes are shown.This book is addressed to students and researchers in the fields of stochastic control and stochastic games. Moreover, it could be of interest also to undergraduate and beginning graduate students because the reader is not supposed to have a high mathematical background: a working knowledge of calculus, linear algebra, probability, and continuous-time Markov chains should suffice to understand the contents of the book.

Average ratings

N/A

Write a review

Solve jigsaw puzzle

Age

Page Count

292

Publisher

World Scientific

Published Date

ISBN 10

1908977639

ISBN 13

9781908977632

Adaptive Markov Control Processes

By: Onesimo Hernandez-Lerma

Write a review Read reviews Take a Quiz Solve Book Puzzle

Adaptive Markov Control Processes

By: Onesimo Hernandez-Lerma

This book is concerned with a class of discrete-time stochastic control processes known as controlled Markov processes (CMP's), also known as Markov decision processes or Markov dynamic programs. Starting in the mid-1950swith Richard Bellman, many contributions to CMP's have been made, and applications to engineering, statistics and operations research, among other areas, have also been developed. The purpose of this book is to present some recent developments on the theory of adaptive CMP's, i. e. , CMP's that depend on unknown parameters. Thus at each decision time, the controller or decision-maker must estimate the true parameter values, and then adapt the control actions to the estimated values. We do not intend to describe all aspects of stochastic adaptive control; rather, the selection of material reflects our own research interests. The prerequisite for this book is a knowledgeof real analysis and prob ability theory at the level of, say, Ash (1972) or Royden (1968), but no previous knowledge of control or decision processes is required. The pre sentation, on the other hand, is meant to beself-contained,in the sensethat whenever a result from analysisor probability is used, it is usually stated in full and references are supplied for further discussion, if necessary. Several appendices are provided for this purpose. The material is divided into six chapters. Chapter 1 contains the basic definitions about the stochastic control problems we are interested in; a brief description of some applications is also provided.

Average ratings

N/A

Write a review

Solve jigsaw puzzle

Age

Page Count

160

Publisher

Springer Science & Business Media

Published Date

ISBN 10

1441987142

ISBN 13

9781441987143

Book's by Onesimo Hernandez-lerma

Markov Chains and Invariant Probabilities

Markov Chains and Invariant Probabilities

Discrete-Time Markov Control Processes

Discrete-Time Markov Control Processes

Further Topics on Discrete-Time Markov Control Processes

Further Topics on Discrete-Time Markov Control Processes

Selected Topics On Continuous-time Controlled Markov Chains And Markov Games

Selected Topics On Continuous-time Controlled Markov Chains And Markov Games

Adaptive Markov Control Processes

Adaptive Markov Control Processes

Username

Password

Username

Password

Password again

Birthday

Email