Adaptive critic methods for stochastic systems with input-dependent noise

Randa Herzallah*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


In this paper, a novel dual heuristic programming (DHP) adaptive-critic-based cautious controller is proposed. The proposed controller avoids the pre-identification training phase of the forward model by taking into consideration model uncertainty when calculating the control law. It is suitable for linear and nonlinear, deterministic and stochastic control systems that are characterized by functional uncertainty. Convergence of the proposed DHP adaptive critic method to the correct value of the cost function is proven by evaluating analytically the correct value of the cost function, which satisfies the Bellman equation, and compares it to that calculated by the proposed method in a simple linear quadratic example. Moreover, the performance of the proposed cautious controller is demonstrated on linear one-dimensional and multidimensional examples.

Original languageEnglish
Pages (from-to)1355-1362
Number of pages8
Issue number8
Early online date14 Jun 2007
Publication statusPublished - 1 Aug 2007


  • Adaptive critic
  • Dual heuristic programming
  • Functional uncertainty
  • Neural networks
  • Stochastic systems


Dive into the research topics of 'Adaptive critic methods for stochastic systems with input-dependent noise'. Together they form a unique fingerprint.

Cite this