Stochastic Optimization in a Cumulative Prospect Theory Framework

Stochastic Optimization in a Cumulative Prospect Theory Framework. Jie, C., L.A., P., Fu, M., Marcus, S., & Szepesvári, C. IEEE Transactions on Automatic Control, 63(9):2867–2882, 2018.

Link

Stochastic Optimization in a Cumulative Prospect Theory Framework [pdf]

Paper abstract bibtex

Cumulative prospect theory (CPT) is a popular approach for modeling human preferences. It is based on probabilistic distortions and generalizes expected utility theory. We bring CPT to a stochastic optimization framework and propose algorithms for both estimation and optimization of CPT-value objectives. We propose an empirical distribution function-based scheme to estimate the CPT-value and then use this scheme in the inner loop of a CPT-value optimization procedure. We propose both gradient-based as well as gradient-free CPT-value optimization algorithms that are based on two well-known simulation optimization ideas: simultaneous perturbation stochastic approximation (SPSA) and model-based parameter search (MPS), respectively. We provide theoretical convergence guarantees for all the proposed algorithms and also illustrate the potential of CPT-based criteria in a traffic signal control application.

@article{JiPrFuMaSz18,
	abstract = {Cumulative prospect theory (CPT) is a popular approach for modeling human preferences. It is based on probabilistic distortions and generalizes expected utility theory.  We bring CPT to a stochastic optimization framework and propose algorithms for both estimation and optimization of CPT-value objectives. We propose an empirical distribution function-based scheme to estimate the CPT-value and then use this scheme in the inner loop of a CPT-value optimization procedure. We propose both gradient-based as well as gradient-free CPT-value optimization algorithms that are based on two well-known simulation optimization ideas: simultaneous perturbation stochastic approximation (SPSA) and model-based parameter search (MPS), respectively.  We provide theoretical convergence guarantees for all the proposed algorithms
and also illustrate the potential of CPT-based criteria in a traffic signal control application.
},
	author = {Jie, C. and Prashanth L.A. and Fu, M.C. and Marcus, S. and Szepesv{\'a}ri, Cs.},
	date = {2018-01},
	date-added = {2018-03-11 17:24:12 +0000},
	date-modified = {2019-07-20 10:15:20 -0600},
	journal = {IEEE Transactions on Automatic Control},
	keywords = {risk-sensitive criteria, Markov Decision Processes, SPSA, optimization, stochastic optimization, reinforcement learning, cumulative prospect theory},
	number = {9},
	pages = {2867--2882},
	rating = {0},
	read = {Yes},
	title = {Stochastic Optimization in a Cumulative Prospect Theory Framework},
	url_link = {http://ieeexplore.ieee.org/document/8014469/},
	url_paper = {2018-cpt-rl-tac.pdf},
	volume = {63},
	year = {2018},
	Bdsk-File-1 = {YnBsaXN0MDDSAQIDBFxyZWxhdGl2ZVBhdGhZYWxpYXNEYXRhXxBKLi4vLi4vLi4vTGlicmFyeS5wYXBlcnMzL0ZpbGVzL0Q5L0Q5QkEzMkVGLTMyQ0QtNERBRS04MkQzLTgyNjY2QTgyOUY0Ri5wZGZPEQHaAAAAAAHaAAIAAAxNYWNpbnRvc2ggSEQAAAAAAAAAAAAAAAAAAAAAAAAAQkQAAf////8fRDlCQTMyRUYtMzJDRC00REFFI0ZGRkZGRkZGLnBkZgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAAAAAAAAAAAAAADAAQAAAogY3UAAAAAAAAAAAAAAAAAAkQ5AAIAWS86VXNlcnM6Y3NhYmE6RG9jdW1lbnRzOkxpYnJhcnkucGFwZXJzMzpGaWxlczpEOTpEOUJBMzJFRi0zMkNELTREQUUtODJEMy04MjY2NkE4MjlGNEYucGRmAAAOAFIAKABEADkAQgBBADMAMgBFAEYALQAzADIAQwBEAC0ANABEAEEARQAtADgAMgBEADMALQA4ADIANgA2ADYAQQA4ADIAOQBGADQARgAuAHAAZABmAA8AGgAMAE0AYQBjAGkAbgB0AG8AcwBoACAASABEABIAV1VzZXJzL2NzYWJhL0RvY3VtZW50cy9MaWJyYXJ5LnBhcGVyczMvRmlsZXMvRDkvRDlCQTMyRUYtMzJDRC00REFFLTgyRDMtODI2NjZBODI5RjRGLnBkZgAAEwABLwAAFQACAAz//wAAAAgADQAaACQAcQAAAAAAAAIBAAAAAAAAAAUAAAAAAAAAAAAAAAAAAAJP},
	Bdsk-Url-1 = {http://ieeexplore.ieee.org/document/8014469/},
	Bdsk-Url-2 = {https://dx.doi.org/10.1109/TAC.2017.2743163}}

Downloads: 0