HYEONG SOO CHANG

RESEARCH INTERESTS (System Modeling and Optimization Lab)

• (Partially Observable) Markov Decision Process a.k.a. Stochastic Dynamic Programming
• Game Theory with emphasis on Markov Games
• Stochastic Modeling and Decision Theory
• Computational Learning Theory
• Computational Intelligence
• Applications of theories in the above areas into Stochastic Resource Control Problems,
  e.g., in Telecommunications Network, Operations Research and Management Science, AI, Computational Biology.

PUBLICATIONS

1. H. S. Chang, M. Fu, J. Hu, and S. I. Marcus, "Simulation-based Algorithms for Markov Decision Processes,"
   Series: Communications and Control Engineering, Springer, 2007.3. About this book.
   ISBN: 978-1-84628-689-6. cover, preface/TOC/bibliography/index. To order, visit Amazon.com.
   

1. R. L. Givan, E. K. P. Chong, and H. S. Chang, "Scheduling Multiclass Packet Streams to Minimize Weighted Loss,"
   QUESTA: Queueing Systems, Theory and Application, Vol. 41, No.3, 2002, pp. 241-270.
2. H. S. Chang, P. Fard, S. I. Marcus, and M. Shayman, "Multi-time Scale Markov Decision Processes,"
   IEEE Trans. on Automatic Control, Vol. 48, No. 6, 2003, pp. 976-987.
3. H. S. Chang and S. I. Marcus, "Approximate Receding Horizon Approach for Markov Decision Processes: Average Reward Case,"
   J. of Mathematical Analysis and Applications, Vol. 286, No. 2, 2003, pp. 636-651.
4. H. S. Chang, "Localization and A Distributed Local Optimal Solution Algorithm for a Class of Multi-agent Markov Decision Processes,"
   Int. J. of Control, Automation, and Systems, Vol. 1, No. 3, 2003, pp. 358-367.
5. H. S. Chang and S. I. Marcus, "Two-Person Zero-Sum Markov Games: Receding Horizon Approach,"
   IEEE Trans. on Automatic Control, Vol. 48, No. 11, 2003, pp. 1951-1961.
6. H. S. Chang, R. L. Givan, and E. K. P. Chong, "Parallel Rollout for Online Solution of Partially Observable Markov Decision Processes,"
   DEDS: Discrete Event Dynamic Systems: Theory and Application, Vol. 14, No. 3, 2004, pp. 309-341.
7. H. S. Chang, "On Ordinal Comparison of Policies in Markov Reward Processes,"
   JOTA: J. of Optimization Theory and Applications, Vol. 122, No. 1, 2004, pp. 207-217.
8. H. S. Chang, "Multi-policy Iteration with a Distributed Voting,"
   ZOR: Mathematical Methods of Operations Research, Vol. 60, No. 2, 2004.11, pp. 299-310.
9. H. S. Chang, "A Model for Multi-time Scaled Sequential Decision Making Processes with Adversary,"
   Mathematical and Computer Modeling of Dynamical Systems, Vol. 10, No. 3-4, 2004.12, pp. 287-302.
10. H. S. Chang, M. Fu, J. Hu, and S. I. Marcus, "An Adaptive Sampling Algorithm for Solving Markov Decision Processes,"
    Operations Research, Vol. 53, No. 1, 2005.1, pp. 126-139. (Addendum for Continuous Action Space)
11. H. S. Chang, "Multi-policy Improvement in Stochastic Optimization with Forward Recursive Function Criteria,"
    J. of Mathematical Analysis and Applications, Vol. 305, No. 1, 2005.5, pp. 130-139.
12. H. S. Chang, "On the Probability of Correct Selection by Distributed Voting in Stochastic Optimization,"
    JOTA: J. of Optimization Theory and Applications, Vol. 125, No. 1, 2005.4, pp. 231-240.
13. H. S. Chang, "Error Bounds for Finite Step Approximations for Solving Infinite Horizon Controlled Markov Set-Chains,"
    IEEE Trans. on Automatic Control, Vol. 50, No. 9, 2005.9, pp. 1413-1418.
14. H. S. Chang, H-G. Lee, M. Fu, and S. I. Marcus, "Evolutionary Policy Iteration for Solving Markov Decision Processes,"
    IEEE Trans. on Automatic Control, Vol. 50, No. 11, 2005.11, pp. 1804-1808.
15. H. S. Chang, "Book Review: Advances in Dynamic Games, Applications to Economics, Finance, Optimization, and Stochastic Control,
    by A. S. Nowak and K. Szajowski (Eds.)"
    Automatica, Vol. 42, No. 1, 2006.1, pp. 190-192.
16. H. S. Chang, "Converging Marriage in Honey-Bees Optimization and Application to Stochastic Dynamic Programming,"
    J. of Global Optimization, Vol. 35, No. 3, 2006.7, pp. 423-441.
17. H. S. Chang, "On Convergence Rate of the Shannon Entropy Rate of Ergodic Markov Chains via Sample-path Simulation,"
    Statistics & Probability Letters, Vol. 76, No. 12, 2006.7, pp. 1261-1264.
18. H. S. Chang, "Perfect Information Two-Person Zero-Sum Markov Games with Imprecise Transition Probabilities,"
    ZOR: Mathematical Methods of Operations Research, Vol. 64, No. 2, 2006.10, pp. 335-351.
19. H. S. Chang, "A Policy Improvement Method in Constrained Stochastic Dynamic Programming,"
    IEEE Trans. on Automatic Control, Vol. 51, No. 9, 2006.9, pp. 1523-1526.
20. H. S. Chang, M. Fu, J. Hu, and S. I. Marcus, "A Survey of Some Simulation-Based Algorithms for Markov Decision Processes,"
    Communications in Information and Systems, Special Issue on Stochastic Control and Filtering to honor the 65th birthday of
    Tyrone Duncan, Vol. 7, No. 1, 2007.7, pp. 59-92.
21. H. S. Chang, "A Policy Improvement Method for Constrained Average Markov Decision Processes,"
    Operations Research Letters, Vol. 35, No. 4, 2007.7, pp. 434-438.
22. H. S. Chang, M. Fu, J. Hu, and S. I. Marcus, "An Asymptotically Efficient Simulation-Based Algorithm for Finite Horizon Stochastic Dynamic Programming,"
    IEEE Trans. on Automatic Control, Vol. 52, No.1, 2007.1, pp. 89-94.
23. H. S. Chang and E. K. P. Chong, "Solving Controlled Markov Set-Chains with Discounting via Multi-policy Improvement,"
    IEEE Trans. on Automatic Control, Vol. 52, No.3, 2007.3, pp. 564-569.
24. H. S. Chang, M. Fu, J. Hu, and S. I. Marcus, "Recursive Learning Automata Approach to Markov Decision Processes,"
    IEEE Trans. on Automatic Control, Vol. 52, No.7, 2007.7, pp. 1349-1355.
25. H. S. Chang, "Finite Step Approximation Error Bounds for Solving Average Reward Controlled Markov Set-Chains,"
    IEEE Trans. on Automatic Control, Vol. 53, No.1, 2008.2, pp. 350-355.
26. H. S. Chang, "Converging Co-Evolutionary Algorithm for Two-Person Zero-Sum Discounted Markov Games with Perfect Information,"
    IEEE Trans. on Automatic Control, Vol. 53, No.2, 2008.3, pp. 596-601.
27. H. S. Chang, "Decentralized Learning in Finite Markov Chains: Revisited,"
    IEEE Trans. on Automatic Control, Vol. 54, No.7, 2009.7, pp. 1648-1653.
28. H. S. Chang, J. Hu, M. Fu, and S. I. Marcus, "Adaptive Adversarial Multi-Armed Bandit Approach to Two-Person Zero-Sum Markov Games,"
    IEEE Trans. on Automatic Control, Vol. 55, No. 2, 2010.2, pp. 463-468.
29. J. Hu, H. S. Chang, M. Fu, and S. I. Marcus, "Dynamic Sample Budget Allocation in Model-Based Optimization,"
    J. of Global Optimization, to appear, 2010.

1. H. S. Chang, R. L. Givan, and E. K. P. Chong, On-line Scheduling via Sampling,
   in Proc. of the 5th Int. Conf. on Artificial Intelligence Planning and Scheduling (AIPS), 2000, pp. 62-71.
2. E. K. P. Chong, R. L. Givan, and H. S. Chang, A Framework for Simulation-based Network Control via Hindsight Optimization,
   in Proc. of the 39th IEEE Conf. on Decision and Control, Vol. 2, 2000, pp. 1433-1438 (Invited).
3. H. S. Chang, P. Fard, S. I. Marcus, and M. Shayman, A Model for Multi-time Scaled Sequential Decision Making Processes,
   in Proc. of the 41st IEEE Conf. on Decision and Control, Vol. 4, 2002, pp. 3813-3818.
4. H. S. Chang and S. I. Marcus, Receding Horizon Approach to Markov Games for Infinite Horizon Discounted Cost,
   in Proc. of the 41st IEEE Conf. on Decision and Control, Vol. 2, 2002, pp. 1380-1385 (Invited).
5. K. Kuo, S. Phuvoravan, R. La, S. Bhattacharjee, M. Shayman, and H. S. Chang, On the use of flow migration for handling short-term overloads,
   in Proc. of the IEEE Globecom, Vol. 6, 2003, pp. 3108-3112.
6. H. S. Chang and M. Fu, A Distributed Algorithm for Solving a Class of Multi-agent Markov Decision Problems,
   in Proc. of the 42nd IEEE Conf. on Decision and Control, Vol. 5, 2003, pp. 5341-5346.
7. H. S. Chang, M. Fu, and S. I. Marcus, An Asymptotically Efficient Algorithm for Solving Finite Horizon Stochastic Dynamic Programming Problems,
   in Proc. of the 42nd IEEE Conf. on Decision and Control, Vol. 4, 2003, pp. 3818-3823.
8. S. W. Kim and H. S. Chang, Parallelizing parallel rollout algorithm for solving Markov decision processes,
   Lecture Notes in Computer Science, Springer-Verlag, Vol. 2176, pp. 122-136, 2003.
9. H. S. Chang, W. J. Gutjahr, J. Yang, and S. Park, An Ant System Approach to Markov Decision Processes,
   in Proc. of the 23rd American Control Conference, Vol. 4, 2004, pp. 3820-3825. (The Best Technical Presentation in Stochastic Methods Session)
   A longer version available as Tech. Rep. 2003-10, Dept. of Statistics and Decision Support Systems, Univ. of Vienna, Vienna, Austria, 2003.
10. H. S. Chang, On the Use of Blackwell's Approachability Theorem for Stochastic Dynamic Programming,
    in Proc. of the IFAC Workshop on Adaptation and Learning in Control and Signal Processing (ALCOSP 04), 2004, pp. 765-770.
    
11. H. S. Chang, An Ant System Based Exploration-Exploitation for Reinforcement Learning,
    in Proc. of the IEEE Conf. on Systems, Man, and Cybernetics, Vol. 4, 2004, pp. 3805-3810.
12. H. S. Chang, An Adaptation of Particle Swarm Optimization for Markov Decision Processes,
    in Proc. of the IEEE Conf. on Systems, Man, and Cybernetics, Vol. 2, 2004, pp. 1643-1648.
13. H. S. Chang and M. Fu, Localization for a Class of Two-Team Zero-Sum Markov Games,
    in Proc. of the 43rd IEEE Conf. on Decision and Control, Vol. 5, 2004, pp. 4844-4849.
14. M. Fu, J. Hu, S. Marcus, and H. S. Chang, Population-Based Evolutionary Approaches for Solving Markov Decision Processes,
    in Proc. of the 17th Triennial Conf. of the Int. Federation of Operational Research Societies (IFORS), 2005.
15. S-Y. Lee and H. S. Chang, An Ant System Based Multicasting in Mobile Ad Hoc Network,
    in Proc. of the IEEE Congress on Evolutionary Computation, Vol. 2, 2005, pp. 1583-1588.
16. H. S. Chang, M. Fu, and S. I. Marcus, Recursive Learning Automata for Control of Partially Observable Markov Decision Processes,
    in Proc. of the Joint 44th IEEE Conf. on Decision and Control and European Control Conf. (CDC-ECC'05), 2005, pp. 6091-6096.
17. H. S. Chang and E. K. P. Chong, On Solving Controlled Markov Set-Chains via Multi-Policy Improvement,
    in Proc. of the Joint 44th IEEE Conf. on Decision and Control and European Control Conf. (CDC-ECC'05), 2005, pp. 8058-8063.
18. S-Y. Lee and H. S. Chang, Durable Distance Vector Multicasting Protocol for Mobile Ad Hoc Networks,
    in Proc. of the IEEE Conf. on Networking, Sensing, and Control, 2006, pp. 6-11.
19. H. S. Chang, Reinforcement Learning with Supervision by Combining Multiple Learnings and Expert Advices,
    in Proc. of the 25th American Control Conference, 2006, pp. 4159-4164.
20. H. S. Chang, On Policy Iteration for Finite Horizon Markov Decision Processes with Target-Level Risk Sensitive Objectives,
    in Proc. of the 17th Int. Symposium on Mathematical Theory of Networks and Systems (MTNS), 2006, pp. 1256-1258.
21. H. S. Chang, On Combining Multiple Heuristic Policies in Minimax Control,
    in Proc. of the 17th Int. Symposium on Mathematical Theory of Networks and Systems (MTNS), 2006, pp. 2738-2741.
22. D. Kim, S. Park, Y. Jin, H. S. Chang, Y. Park, I. Ko, K. Lee, J. Lee, Y. Park, and S. Lee, SHAGE: A Framework for Self-Managed Robot Software,
    in Proc. of the Int. Workshop on Self-Adaptation and Self-Managing systems, 2006, pp. 79-85.
23. H. S. Chang, M. Fu, and S. I. Marcus, Adversarial Multi-Armed Bandit Approach to Stochastic Optimization,
    in Proc. of the 45th IEEE Conf. on Decision and Control, 2006, pp. 5681-5686.
24. H. S. Chang, M. Fu, and S. I. Marcus, Adversarial Multi-Armed Bandit Approach to Two-Person Zero-Sum Markov Games,
    in Proc. of the 46th IEEE Conf. on Decision and Control, 2007, pp. 127-132.
25. J. Hu, H. S. Chang, M. Fu, and S. I. Marcus, Randomized Algorithms for Solving Markov Decision Processes,
    in Proc. of the 1st Int. Workshop in Sequential Methodologies, 2007.
26. H. S. Chang, Stochastic Iterative Approximation for Parallel Rollout and Policy Switching,
    in Proc. of the 17th Int. Federation of Automatic Control (IFAC) World Congress, 2008, pp. 15475-15479.
27. J. Hu and H. S. Chang, A Population-Based Cross-Entropy Method with Dynamic Sample Allocation,
    in Proc. of the 47th IEEE Conf. on Decision and Control, 2008, pp. 2426-2431.

• "Computational Intelligence Optimization: Particle Swarm Optimization and Ant Colony Optimization, A Gentle Introduction," Invited Talk for the Graduate Seminar, Department of Electronic and Electrical Engineering, POSTECH, 2005.9. talk slide
• "Reinforcement Learning with Supervision by Combining Multiple Learnings and Expert Advices," Invited Talk for the Control and Dynamical Systems Invited Lecture Series, Institute for Systems Research, University of Maryland at College Park, 2006. 6. 19.
• S. H. Jeon, H. S. Chang, M. Hur, K. H. Kwon, S. Kim, H. S. Kim, J. S. Yoo, S. Park, and H. B. Oh, "Speed Improvement of an FTICR Mass Spectra Analysis Program by Simple Modifications," The Bulletin of the Korean Chemical Society, Vol. 30, No. 9, 2009, pp. 2061-2065.
• H. S. Chang, "A Short Note on Combining Multiple Policies in Risk-Sensitive Exponential Average Reward Markov Decision Processes," TR2009-14, ISR, Univ. of Maryland.
• H. S. Chang, "On Convergence of Evolutionary Computation for Stochastic Combinatorial Optimization," TR2009-16, ISR, Univ. of Maryland.
• Member of the Program Committee, the 46th IEEE Conference on Decision and Control, 2007.
• Visiting Associate Professor, the Institute for Systems Research (ISR) and the Department of Electrical and Computer Engineering (ECE), Univ. of Maryland, 2009.
• Advances in Discrete Event, Nonlinear, and Stochastic Systems Modeling and Control: A Symposium in Honor of Steven I. Marcus, 2009. 4.
• Listed in the 9th, 10th Edition of Marquis Who's Who in Science and Engineering and in the 1st Edition of Marquis Who's Who in Asia.
• Listed in the 25th Silver, 26th, 27th/the 110th Anniversary Edition of Marquis Who's Who in the World/in America.
• Domestic papers
• Courses
• Some of my Favorites: Quotes || Rene Magritte || Sol LeWitt

Some Readable Articles

• Ten Simple Rules for Mathematical Writing by D. Bertsekas.
• The life and work of A. Markov || R. Bellman On the Birth of Dynamic Programming by S. Dreyfus.
• Comments on the Origin and Application of MDPs by R. Howard.
• On the Significance of Markov Decision Processes by R. Sutton.
• Creating a Mathematical Theory of Computer Networks by L. Kleinrock.
• The Commencement Address by Steve Jobs at Stanford delivered on June 12, 2005.