Control Policies for a Water-Treatment System Using the Markov Decision Process. Part 2: Simulation and Analysis
Abstract:This article is the second part of a two-part research study of a theoretical water-treatment system based on the NASA Baseline Values and Assumptions Document (BVAD) . It focuses on the decision-making model created to choose the "best" policy to be applied to the water-treatment system based on hourly system conditions. Due to the resemblance between the behavior of this system and the Markovian process, this system is constructed based on the Markovian model. The water system consists of two subsystems: a hygiene-water subsystem that supplies water for laundry, urinal flush, dish wash, oral hygiene and shower; and a potable-water subsystem that supplies water for drinking and food rehydration. In order to assess the conditions of the water system, various aspects of the system, such as hourly and accumulated water deficiency, and amount of clean water available for use, are captured on an hourly basis. A baseline policy and policies derived from it are tested to find the best policy for the system to operate under the most economical conditions while providing enough clean water for crew consumption. The best policy is obtained through various mathematical modeling techniques. Outcomes are compared against a system that uses the baseline policy. Results show that an intuitively "good" policy may not always be the best policy for the system. The system performance is measured in terms of a reward value, which is assigned based on the system conditions.
Document Type: Research Article
Publication date: June 1, 2009
More about this publication?
- Habitation, International Journal for Human Support Research, is designed to meet the needs of an emerging field of study necessitated by the need to develop new technologies to support human activities within controlled environments.