{"id":606577,"date":"2025-07-14T17:11:24","date_gmt":"2025-07-14T22:11:24","guid":{"rendered":"https:\/\/towardsdatascience.com\/?p=606577"},"modified":"2025-07-14T17:12:20","modified_gmt":"2025-07-14T22:12:20","slug":"dynamic-inventory-optimization-with-censored-demand-2","status":"publish","type":"post","link":"https:\/\/towardsdatascience.com\/dynamic-inventory-optimization-with-censored-demand-2\/","title":{"rendered":"Dynamic Inventory Optimization with Censored Demand"},"content":{"rendered":"\n

We often make decisions<\/mdspan> under uncertainty. Not just once, but in a sequence over time. We rely on our past experiences and expectations of the future to make the most informed and optimal choices possible.<\/p>\n\n\n\n

Think of a business that offers several products. These products are procured at a cost and sold for a profit. However, unsold inventory may incur a restocking fee, may carry salvage value, or in some cases, must be scrapped entirely.<\/p>\n\n\n\n

Businesses, therefore, faces a crucial question: how much to stock? This decision must often be made before demand is fully known; that is, under censored demand. If the business overstocks, it observes the full demand, since all customer requests are fulfilled. But if it understocks, it only sees that demand exceeded supply and the actual demand remains unknown, making it a censored observation.<\/p>\n\n\n\n

\"\"
Image Generated by Author via DALL-E<\/figcaption><\/figure>\n\n\n\n

This type of problem is often referred to as a Newsvendor Model<\/strong>. In fields such as operations research and applied mathematics, the optimal stocking decision has been studied by framing it as a classic newspaper stocking problem; hence the name.<\/p>\n\n\n\n

In this article, we explore a Sequential Decision-Making<\/strong> framework for the stocking problem under uncertainty and develop a dynamic optimization algorithm using Bayesian learning.<\/p>\n\n\n\n

Our approach closely follows the framework laid out by Warren B. Powell, Reinforcement Learning and Stochastic Optimization (2019)<\/a> and implements the paper from Negoescu, Powell, and Frazier (2011), Optimal Learning Policies for the Newsvendor Problem with Censored Demand and Unobservable Lost Sales<\/a>, published in Operations Research.<\/p>\n\n\n\n

Problem Setup<\/h2>\n\n\n\n

Following a similar setup to that of Negoescu et al., we frame the problem as optimizing the inventory level for a single item over a sequence of time steps. The cost and selling price are considered fixed. Unsold inventory is discarded with no salvage value, while each unit sold generates revenue. Demand is unknown, and when the available stock is less than actual demand, the demand observation is considered censored. <\/p>\n\n\n\n

Demand \\( W \\) in each period is drawn from an exponential distribution with an unknown rate parameter, for simulation purposes.<\/p>\n\n\n

\\[
\n\\begin{aligned}
\nx &\\in \\mathbb{R}_+ &&: \\text{Order quantity (decision variable)} \\\\
\nW &\\sim \\mathrm{Exponential}(\\lambda) &&: \\text{Random demand with unknown rate parameter } \\lambda \\\\
\n\\lambda &&&: \\text{Demand rate (unknown, to be estimated)} \\\\
\nc &&&: \\text{Unit cost to procure or produce the item} \\\\
\np &&&: \\text{Unit selling price (assume } p > c \\text{ for profitability)}
\n\\end{aligned}
\n\\]<\/p>\n\n\n\n

The parameter \\(\\lambda\\) in the exponential distribution represents the rate of demand<\/strong>; that is, how quickly demand events occur. The average demand<\/strong> is given by \\(\\mathbb{E}[W] = \\frac{1}{\\lambda}\\).<\/p>\n\n\n\n

\"\"
Image by Author<\/figcaption><\/figure>\n\n\n\n

We can observe from the Probability Density Function<\/strong> (PDF) of the Exponential distribution that higher values of demand \\(W\\) become less likely. Thus, the Exponential distribution serves as an appropriate choice for demand modeling.<\/p>\n\n\n\n

Sequential Decision Formulation<\/h2>\n\n\n\n

We formulate the inventory control problem as sequential decision process under uncertainty. The goal is to maximize total expected profit over a finite time horizon \\( N \\), while learning unknown demand rate by applying Bayesian Learning principals.<\/p>\n\n\n\n

We define a model with an initial state and a probabilistic model that represents its belief about future states over time. At each time step, the model makes a decision based on a policy that maps its current belief to an action. The goal is to find the optimal policy<\/strong> that maximizes a predefined reward function.<\/p>\n\n\n\n

After taking an action, the model observes the resulting state and updates its belief, accordingly, continuing this cycle of decision, observation, and belief update.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

1) State Variable<\/h3>\n\n\n\n

We model demand in each period as a random variable drawn from an Exponential distribution with an unknown rate parameter \u03bb. Since \\( \\lambda \\) is not directly observable, we encode our uncertainty about its value using a Gamma prior: <\/p>\n\n\n

\\[
\n\\lambda \\sim \\mathrm{Gamma}(a_0, b_0)
\n\\]<\/p>\n\n\n\n

The parameters \\( a_0 \\) and \\( b_0 \\) define the shape and rate of our initial belief about the demand rate. These two parameters serve as our state variables<\/strong>. At each time step, they summarize all past information and are updated as new demand observations become available.<\/p>\n\n\n\n

As we collect more data, the posterior distribution over \\(lambda\\) evolves from a wide and uncertain shape to a narrower and more confident one, gradually concentrating around the true demand rate. <\/p>\n\n\n\n

This process is captured naturally by the Gamma distribution, which flexibly adjusts its shape based on the amount of information we’ve seen. Early on, the distribution is diffuse, signaling high uncertainty. As observations accumulate, the belief becomes sharper, allowing for more reliable and responsive decision-making. Probability Density Function (PDF) of Gamma distribution can be seen below:<\/p>\n\n\n\n

\"\"
Image by Author<\/figcaption><\/figure>\n\n\n\n

We will later define a transition function<\/strong> that updates the state, that is, how \\( (a_n, b_n) \\) evolves to \\( (a_{n+1}, b_{n+1}) \\), based on newly observed data. This allows the model to continuously refine its belief about demand and make more informed inventory decisions over time.<\/p>\n\n\n\n

Note that expected value of the Gamma distribution is defined as:<\/p>\n\n\n

\\[
\n\\mathbb{E}[\\lambda] = \\frac{a}{b}
\n\\]<\/p>\n\n\n\n

2) Decision Variable<\/h3>\n\n\n\n

The decision variable<\/strong> at time \\( n \\) is the stocking level:<\/p>\n\n\n

\\[
\nx_n \\in \\mathbb{R}_+
\n\\]<\/p>\n\n\n\n

This is the number of units to order before demand \\( W_{n+1} \\) is realized. The decision depends only on the current belief \\( (a_n, b_n) \\).<\/p>\n\n\n\n

3) Exogenous Information<\/h3>\n\n\n\n

After selecting \\( x_n \\), demand \\( W_{n+1} \\) is revealed:<\/p>\n\n\n

\\[
\nW_{n+1} \\sim \\text{Exp}(\\lambda)
\n\\]<\/p>\n\n\n\n

Since \\( \\lambda \\) is unknown, demand is random. Observations are:<\/p>\n\n\n\n