{"id":3545,"date":"2024-08-16T14:17:21","date_gmt":"2024-08-16T14:17:21","guid":{"rendered":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/"},"modified":"2025-01-08T15:35:31","modified_gmt":"2025-01-08T15:35:31","slug":"optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878","status":"publish","type":"post","link":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/","title":{"rendered":"Optimizing Marketing Campaigns with Budgeted Multi-Armed Bandits"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Let&#8217;s dive right into a running example. Suppose a bank or a telecom company launches a new product\/plan for existing customers. To promote this product, the company creates several call templates (scripts) for its sales representatives. The goal is to effectively convince customers to buy the new product or sign up for the new plan.<\/p>\n<p class=\"wp-block-paragraph\">Here&#8217;s how the campaign works:<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>Call script creation:<\/strong> The marketing team develops multiple versions of the call script, each with a different approach to promoting the new product or plan.<\/li>\n<li><strong>Agent calls:<\/strong> Sales agents use these scripts to call a subset of customers. Each customer interaction uses one of the predefined scripts.<\/li>\n<li><strong>Data Collection:<\/strong> During the calls, the company collects data on customer responses, such as interest shown, questions asked, and, ultimately conversion rates (i.e., how many customers buy the new product or sign up for the new plan).<\/li>\n<li><strong>Real-time analysis:<\/strong> The company analyzes this data in real time to evaluate the effectiveness of each script. This analysis helps determine which scripts are more successful in converting customers to the new plan\/product.<\/li>\n<li><strong>Strategy Update:<\/strong> Based on ongoing analysis, the company dynamically adjusts the frequency of use of each script. Scripts with higher conversion rates are used more often, ensuring that the campaign becomes increasingly effective over time.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">Next, we show how to model a simple version of this campaign using the conventional multi-armed bandit problem. As we add more details to make this model more realistic, we demonstrate how existing solutions and their simple adaptations fall short. We then present a new budgeted multi-armed bandit algorithm from our paper accepted for the KDD 2024 conference that performs very well at this task. We also provide links to the code and a short video summarizing the paper.<\/p>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>In this story, &quot;we&quot; is used because I am writing it together with Marco Heyden (<a href=\"https:\/\/www.linkedin.com\/in\/marco-heyden-036375156\/\">Linkedin<\/a>, <a href=\"https:\/\/github.com\/heymarco\">Github<\/a>), the author of the algorithm idea and of our paper [1]. All subsequent plots are created by us and with the code from this <a href=\"https:\/\/github.com\/heymarco\/OmegaUCB\/blob\/medium_post\/MediumGraphs.ipynb\">Jupyter notebook<\/a>.<\/p><\/blockquote>\n<h3 class=\"wp-block-heading\">Multi-armed Bandit Solution<\/h3>\n<p class=\"wp-block-paragraph\">Our scenario is similar to a multi-armed bandit (MAB) problem. Imagine a player in a casino facing a slot machine (&quot;bandit&quot;) with multiple arms, each with an unknown payoff (reward) distribution. The player&#8217;s goal is to maximize their total winnings by deciding which arms to play and how often to play each one. The challenge lies in balancing <strong>exploration<\/strong>, trying different arms to gather information about their rewards, and <strong>exploitation<\/strong>, using the information gathered to play the arm with the highest known reward.<\/p>\n<p class=\"wp-block-paragraph\">In our example, each call script acts like a slot machine arm, where the reward is the success of the script. The reward is 1 if the customer signs up for the new plan or buys the new product and 0 otherwise. For instance, three call scripts with conversion rates of <em>0.1<\/em>, <em>0.3<\/em>, and <em>0.7<\/em> have successes following a Bernoulli distribution with expected values of <em>0.1<\/em>, <em>0.3<\/em>, and <em>0.7<\/em>, respectively. The figure below illustrates the cumulative rewards for different strategies. The purple line represents using the <strong>script 1<\/strong> with a conversion rate of <em>0.1<\/em>, while the green line represents using the <strong>script 3<\/strong> with a conversion rate of <em>0.7<\/em>. These lines define the range of probable rewards. The light blue line shows the cumulative reward for a strategy that randomly selects a script for each call. In a realistic environment where only <strong>estimates<\/strong> of conversion rates are available, the cumulative reward of a good strategy should be close to the green line and at least above the light blue line.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"1b1d1c\" data-has-transparency=\"true\" style=\"--dominant-color: #1b1d1c;\" loading=\"lazy\" decoding=\"async\" width=\"1000\" height=\"500\" class=\"wp-image-314223 has-transparency\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1LjL3P0oZR3re7Wghy6-6eA-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1LjL3P0oZR3re7Wghy6-6eA-1.png 1000w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1LjL3P0oZR3re7Wghy6-6eA-1-300x150.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1LjL3P0oZR3re7Wghy6-6eA-1-768x384.png 768w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><\/figure>\n<p class=\"wp-block-paragraph\">A popular strategy for the multi-armed bandit problem is the Upper Confidence Bound (UCB) algorithm [2]. It assigns an upper confidence bound to the expected reward of each arm (call script) and plays the arm with the highest upper confidence bound. In this way, the algorithm actively explores actions with high uncertainty while exploiting actions with high known rewards. Mathematically, the UCB algorithm plays the arm <em>i,<\/em> which solves<\/p>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"f1f1f1\" data-has-transparency=\"true\" style=\"--dominant-color: #f1f1f1;\" loading=\"lazy\" decoding=\"async\" width=\"1444\" height=\"155\" class=\"wp-image-314224 has-transparency\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1hfPmXvXm9Kks7O3ilCisEg-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1hfPmXvXm9Kks7O3ilCisEg-1.png 1444w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1hfPmXvXm9Kks7O3ilCisEg-1-300x32.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1hfPmXvXm9Kks7O3ilCisEg-1-1024x110.png 1024w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1hfPmXvXm9Kks7O3ilCisEg-1-768x82.png 768w\" sizes=\"auto, (max-width: 1444px) 100vw, 1444px\" \/><\/figure>\n<ul class=\"wp-block-list\">\n<li><em>r\u1d62(t)<\/em> is the <strong>empirical<\/strong> mean reward of arm <em>i<\/em> up to time <em>t<\/em>.<\/li>\n<li><em>N\u1d62(t)<\/em> is the number of times arm <em>i<\/em> has been played up to time <em>t<\/em>.<\/li>\n<li><em>t<\/em> is the total number of plays so far.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">The white line in the figure below shows this strategy in action for our example.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"1c1e1d\" data-has-transparency=\"true\" style=\"--dominant-color: #1c1e1d;\" loading=\"lazy\" decoding=\"async\" width=\"1000\" height=\"500\" class=\"wp-image-314225 has-transparency\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/13bvdV0C6nX84cGm5z_LUVQ-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/13bvdV0C6nX84cGm5z_LUVQ-1.png 1000w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/13bvdV0C6nX84cGm5z_LUVQ-1-300x150.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/13bvdV0C6nX84cGm5z_LUVQ-1-768x384.png 768w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><\/figure>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>This upper bound is based on the Chernoff-Hoeffding bounds assuming a payoff distribution with support in [0,1], which is exactly our case. For reward distributions with different support [a_\u1d62_\u02b3,b_\u1d62_\u02b3], where a\u1d62\u02b3 and a_\u1d62_\u02b3 are finite, the UCB should be scaled accordingly:<\/p><\/blockquote>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"f6f6f6\" data-has-transparency=\"true\" style=\"--dominant-color: #f6f6f6;\" loading=\"lazy\" decoding=\"async\" width=\"1404\" height=\"142\" class=\"wp-image-314226 has-transparency\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1k303MnSEvtkzOq_9jFmTbg-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1k303MnSEvtkzOq_9jFmTbg-1.png 1404w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1k303MnSEvtkzOq_9jFmTbg-1-300x30.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1k303MnSEvtkzOq_9jFmTbg-1-1024x104.png 1024w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1k303MnSEvtkzOq_9jFmTbg-1-768x78.png 768w\" sizes=\"auto, (max-width: 1404px) 100vw, 1404px\" \/><\/figure>\n<h3 class=\"wp-block-heading\">The Importance of the Budget<\/h3>\n<p class=\"wp-block-paragraph\">So far we have focused on maximizing the sum of rewards after a given number of calls. However, it is unrealistic to expect calls corresponding to different scripts to have the same duration. If the marketing team&#8217;s capacity does not allow them to reach all customers within a given time budget (e.g., several months), it would be more practical to maximize the cumulative reward for a given call duration rather than the number of calls.<\/p>\n<p class=\"wp-block-paragraph\">In our example, assume that the call duration (=costs) for <strong>scripts 1<\/strong>, <strong>2<\/strong>, and <strong>3<\/strong> are constant (we will relax this assumption later) and equal <em>1<\/em>, <em>2<\/em>, and <em>8<\/em> minutes, respectively. If we now plot the results based on the total call duration instead of the number of calls, the strategy that always uses <strong>script 2<\/strong> becomes the best choice, while the strategy that uses <strong>script 3<\/strong> becomes the worst. The above UCB strategy that only considers conversion rates now performs much worse than a random strategy.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"111313\" data-has-transparency=\"true\" style=\"--dominant-color: #111313;\" loading=\"lazy\" decoding=\"async\" width=\"1000\" height=\"500\" class=\"wp-image-314227 has-transparency\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1cYiDpk0s9QrUFv_5VqkglQ-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1cYiDpk0s9QrUFv_5VqkglQ-1.png 1000w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1cYiDpk0s9QrUFv_5VqkglQ-1-300x150.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1cYiDpk0s9QrUFv_5VqkglQ-1-768x384.png 768w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><\/figure>\n<p class=\"wp-block-paragraph\">It is not difficult to cure the UCB strategy by normalizing reward estimates <em>r\u1d62(t)<\/em> wit<em>h c\u1d62<\/em> counterparts and adjusting the UCB as in formula (2) above:<\/p>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"f6f6f6\" data-has-transparency=\"true\" style=\"--dominant-color: #f6f6f6;\" loading=\"lazy\" decoding=\"async\" width=\"1403\" height=\"143\" class=\"wp-image-314228 has-transparency\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1xe-lor5QR45Z9pmLkm1_jw-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1xe-lor5QR45Z9pmLkm1_jw-1.png 1403w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1xe-lor5QR45Z9pmLkm1_jw-1-300x31.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1xe-lor5QR45Z9pmLkm1_jw-1-1024x104.png 1024w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1xe-lor5QR45Z9pmLkm1_jw-1-768x78.png 768w\" sizes=\"auto, (max-width: 1403px) 100vw, 1403px\" \/><\/figure>\n<p class=\"wp-block-paragraph\">The UCB strategy that was updated in this way is working quite well again:<\/p>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"111413\" data-has-transparency=\"true\" style=\"--dominant-color: #111413;\" loading=\"lazy\" decoding=\"async\" width=\"1000\" height=\"500\" class=\"wp-image-314229 has-transparency\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1rxOe7afyLwLEWZoydWvADA-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1rxOe7afyLwLEWZoydWvADA-1.png 1000w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1rxOe7afyLwLEWZoydWvADA-1-300x150.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1rxOe7afyLwLEWZoydWvADA-1-768x384.png 768w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><\/figure>\n<h3 class=\"wp-block-heading\">Random Call Duration<\/h3>\n<p class=\"wp-block-paragraph\">Observe that the assumption of a fixed call duration is also unrealistic. When both reward and cost are not fixed, there are several ways to extend the UCB strategy to this case, such as<\/p>\n<p class=\"wp-block-paragraph\"><strong>Reductive:<\/strong> By treating the reward to cost ratio <em>v\u1d62=r\u1d62\/c\u1d62<\/em> as a single random variable and pulling the arm with the highest upper bound UCB<em>\u1d62\u1d5b<\/em> of it:<\/p>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"f6f6f6\" data-has-transparency=\"true\" style=\"--dominant-color: #f6f6f6;\" loading=\"lazy\" decoding=\"async\" width=\"1407\" height=\"143\" class=\"wp-image-314230 has-transparency\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1ac5DIY_9ZBL7SmHlWiVB6g-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1ac5DIY_9ZBL7SmHlWiVB6g-1.png 1407w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1ac5DIY_9ZBL7SmHlWiVB6g-1-300x30.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1ac5DIY_9ZBL7SmHlWiVB6g-1-1024x104.png 1024w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1ac5DIY_9ZBL7SmHlWiVB6g-1-768x78.png 768w\" sizes=\"auto, (max-width: 1407px) 100vw, 1407px\" \/><\/figure>\n<p class=\"wp-block-paragraph\"><strong>United:<\/strong> By ignoring the variation in costs and using their estimates <em>c\u1d62(t)<\/em> to scale the UCB<em>\u1d62\u02b3<\/em> of rewards similarly to formula (3):<\/p>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"f5f5f5\" data-has-transparency=\"true\" style=\"--dominant-color: #f5f5f5;\" loading=\"lazy\" decoding=\"async\" width=\"1406\" height=\"143\" class=\"wp-image-314231 has-transparency\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1pzb92QcPFep9mDK6HE-g5Q-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1pzb92QcPFep9mDK6HE-g5Q-1.png 1406w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1pzb92QcPFep9mDK6HE-g5Q-1-300x31.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1pzb92QcPFep9mDK6HE-g5Q-1-1024x104.png 1024w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1pzb92QcPFep9mDK6HE-g5Q-1-768x78.png 768w\" sizes=\"auto, (max-width: 1406px) 100vw, 1406px\" \/><\/figure>\n<p class=\"wp-block-paragraph\"><strong>Composite:<\/strong> By pulling the arm that maximizes the ratio UCB_\u1d62\u02b3\/_LCB<em>\u1d62\u1d9c<\/em> of the upper reward bound to the lower cost bound:<\/p>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"f6f6f6\" data-has-transparency=\"true\" style=\"--dominant-color: #f6f6f6;\" loading=\"lazy\" decoding=\"async\" width=\"1405\" height=\"193\" class=\"wp-image-314232 has-transparency\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1Hq9AUaWKeiUKvibz8cCz0g-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1Hq9AUaWKeiUKvibz8cCz0g-1.png 1405w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1Hq9AUaWKeiUKvibz8cCz0g-1-300x41.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1Hq9AUaWKeiUKvibz8cCz0g-1-1024x141.png 1024w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1Hq9AUaWKeiUKvibz8cCz0g-1-768x105.png 768w\" sizes=\"auto, (max-width: 1405px) 100vw, 1405px\" \/><\/figure>\n<p class=\"wp-block-paragraph\">In (6) we assume that the rewards come from <em>[0,1]<\/em>, as before, to keep the formula a bit simpler.<\/p>\n<p class=\"wp-block-paragraph\">All the above strategies have problems, being either excessively optimistic, too pessimistic, or just optimizing the wrong quantity.<\/p>\n<p class=\"wp-block-paragraph\">The <strong>Reductive<\/strong> strategy (4) **** seeks to maximize the expectation of the reward-cost ratio \ud835\udd3c(<em>v\u1d62<\/em>)=\ud835\udd3c(r<em>\u1d62\/c\u1d62)<\/em>. This is different from the campaign goal of maximizing reward while keeping costs within a given budget. For sufficiently high budgets, the latter is equivalent to maximizing the ratio of reward and cost expectations \ud835\udd3c(r\u1d62_)\/<em>\ud835\udd3c<\/em>(<em>c\u1d62)<\/em>._ To see why \ud835\udd3c(r\u1d62\/c_\u1d62)\u2260\ud835\udd3c(_r<em>\u1d62<\/em>)\/\ud835\udd3c_(c<em>\u1d62<\/em>)_, o_bs_erve that if both r\u1d62 and _c_\u1d62 are <em>i<\/em>.i.d. Bernoulli distributed variables, then \ud835\udd3c(r\u1d62)\/\ud835\udd3c(c<em>\u1d62)<\/em>=<em>1<\/em>, w_hi_l<em>e<\/em> \ud835\udd3c(r\u1d62\/c\u1d62) is _infi_nite or undefined, depending on how you resolve the division by zero. The support of v\u1d62=r\u1d62\/c\u1d62 <em>is also<\/em> infinite in this case, making the UCB formula (4) useless.<\/p>\n<p class=\"wp-block-paragraph\">Because the <strong>United<\/strong> strategy does not account for cost variation, it often produces upper confidence bounds that are too tight, limiting the exploration part of the strategy. This happens whenever the empirical mean of the costs exceeds the true mean, e.g. in 50% of the cases with a symmetric cost distribution.<\/p>\n<p class=\"wp-block-paragraph\">The <strong>Composite<\/strong> strategy explicitly models uncertainty in costs. However, this model is too conservative, allowing the denominator to be zero or even negative(!). As a result, the composite strategy spends too many resources on exploration and undervalues the exploitation step.<\/p>\n<p class=\"wp-block-paragraph\">Note also that the upper bound for the mean reward discussed so far can be well above 1, even though we know that the reward takes values in <em>{0,1}<\/em>. In such a case, this bound is obviously too loose, which may increase exploration and thus partially offset the effect of considering the costs fixed in the <strong>United<\/strong> strategy.<\/p>\n<p class=\"wp-block-paragraph\">The following plot shows the performance of all three strategies for our example setting, where we now model the call duration with beta distributions with support <em>[1,10]<\/em> and expected values of <em>1<\/em>, <em>2<\/em>, and <em>8<\/em> minutes for <strong>scripts 1<\/strong>, <strong>2<\/strong>, and <strong>3<\/strong>, respectively. The <strong>Reductive<\/strong> strategy performs almost as poorly as selecting a random script, the <strong>Composite<\/strong> strategy performs slightly better, and the <strong>United<\/strong> strategy emerges as the clear winner. But can one beat the <strong>United<\/strong> strategy?<\/p>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"131515\" data-has-transparency=\"true\" style=\"--dominant-color: #131515;\" loading=\"lazy\" decoding=\"async\" width=\"1000\" height=\"500\" class=\"wp-image-314233 has-transparency\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1eDbYFlOrAe1WCp260w-4RQ-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1eDbYFlOrAe1WCp260w-4RQ-1.png 1000w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1eDbYFlOrAe1WCp260w-4RQ-1-300x150.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1eDbYFlOrAe1WCp260w-4RQ-1-768x384.png 768w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><\/figure>\n<h3 class=\"wp-block-heading\">Our Solution: Asymmetric Confidence Intervals<\/h3>\n<p class=\"wp-block-paragraph\">Our \u03c9-UCB strategy addresses the shortcomings of the previously described solutions. While we start with the same ratio of UCB<em>\u1d62\u02b3<\/em>\/LCB<em>\u1d62\u1d9c<\/em> as the composite strategy, we use a different method to compute these bounds. Our confidence intervals are more precise and remain within the support of the rewards or costs. Specifically, let <em>p<\/em> be a bounded random variable &#8211; either a reward or a cost &#8211; with support <em>[a,b]<\/em>. We compute the confidence interval _[_LCB<em>\u1d56<\/em>, UCB<em>\u1d56]<\/em> for <em>p<\/em> as follows.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"f1f1f1\" data-has-transparency=\"true\" style=\"--dominant-color: #f1f1f1;\" loading=\"lazy\" decoding=\"async\" width=\"1408\" height=\"117\" class=\"wp-image-314234 has-transparency\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1Hckt8wOT6r73-HVv7edhqA-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1Hckt8wOT6r73-HVv7edhqA-1.png 1408w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1Hckt8wOT6r73-HVv7edhqA-1-300x25.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1Hckt8wOT6r73-HVv7edhqA-1-1024x85.png 1024w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1Hckt8wOT6r73-HVv7edhqA-1-768x64.png 768w\" sizes=\"auto, (max-width: 1408px) 100vw, 1408px\" \/><\/figure>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"ebebeb\" data-has-transparency=\"false\" style=\"--dominant-color: #ebebeb;\" loading=\"lazy\" decoding=\"async\" width=\"1406\" height=\"64\" class=\"wp-image-314235 not-transparent\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1M8N_VTXZdDl1HMjkBbcBMA-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1M8N_VTXZdDl1HMjkBbcBMA-1.png 1406w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1M8N_VTXZdDl1HMjkBbcBMA-1-300x14.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1M8N_VTXZdDl1HMjkBbcBMA-1-1024x47.png 1024w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1M8N_VTXZdDl1HMjkBbcBMA-1-768x35.png 768w\" sizes=\"auto, (max-width: 1406px) 100vw, 1406px\" \/><\/figure>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"f1f1f1\" data-has-transparency=\"false\" style=\"--dominant-color: #f1f1f1;\" loading=\"lazy\" decoding=\"async\" width=\"1405\" height=\"150\" class=\"wp-image-314236 not-transparent\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/16SbygIqPvPLWnnnyznQtdw-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/16SbygIqPvPLWnnnyznQtdw-1.png 1405w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/16SbygIqPvPLWnnnyznQtdw-1-300x32.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/16SbygIqPvPLWnnnyznQtdw-1-1024x109.png 1024w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/16SbygIqPvPLWnnnyznQtdw-1-768x82.png 768w\" sizes=\"auto, (max-width: 1405px) 100vw, 1405px\" \/><\/figure>\n<ul class=\"wp-block-list\">\n<li><em>\u03bc<\/em> is the empirical mean of <em>p<\/em> at time <em>t<\/em>,<\/li>\n<li><em>\u03c3\u00b2<\/em> is the empirical variance of <em>p<\/em> at time <em>t<\/em>,<\/li>\n<li><em>N<\/em> is the number of observations of <em>p<\/em> up to time <em>t<\/em>,<\/li>\n<li><em>t<\/em> is the current time,<\/li>\n<li><em>c<\/em> is the number of arm pulls after which one expects reliable estimates of mean and variance; in practice, <em>c=30<\/em> works well,<\/li>\n<li><em>\u03c1<\/em> is the parameter of \u03c9-UCB; <em>\u03c1=1<\/em> gives better asymptotic properties of \u03c9-UCB, but for practical use, when the number of games is <em>~10\u2074<\/em> or less, we recommend <em>\u03c1=1\/4.<\/em><\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">The following figure shows how good the \u03c9-UCB is. Using it provides almost the maximum possible cumulative reward.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-dominant-color=\"131514\" data-has-transparency=\"true\" style=\"--dominant-color: #131514;\" loading=\"lazy\" decoding=\"async\" width=\"1000\" height=\"500\" class=\"wp-image-314237 has-transparency\" src=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1qRgWQq3IRIRBIivfmBcsiQ-1.png\" alt=\"\" srcset=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1qRgWQq3IRIRBIivfmBcsiQ-1.png 1000w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1qRgWQq3IRIRBIivfmBcsiQ-1-300x150.png 300w, https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1qRgWQq3IRIRBIivfmBcsiQ-1-768x384.png 768w\" sizes=\"auto, (max-width: 1000px) 100vw, 1000px\" \/><\/figure>\n<p class=\"wp-block-paragraph\">We also created a 2-minute video summarizing the idea of the \u03c9-UCB:<\/p>\n<iframe loading=\"lazy\" class=\"wp-block-html\" src=\"https:\/\/cdn.embedly.com\/widgets\/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FtRQ1-DiXIME%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DtRQ1-DiXIME&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FtRQ1-DiXIME%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube\" height=\"480\" width=\"854\"><\/iframe>\n<h3 class=\"wp-block-heading\">Final Thoughts<\/h3>\n<p class=\"wp-block-paragraph\">By now, you are well equipped with insights on how to optimize a marketing campaign in real time using instant customer feedback. We have described a powerful algorithm to help you do this. However, it would be overly optimistic to assume that this alone will guarantee success. Below, we outline a few additional considerations that can further enhance a campaign&#8217;s effectiveness.<\/p>\n<p class=\"wp-block-paragraph\">First, it is unlikely that the reward will be known immediately. Often, the best you can hope for is an indication of interest from the customer. Therefore, constructing a reliable proxy for the reward, perhaps using data from previous campaigns, is essential.<\/p>\n<p class=\"wp-block-paragraph\">Next, this discussion has focused on selecting the best script for an average or representative customer. Digging a little deeper, it is likely that different scripts will work better for different groups of customers. The best script may vary by group. A simple approach would be to segment customers and treat each segment-script combination as a separate arm in the Budgeted MAB algorithm we described. In an earlier story, I discussed a method for identifying interesting customer segments; for a campaign, selecting an appropriate target variable for this method will be important.<\/p>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p><a href=\"https:\/\/towardsdatascience.com\/find-unusual-segments-in-your-data-with-subgroup-discovery-2661a586e60c\"><strong>Find Unusual Segments in Your Data with Subgroup Discovery<\/strong><\/a><\/p><\/blockquote>\n<p class=\"wp-block-paragraph\">Finally, in addition to customer characteristics, &quot;environmental&quot; factors such as time of day or day of week can affect the relative performance of scripts. To account for all of these factors, you might consider extending the methodology to <strong>contextual budgeted bandits<\/strong>, which is the subject of a separate story.<\/p>\n<h3 class=\"wp-block-heading\">References<\/h3>\n<p class=\"wp-block-paragraph\">[1] Marco Heyden, Vadim Arzamasov, Edouard Fouch\u00e9, and Klemens B\u00f6hm. &quot;Budgeted Multi-Armed Bandits with Asymmetric Confidence Intervals.&quot; <em>KDD<\/em> &#8217;24<\/p>\n<p class=\"wp-block-paragraph\">[2] Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. &quot;Finite-time analysis of the multiarmed bandit problem.&quot; <em>Machine learning<\/em> 47 (2002): 235\u2013256.<\/p>","protected":false},"excerpt":{"rendered":"<p>With demos, our new solution, and a video<\/p>\n","protected":false},"author":18,"featured_media":3546,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"is_member_only":false,"sub_heading":"With demos, our new solution, and a video","footnotes":""},"categories":[44,22],"tags":[448,453,446,495,496],"sponsor":[],"coauthors":[32124],"class_list":["post-3545","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-science","category-machine-learning","tag-data-science","tag-editors-pick","tag-machine-learning","tag-marketing-campaign","tag-multi-armed-bandit"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Optimizing Marketing Campaigns with Budgeted Multi-Armed Bandits | Towards Data Science<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Optimizing Marketing Campaigns with Budgeted Multi-Armed Bandits | Towards Data Science\" \/>\n<meta property=\"og:description\" content=\"With demos, our new solution, and a video\" \/>\n<meta property=\"og:url\" content=\"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/\" \/>\n<meta property=\"og:site_name\" content=\"Towards Data Science\" \/>\n<meta property=\"article:published_time\" content=\"2024-08-16T14:17:21+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-01-08T15:35:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1vilI3Q4nlwqsAQLq3TOzSA.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1792\" \/>\n\t<meta property=\"og:image:height\" content=\"1024\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Vadim Arzamasov\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@TDataScience\" \/>\n<meta name=\"twitter:site\" content=\"@TDataScience\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Vadim Arzamasov\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/\"},\"author\":{\"name\":\"TDS Editors\",\"@id\":\"https:\/\/towardsdatascience.com\/#\/schema\/person\/f9925d336b6fe962b03ad8281d90b8ee\"},\"headline\":\"Optimizing Marketing Campaigns with Budgeted Multi-Armed Bandits\",\"datePublished\":\"2024-08-16T14:17:21+00:00\",\"dateModified\":\"2025-01-08T15:35:31+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/\"},\"wordCount\":1925,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/towardsdatascience.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1vilI3Q4nlwqsAQLq3TOzSA.jpg\",\"keywords\":[\"Data Science\",\"Editors Pick\",\"Machine Learning\",\"Marketing Campaign\",\"Multi Armed Bandit\"],\"articleSection\":[\"Data Science\",\"Machine Learning\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/\",\"url\":\"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/\",\"name\":\"Optimizing Marketing Campaigns with Budgeted Multi-Armed Bandits | Towards Data Science\",\"isPartOf\":{\"@id\":\"https:\/\/towardsdatascience.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1vilI3Q4nlwqsAQLq3TOzSA.jpg\",\"datePublished\":\"2024-08-16T14:17:21+00:00\",\"dateModified\":\"2025-01-08T15:35:31+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/#primaryimage\",\"url\":\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1vilI3Q4nlwqsAQLq3TOzSA.jpg\",\"contentUrl\":\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1vilI3Q4nlwqsAQLq3TOzSA.jpg\",\"width\":1792,\"height\":1024,\"caption\":\"Image created by authors with GPT-4o\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/towardsdatascience.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Optimizing Marketing Campaigns with Budgeted Multi-Armed Bandits\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/towardsdatascience.com\/#website\",\"url\":\"https:\/\/towardsdatascience.com\/\",\"name\":\"Towards Data Science\",\"description\":\"Publish AI, ML &amp; data-science insights to a global community of data professionals.\",\"publisher\":{\"@id\":\"https:\/\/towardsdatascience.com\/#organization\"},\"alternateName\":\"TDS\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/towardsdatascience.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/towardsdatascience.com\/#organization\",\"name\":\"Towards Data Science\",\"alternateName\":\"TDS\",\"url\":\"https:\/\/towardsdatascience.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/towardsdatascience.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/tds-logo.jpg\",\"contentUrl\":\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/tds-logo.jpg\",\"width\":696,\"height\":696,\"caption\":\"Towards Data Science\"},\"image\":{\"@id\":\"https:\/\/towardsdatascience.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/TDataScience\",\"https:\/\/www.youtube.com\/c\/TowardsDataScience\",\"https:\/\/www.linkedin.com\/company\/towards-data-science\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/towardsdatascience.com\/#\/schema\/person\/f9925d336b6fe962b03ad8281d90b8ee\",\"name\":\"TDS Editors\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/towardsdatascience.com\/#\/schema\/person\/image\/23494c9101089ad44ae88ce9d2f56aac\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g\",\"caption\":\"TDS Editors\"},\"description\":\"Building a vibrant data science and machine learning community. Share your insights and projects with our global audience: bit.ly\/write-for-tds\",\"url\":\"https:\/\/towardsdatascience.com\/author\/towardsdatascience\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Optimizing Marketing Campaigns with Budgeted Multi-Armed Bandits | Towards Data Science","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/","og_locale":"en_US","og_type":"article","og_title":"Optimizing Marketing Campaigns with Budgeted Multi-Armed Bandits | Towards Data Science","og_description":"With demos, our new solution, and a video","og_url":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/","og_site_name":"Towards Data Science","article_published_time":"2024-08-16T14:17:21+00:00","article_modified_time":"2025-01-08T15:35:31+00:00","og_image":[{"width":1792,"height":1024,"url":"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1vilI3Q4nlwqsAQLq3TOzSA.jpg","type":"image\/jpeg"}],"author":"Vadim Arzamasov","twitter_card":"summary_large_image","twitter_creator":"@TDataScience","twitter_site":"@TDataScience","twitter_misc":{"Written by":"Vadim Arzamasov","Est. reading time":"10 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/#article","isPartOf":{"@id":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/"},"author":{"name":"TDS Editors","@id":"https:\/\/towardsdatascience.com\/#\/schema\/person\/f9925d336b6fe962b03ad8281d90b8ee"},"headline":"Optimizing Marketing Campaigns with Budgeted Multi-Armed Bandits","datePublished":"2024-08-16T14:17:21+00:00","dateModified":"2025-01-08T15:35:31+00:00","mainEntityOfPage":{"@id":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/"},"wordCount":1925,"commentCount":0,"publisher":{"@id":"https:\/\/towardsdatascience.com\/#organization"},"image":{"@id":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/#primaryimage"},"thumbnailUrl":"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1vilI3Q4nlwqsAQLq3TOzSA.jpg","keywords":["Data Science","Editors Pick","Machine Learning","Marketing Campaign","Multi Armed Bandit"],"articleSection":["Data Science","Machine Learning"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/","url":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/","name":"Optimizing Marketing Campaigns with Budgeted Multi-Armed Bandits | Towards Data Science","isPartOf":{"@id":"https:\/\/towardsdatascience.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/#primaryimage"},"image":{"@id":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/#primaryimage"},"thumbnailUrl":"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1vilI3Q4nlwqsAQLq3TOzSA.jpg","datePublished":"2024-08-16T14:17:21+00:00","dateModified":"2025-01-08T15:35:31+00:00","breadcrumb":{"@id":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/#primaryimage","url":"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1vilI3Q4nlwqsAQLq3TOzSA.jpg","contentUrl":"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2024\/08\/1vilI3Q4nlwqsAQLq3TOzSA.jpg","width":1792,"height":1024,"caption":"Image created by authors with GPT-4o"},{"@type":"BreadcrumbList","@id":"https:\/\/towardsdatascience.com\/optimizing-marketing-campaigns-with-budgeted-multi-armed-bandits-a65fccd61878\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/towardsdatascience.com\/"},{"@type":"ListItem","position":2,"name":"Optimizing Marketing Campaigns with Budgeted Multi-Armed Bandits"}]},{"@type":"WebSite","@id":"https:\/\/towardsdatascience.com\/#website","url":"https:\/\/towardsdatascience.com\/","name":"Towards Data Science","description":"Publish AI, ML &amp; data-science insights to a global community of data professionals.","publisher":{"@id":"https:\/\/towardsdatascience.com\/#organization"},"alternateName":"TDS","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/towardsdatascience.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/towardsdatascience.com\/#organization","name":"Towards Data Science","alternateName":"TDS","url":"https:\/\/towardsdatascience.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/towardsdatascience.com\/#\/schema\/logo\/image\/","url":"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/tds-logo.jpg","contentUrl":"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/tds-logo.jpg","width":696,"height":696,"caption":"Towards Data Science"},"image":{"@id":"https:\/\/towardsdatascience.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/TDataScience","https:\/\/www.youtube.com\/c\/TowardsDataScience","https:\/\/www.linkedin.com\/company\/towards-data-science\/"]},{"@type":"Person","@id":"https:\/\/towardsdatascience.com\/#\/schema\/person\/f9925d336b6fe962b03ad8281d90b8ee","name":"TDS Editors","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/towardsdatascience.com\/#\/schema\/person\/image\/23494c9101089ad44ae88ce9d2f56aac","url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","caption":"TDS Editors"},"description":"Building a vibrant data science and machine learning community. Share your insights and projects with our global audience: bit.ly\/write-for-tds","url":"https:\/\/towardsdatascience.com\/author\/towardsdatascience\/"}]}},"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Towards Data Science","distributor_original_site_url":"https:\/\/towardsdatascience.com","push-errors":false,"_links":{"self":[{"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/posts\/3545","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/users\/18"}],"replies":[{"embeddable":true,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/comments?post=3545"}],"version-history":[{"count":0,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/posts\/3545\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/media\/3546"}],"wp:attachment":[{"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/media?parent=3545"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/categories?post=3545"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/tags?post=3545"},{"taxonomy":"sponsor","embeddable":true,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/sponsor?post=3545"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/coauthors?post=3545"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}