Budget Reallocation Strategies for Programmatic Advertising Using Reinforcement Learning and Historical ROAS Signals

Abstract

Programmatic advertising campaigns need budget changes because of performance and consumer actions. Usual methods with set budgets plus fixed rules do not work when audiences or competition change. This paper describes a reinforcement learning framework that reallocates budget to different times of the week and day. The framework uses historical Return on Ad Spend but also sales lift data for optimization.
The system uses a Deep Q-Network (DQN) with several hidden layers and experience replay for budget work. This study employs a DQN model, trained on e-commerce campaign data. The data held $1.5 million in advertising spend and 1.2 million impressions across 12 months. The RL agent targets segments that show high expected ROAS, plus it keeps cost limits. The agent adjusts budgets every hour - this approach raised ROAS by up to 21.8 % and dropped cost-per-acquisition (CPA) by 18.4 %. This occurred when compared to a constant baseline but also a rule-based system. A web-based Opportunity Dashboard also shows future sales growth and the best budget changes as they happen.
The results show that over a variety of time periods, RL-driven budget reallocation significantly increases campaign effectiveness and reduces unnecessary advertising expenditures. The study concludes with a detailed analysis of deployment factors, including whether to expand to multi-channel campaigns on various digital advertising platforms, how to create rewards, and how frequently to retrain the model.