Cloud providers have long sold their leftover computing capacity at a discount, but deciding how much of it to hand out is harder than it looks. Accept too many requests and customers face constant interruptions; accept too few and valuable hardware sits idle.
A new study published in the Journal of Cloud Computing proposes a reinforcement learning approach to that problem. Authors Jing Jin and Yan Cheng describe an algorithm called TPD-SAC that makes admission decisions in real time as conditions inside a data center change, according to a report by Bioengineer.org.
How Preemptible Cloud Services Work
Preemptible services run on computing resources that are temporarily unused by customers on standard contracts. Those contract-based workloads keep top priority, meaning they can reclaim the resources whenever servers become overloaded.
Because of that interruption risk, providers price preemptible capacity cheaply and market it to cost-conscious users. The arrangement lets operators squeeze more revenue from infrastructure that would otherwise generate nothing during quieter periods.
The Overbooking Dilemma
The core challenge for platform operators is deciding how many preemptible requests to let in. Admitting too many creates an overbooking effect across shared infrastructure, which drives up the rate at which jobs get preempted.
High preemption rates frustrate users and, over time, cut into provider revenue. A cautious policy avoids that problem but leaves capacity unsold, so operators give up income they could have earned.
Why Shifting Capacity Makes the Problem Hard
According to the researchers, the amount of idle capacity in a data center does not follow a stable statistical pattern. Instead, the underlying distribution changes continuously, which the authors describe as a non-stationary stochastic process.
To capture that behavior, Jin and Cheng framed admission control, including the effects of overbooking, as a Non-Stationary Markov Decision Process. This formal model treats each admission choice as a decision made under conditions that keep evolving.
Inside the TPD-SAC Approach
The team built its solution on soft actor-critic, a widely used family of reinforcement learning methods, and adapted it to the non-stationary model. A key addition is turning point detection, which identifies moments when the pattern of capacity fluctuations shifts.
That detection is handled by a long short-term memory network enhanced with regularization and additional features. Combining it with the actor-critic framework produced TPD-SAC, an agent that determines whether to admit each service request as it arrives.
Testing on Google Cloud Data and Pricing Models
The researchers evaluated the algorithm using empirical datasets from Google Cloud and reported that it performed effectively in non-stationary operating conditions.
They also ran simulations comparing two pricing approaches: a traditional uniform discount and a discount tied to interruptions. Dynamic admission strategies produced substantially higher revenue contributions under the interruption-based discount than under uniform discounting, according to the study.
What It Means for Cloud Providers and Customers
For cloud operators, the work points to a way of managing spare capacity that adapts to changing conditions rather than relying on fixed rules. That could help providers raise utilization without letting interruption rates spiral.
The pricing results may be equally relevant. They suggest that linking discounts to actual interruptions, rather than applying a flat rate, can pair well with smarter admission control, a consideration for providers and for startups that depend on low-cost, interruptible compute.
Source: Bioengineer.org





