Learning to Slice Wi-Fi Networks: A State-Augmented Primal-Dual Approach

Figure 1. A network slicing diagram. Frequency subbands are allocated to three types of traffic in each slicing window.

In enterprise settings, it is vital to manage network operations to support multiple use cases with different requirements. Additionally, 3GPP includes architectures to integrate Wi-Fi in converged connectivity (5G + Wi-Fi) in enterprise. Network slicing allows an access point (AP) to allocate the network resources across service-level agreement (SLA) categories, rather than individually across many users/flows. Defining dynamic slicing in Wi-Fi with guaranteed quality-of-service (QoS), in a similar fashion as 5G, has the potential to provide mechanisms for seamless dynamic traffic steering across licensed and unlicensed networks. Current Wi-Fi slicing solutions are based on a multi-tenant architecture on a single AP by installing different SSIDs. Despite being simple to implement, this method has drawbacks, including the in- creased overhead for beacon transmission and probe responses by virtual APs, the lack of QoS differentiation inside each SSID slice, and the limitation of association to one slice.

Machine learning approaches are ubiquitous in resource allocation/optimization problems in wireless networks, and growing attention has been given to tackling network slicing problems through learning-based approaches, most commonly reinforcement learning (RL) methods. However, prior work analyzing network slicing with constraints imposed by QoS requirements is relatively scant. Existing approaches generally lack feasibility guarantees—which is partially remedied by projections to feasible policies—and are highly sensitive to suitable selection of penalty weights.

Alejandro Ribeiro’s team explores constraint-aware online deep reinforcement learning (DRL) training by leveraging Lagrangian primal-dual methods with a proactive baseline switching. Most recently, they defined a constrained learning framework to enable dynamic network slicing in Wi-Fi, and introduced a novel flexible Wi-Fi network slicing framework. Formulating the network slicing problem as a constrained radio resource management (RRM) optimization problem, they develop an unsupervised state-augmented primal-dual algorithm. The notion of state augmentation—developed at TILOS for constrained RL—incorporates the Lagrangian dual multipliers into the state space, similar to a closed-loop feedback system. This results in a practical online algorithm that samples from an optimal policy and exhibits feasibility and near-optimality guarantees, which do not hold for the regularized and conventional primal-dual methods. Ribeiro’s team demonstrates these attributes of state augmentation in the context of network slicing with ergodic throughput and latency QoS requirements through numerical experiments.

Figure 2. Test evolutions of high-throughput (left), low-latency (middle-left) QoS constraints, best-effort throughput objective (middle-right) and average rate of QoS violations (right) for the proposed state-augmented primal-dual algorithm and the baselines. Objective and the constraints are averaged (solid lines) over 128 test network realizations and the shadowed bands cover to the worst (99th percentile) realization (dashed lines). Red dashed lines indicate the respective QoS requirements.