Logo: to the web site of Uppsala University

uu.sePublications from Uppsala University
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Mean-Field Multi-Agent Reinforcement Learning For Buffered Network Optimization
(English)Manuscript (preprint) (Other academic)
Abstract [en]

This paper proposes a mean-field multi-agent reinforcement learning (MARL) framework for optimizing transmission control in buffered cellular networks. Each base station is modeled as an autonomous agent with finite queuing capacity, interacting with neighboring stations through interference on a Voronoi-based network topology. To address scalability issues in dense networks, a mean-field approximation is used so that agents respond to the average behavior of their neighbors rather than to full global states. A mean-field Q-learning algorithm and a corresponding reward function are derived to jointly balance Shannon capacity, buffer occupancy, and delay. Convergence of the learning dynamics is formally proved, and performance is evaluated via greedy,  tabular, and deep Q-network (DQN) approaches. Simulation results show that the proposed implementation significantly lowers delays and signal losses, and hence achieves better overall performance.

Keywords [en]
reinforcement learning, multi-agent, mean-field, deep Q-network, cellular network, retransmission policy
National Category
Engineering and Technology Computer Sciences Mathematical sciences
Research subject
Applied Mathematics and Statistics
Identifiers
URN: urn:nbn:se:uu:diva-571406OAI: oai:DiVA.org:uu-571406DiVA, id: diva2:2013013
Available from: 2025-11-11 Created: 2025-11-11 Last updated: 2025-11-26
In thesis
1. Modelling and Performance of Cellular Networks: Stochastic Geometry, Queuing, and Learning Approaches
Open this publication in new window or tab >>Modelling and Performance of Cellular Networks: Stochastic Geometry, Queuing, and Learning Approaches
2025 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

This thesis is based on seven papers concerning mathematical models for wireless cellular networks with retransmissions, buffering, and interference. The analysis combines stochastic geometry with queuing theory to capture complex stochastic aspects of the physical model. Paper I introduces a downlink model with transmitter buffers, providing performance measures such as coverage probability, delay, and loss probability. Paper II extends the modeling approach to quantify Shannon capacity under finite and infinite buffer regimes. Paper III studies multi-tier networks, extending the previous approach. The paper introduces biased load balancing and discusses the increase in capacity compared with single-tier systems. Pa-per IV derives a stability condition for buffered uplink traffic, for a special case of no noise and unbounded attenuation. The paper further refines the analytical stability bound through simulations. Paper V considers the network with heterogeneous users with different arrival rates and powers, and establishes user-specific stability bounds. Paper VI uses the well-known Foster criteria for two-dimensional Markov chains and extends them to derive both stability and transience criteria for Markov chains in higher dimensions with monotone drifts. Finally, Paper VII studies a model of a buffered cellular network in terms of reinforcement learning (RL) methodology. It introduces a decentralized mean-field RL method, where base stations act as agents who aim to maximize their channel capacity via dynamically adjusting the transmission intensity.

Place, publisher, year, edition, pages
Uppsala: Acta Universitatis Upsaliensis, 2025. p. 64
Series
Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology, ISSN 1651-6214 ; 2615
Keywords
Cellular networks, performance evaluation, stochastic geometry, stochastic modelling, Shannon capacity, coverage probability, Markov chains, reinforcement learning.
National Category
Communication Systems Mathematical sciences
Research subject
Applied Mathematics and Statistics
Identifiers
urn:nbn:se:uu:diva-571533 (URN)978-91-513-2675-7 (ISBN)
Public defence
2026-01-14, Polhemsalen, Ångströmlaboratoriet, Lägerhyddsvägen 1, Uppsala, 17:29 (English)
Opponent
Supervisors
Available from: 2025-12-18 Created: 2025-11-13 Last updated: 2025-12-18

Open Access in DiVA

No full text in DiVA

Engineering and TechnologyComputer SciencesMathematical sciences

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 36 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf