Deep Learning Applications in Beyond 5G Mobile Networks

The thesis is motivated by the evolution of mobile networks needing to match novel applications. Cloud radio access network (C-RAN) is a network architecture that solves power consumption issues, improves scaling, and reduces the operational expenditures for operators. Due to capacity limitations of fronthaul links, C-RAN fails to meet the expectations placed on it in solving the network objectives. Fog radio access network (F-RAN) has been proposed to solve the fronthaul loading. By placing radio processing and caching functionality at the edge nodes latency can be reduced and fronthaul experiences less load, increasing performance gains of the cloud infrastructure.

F-RAN faces multiple issues since the fog access points (FAPs) are a novel addition to the network architecture. FAPs have to communicate with each other to solve optimisation problems and provide a satisfactory quality of service (QoS). Further, compared to the baseband unit (BBU), FAPs are equipped with much smaller caches, so algorithms have to proactively cache content that users would like to access. Due to the stringent low latency requirement set for the future generation of mobile networks, some information cannot be exchanged, thus FAPs have to make assumptions about the global state of the environment. For this reason, the thesis looks at machine learning options, which are known to be able to predict and generalise outcomes based on historical data.

Reinforcement learning (RL) is chosen as it learns from an environment that follows a Markov decision process (MDP). Double deep Q-network (DDQN) is known to be one of the best performers, as it provides convergence guarantees for MDP environments and improved learning stability through the use of experience replay. The thesis uses C-RAN architecture as an environment for a DDQN deployment to control flexible ethernet (FlexE) calendar slot allocation as well as remote unit (RU) load-balancing. The operation of the method is assessed and its performance is compared to auto regressive integrated moving average (ARIMA) algorithm, which uses statistical network information to perform resource allocation for FlexE links.

Assuming information about the global state is unknown, the environment becomes partially observable. This is expected to be the case for F-RAN considering the latency constraints. In a partially observable Markov decision process (POMDP) this leads to uncertainty in making decisions, as the outcomes are perceived as stochastic by the agent. With the goal of maximising the number of users served by multiple FAPs in a POMDP under stringent constraints, DDQN fails to converge to a solution. For this problem, a multi-agent double deep Q-network (MA-DDQN) solution is proposed, in which FAPs exchange information about their experiences after every learning step. The method is tested in a simulated F-RAN environment and its performance is discussed.

While focusing on allocating resources provides satisfactory performance it has been shown to only be a half of the complete solution. Since FAPs are capable of caching data or applications close to the user, controlling what is being cached is equally as important. This makes the problem a capacitated facility location problem (CFLP). A method is proposed that chooses which services to cache in the fog layer in the first step and how to distribute the content to the users in the second step. An algorithm based on MA-DDQN, termed focused batch DDQN (FB-DDQN) is used to solve the optimisation problem of improving the cache hit ratio in the fog layer. FB-DDQN focuses on areas of the state space that the agents know they are uncertain with and requests other agents to share their solutions to similar states, which improves the generalisation of the less knowledgeable agent. The results are compared to the base MA-DDQN algorithm, as well as a RL algorithm that is written for this task.