To all:

This page is my draft note of the paper. I just picked some useful sentences (in my view) for my project MAB-based algorithm + wireless resource allocation, so it is unorganized and unrevised. I'm afraid that it won't be a big help to you. If you are interested in the paper, please refer to the following link.

Autonomous Resource Allocation for dense LTE networks: A Multi Armed Bandit formulation
Afef Feki, Veronique Capdevielle
Alcatel-Lucent Bell Labs, France
2011 IEEE 22nd International Symposium on Personal, Indoor and Mobile Radio Communications

Abstract — Resource allocation is an important prerequisite for the effective deployment of Pico Cells (PCs). This topic becomes even more challenging in the case of heterogeneous networks, where autonomous interference management mechanisms are necessary. In this article, we propose a resource sharing method inspired from the reinforcement learning theory and particularly the methods used to solve the Multi Armed Bandit (MAB) problem. The main goal resides in giving the ability for each cell to make its decision autonomously while dynamically taking into
account the resource occupation of each surrounding cell. We set up the global framework for the MAB based resource allocation strategies in the case of total frequency overlapping PCs. The performances of the proposed method are evaluated in the case of Long Term Evolution (LTE) Pico Cells deployment and compared to static allocation schemes. The results demonstrate the efficiency of our method.

Index Terms—Inter-Cell Interference (ICI), Pico Cell (PC), LTE, Resource allocation, MAB, Reinforcement Learning theory

NOTES

• Autonomous interference management mechanism in heterogeneous networks

• A resource sharing method inspired from the reinforcement learning theory, particularly the methods used to solve MAB problem.

• Main Goal: The ability of each cell to make its decision autonomously while dynamically taking into account the resource occupation of each surrounding cell.

• MAB-based strategies for total frequency overlapping PCs.

• LTE: Long Term Evolution

• ICI: Inter Cell Interference

• Real problem: Effective deployment of Pico Cells

• Main idea: selecting only a set of Physical Resource Blocks(PRBs) from the total available frequency band to schedule the attached users. PRB corresponds to the SMALLEST RADIO RESOURCE in LTE that can be allocated to one user and corresponds to a time-frequency block of 1 slot and 12 sub-carriers

• Different ICI avoidance schemes( static and dynamic) proposed.

• ENHANCE the dynamic frequency partitioning schemes by setting up smart procedures that use only local information available at the cell level, without information exchange with neighboring cells.( online MAB algorithm)

• EE Dilemma here: (1)Exploit cumulated knowledge by choosing the mostpropriate band and transmitting on them (2) Exploring other frequency sub-bands within the available whole band to detect other resources that could be interesting to exploit.

• Similar approach has been used for dynamic spectrum access in Cognitive Radio. Here the special case is LTE Pico Cells.

• MAJOR OBSTACLE for deploying and operating PCs is Inter-Cell Interference(ICI) between close-by cells that contend for the same frequency bandwidth. [Intra-cell interference is canceled thanks to OFDMA used in LTE.]

• The total available band is divided into equal parts and each cell need to choose autonomously the part to transmit on.

• GOAL: Steer the decision of each cell to choose the most appropriate resources with minimum interference level while ensuring reactivity to the possible changes that can occur in the resources usage.

• SB = argmax (DF) Explain

• Def: DF = …

• : Mean reward, considering the performance encountered by the cell when transmitting on SB during a period T(Assumed time unit) It corresponds for example to the Signal to Interference Rayio(SIR), Channel Quality Indicator(CQI) throughput values, ACK/NACK counter or others.

• : The number of times SB_j is chosen by the cell I until time t
  a/b the ration controls the tendency to explore( tuning the exploration degree) this arm(action)
  Another example about a : set as the variance of the rewards -> choose more frequently the sub-bands with high variance.

• Evalueation: A regret function regret bound of UCB. Pic attached

• Pros: computationally efficient, does not require prior knowledge of the reward distribution (online learning)

• Introduce the UCB-based algorithm

Simulation:

• Evaluated through two metrics: 1. effective SINR(Signal to Interference plus Noise Ratio) 2. Instantaneous throuput

• Cumulative Distribution Fucntion(CDF) of respective metrics

WHAT Is CDF???

• SB(subBand) Size: 20RBS, 33RBs and 50 RBs. (RB stands for resource blocks)

• Total band is 20MHz, corresponding 100RBs.

Conclusion:

• MAB is able to find autonomously the best choices and then reach the performances of a static scheme
• A smart resource sharing method to coordinate interference between poci cells that operate on the same frequency.
• Outperforms the static frequency planning schemes In both post-scheduling SINR and throughput.

Some Background

Pico Cells(PC): https://en.wikipedia.org/wiki/Picocell

Heterogeneous network: https://en.wikipedia.org/wiki/Heterogeneous_network

• For example, a wireless network which provides a service through a wireless LAN and is able to maintain the service when switching to a cellular network is called a wireless heterogeneous network.

Cognitive Radio: https://en.wikipedia.org/wiki/Cognitive_radio

• **A cognitive radio ( CR ) is a radio that can be programmed and configured dynamically to use the best wireless channels in its vicinity to avoid user interference and congestion. Such a radio automatically detects available channels in wireless spectrum, then accordingly changes its transmission or reception parameters to allow more concurrent wireless communications in a given spectrum band at one location. This process is a form of dynamic spectrum management.

在LTE网络运营初期，运营商的主要目标是建设大型的宏站（Macro Cell）小区，取得尽可能大的地域覆盖率。随着用户的增加，宏站（MacroCell）小区进行小区分裂，变得越来越小，同时宏站（Macro Cell）覆盖盲区仍然存在，热点地区的高话务量也无法承载，微基站（Micro Cell）技术就是为了解决以上难题而产生的。微基站（Microcell）的应用主要有两方面：一是提高覆盖率，应用于一些宏站（Macrocell）很难覆盖到的盲点地区，如地铁、地下室；二是提高容量，主要应用在高话务量地区，如繁华的商业街、购物中心、体育场等。

微基站（Microcell）是小站（Small Cell）的一种形态，小站（Small Cell）在Femtocell技术基础上逐渐演变，融合了Femtocell（面向家庭场景），Picocell（面向大型写字楼场景），微基站（Microcell）和分布式无线技术（如RRH射频拉远技术）。今天，随着WLAN的流行，面向家庭的Femtocell也升级为Nanocell（集成了WLAN）。