Performance evaluation of a WISP for the 2.4GHz and 5.8GHz bands supported in a factorial model

Evaluación del rendimiento de un WISP para las bandas de 2.4GHz y 5.8GHz soportado en un modelo factorial

Juan Carlos Vesga-Ferreira^1*
José Antonio Vesga-Barrera^2*
Martha Fabiola Contreras-Higuera^3*

---

^1*Doctor en Ingeniería – área Telecomunicaciones, juan.vesga@unad.edu.co. ORCID: 0000-0003-3764-7265. Universidad Nacional Abierta y a Distancia, Colombia.

^2*Magister en Potencia Eléctrica, jose.vesga@unicienciabga.edu.co. ORCID: 0000-0003-2403-4646. Corporación Universitaria de Ciencia y Desarrollo – UNICIENCIA, Colombia.

^3*PhD(c). Ingeniería – Área Telecomunicaciones, martha.contreras@unad.edu.co. ORCID: 0000-0002-4394-2589. Universidad Nacional Abierta y a Distancia, Colombia.

---

How to cite: J.C. Vesga-Ferreira. J.A. Vesga-Barrera, M.F. Contreras-Higuera, “Alternatives to improve operational traffic in roundabouts using microsimulation.” Respuestas, vol. 26, no. 1, pp. 30-44, 2021.

© Peer review is the responsibility of the Universidad Francisco de Paula Santander.This is an article under the license CC BY-NC 4.0.

---

Received: Febraury 7, 2020
Approved: April 5, 2020.

---

---

---

Introduction

The transfer of data in large volumes and reliable connections over time became a necessity that experts in this field have had to meet. Large companies, institutions of different kinds, organizations and others, allocate many of their resources (human, economic, technological, etc.) to the research that leads to the development of high performance and capacity technologies [1]. Wireless networks with 802.11 protocol went from being an alternative solution in areas of difficult access where wired-type technologies were unfeasible due to their complexity, to be an option that provides solutions in different environments, with the highest standards of quality and reliability thanks to the constant evolution that it presents [2].

Wireless networks play an important role in the field of connectivity, due to the benefits related to mobility, availability, speed and low cost of implementation [3]. These types of networks are mainly made up of the use of concentrating devices called Access Point (AP), which allow users to connect wirelessly within a limited area, which is why the proliferation of AP is increasing due to the need for more connection coverage.

Currently, two unlicensed frequency spectrum bands are available for use in the IEEE 802.11 WLAN standard: 2.4 GHz assigned to the ISM (Industrial Scientific Medical) and 5GHz bands corresponding to the U-NII (Unlicensed National bands) Information Infrastructure). While the IEEE 802.11b / g standard operates in the 2.4 GHz band, the IEEE 802.11a standard makes use of the 5 GHz band. The number of available channels may vary from one country to another due to the regulations of each in relation to the assignment of the radio spectrum [4]. Additionally, the IEEE 802.11 specifies a spectral mask that defines the permitted distribution of energy through each channel.

In Colombia, the last decade has been characterized by a massive expansion of 802.11 in both simple home or office applications with low transmission levels and indoor environments, as well as in large business applications with MAN or WAN type network infrastructure in mixed environments (indoor and / or outdoor) that require transmitting different types of traffic with capacities up to the order of Gbps and without any errors and cuts in communication [5]. The evolution in telecommunications in our country is supported by the need for massification of the Internet as a tool for social and economic development in all sectors; being the Live Digital Plan a government flag project whose main objective is the reduction of unemployment, poverty reduction and increase in global competitiveness.

Due to the increase in the density of APs in WLAN networks, problems related to their performance have been found, which may be the result of lack of policies for user management, about traffic load in AP, inadequate selection of frequencies between channels, electromagnetic interference, among other aspects; generating that the APs are not able to attend the required volumes of traffic in an adequate way in order to offer adequate levels of QoS (Quality of Service) to the users who need to make use of this service [6]. In this scenario, WLAN networks are unable to harness their full potential. However, the incorporation of policies for intelligent management of radio resources RRM (Radio Resource Management), could be applied to minimize the harmful effects of interference and a non-uniform load distribution, according to studies conducted by Arunesh Mishra, Hector Velayos [7].

This article seeks to analyze the behavior of a WISP network with a previously operational infrastructure and generate an empirical model that evaluates the performance of this type of networks, evaluating parameters such as Throughput and Transmission Delay of communication between stations. and the Access Point, whose data is collected experimentally by injecting random data.

---

Methodology

A fundamental aspect in the design of experiments is to decide which tests or treatments are going to be executed in the process and the amount of repetitions of each one, so that the maximum information is obtained at the minimum cost on the object of study. The arrangement constituted by the different process conditions that will be executed, including the repetitions is called the “design matrix” [8].

An experimental design is more than a set of test conditions: it is a sequence of stages or activities that must be performed to successfully achieve the objectives pursued. Currently, there are several types of experimental designs where each of them allows to study situations that occur in practical life, adjusting to the needs of the researcher. It is very important to know how to choose the most suitable for the problem you want to solve [9].

---

Parameters of interest to evaluate the performance of a network

In any type of networks and especially in wireless networks (due to their sensitivity in the transmission medium) it is very important to know the quality of communication. For this it is necessary to make an analysis of the network and thus determine its performance. In the present study, basic factors such as: Throughput and delay will be taken into account.

Throughput: It is the most relevant parameter when analyzing the performance of a data network. It is defined as the relationship between useful information and the transmission time of the communication [10].

The mathematical expression is given by:

Where,
LM: Total message length
LC: Message Control Bits
TM: Message transmission time
TACC: Media access time

Delay: The delay obtained is calculated as the average of the differences between the shipping and receiving times of the packages. For the purposes of sampling, the value of “Standard Deviation of Delay” (σ) is taken which is calculated according to the following expression [11]:

Where,
N: Number of packages considered
di: Package Delay i
d ̅: Average delay of packages

The experimental design selected is of the factorial type, whose objective is to study the effect of several factors on one or several responses. That is, what is sought is to study the relationship between the factors and the response, in order to know better how this relationship is and generate knowledge that allows actions and decisions to be taken that improve the performance of the process. To identify the way in which each factor influences the variable of interest, it is necessary to choose at least two test levels for each of them. With the complete factorial design, all possible combinations that can be formed with the selected levels are randomly run in the process [12]. In the end, it is expected to define the influence that one or more input factors have on the output or response variable, looking for tools that represent important information and allow to evaluate the performance of a WISP wireless network.

---

Devices used and technical aspects

For the development of the tests in the proposed investigation, CPE Ubiquiti brand model Nanostation M series, Access Point brand Ubiquiti model Rocket series, Ubiquiti brand antenna model M series, mainly laptops were used. Figures 1, 2 and 3 as well as tables 1,2 and 3 illustrate the physical appearance and technical specifications of the Access Point, Antennas and CPEs used during the experiment respectively.

To perform the tests, Ubiquiti brand antennas were used at frequencies of 2.4Ghz and 5.8Ghz. The models used were AM-5G20-90 and AM-2G15-120

The CPEs used were Ubiquiti brand model Nanostation M.

---

Experiment Description

Currently in the market of computer networks there are different kinds of software such as: IPERF, DITG, MGEN, among others, which provide the possibility of generating artificial traffic, simulating the behavior of certain types of data; however, for the present project it was decided to make measurements with real traffic generated from a server (located in the distribution node), where a file was housed to perform the respective download tests with limited speed only by the capacity of the channel or medium of transmission (wireless medium).

To establish an infrastructure-type wireless communication between two terminals, at least one device that acts as an access point and the corresponding terminals is required. In the case of study, the infrastructure described in Figure 4 is available to perform the respective tests and measurements [13].

The experiment consists of connecting a UBIQUITI brand CPE type wireless test device, model NM2 or NM5 to the existing UBIQUITI brand access points, model ROCKET M2 or M5, at a certain distance. The connection mechanism between the host (user) and the server is done with an architecture where the terminal is configured in station-router mode and the access point in AP-Bridge mode acting transparently thanks to the WDS (Wireless distribution system) protocol).Figure 4. WISP TopologyEach experimental sample consists of connecting a test CPE device to the corresponding access point in the places already strategically defined with the established distances; Once the connection has been made, the download of the file hosted on the server is started, the time measurement is taken from the start of the download until it ends. Each measurement is taken from the average of five repetitions looking for greater reliability in the data obtained. The measurement of the total download time of the file and its size were taken as a reference for the throughput measurement; using equation (3) to obtain the desired parameter value.The factors and levels considered in the development of the experiment are presented in Table IV:

Each experimental sample consists of connecting a test CPE device to the corresponding access point in the places already strategically defined with the established distances; Once the connection has been made, the download of the file hosted on the server is started, the time measurement is taken from the start of the download until it ends. Each measurement is taken from the average of five repetitions looking for greater reliability in the data obtained. The measurement of the total download time of the file and its size were taken as a reference for the throughput measurement; using equation (3) to obtain the desired parameter value.

The factors and levels considered in the development of the experiment are presented in Table IV:

In relation to the data presented in the previous table, a mixed factorial design of 2 x 3 x 3 = 18 combinations of treatments or experimental samples is chosen. The number of replications of the experiment is five (5), which represents a total of 18 x 5 = 90 measurements for each output or response variable, data sufficiently reliable to obtain results of the effects of each main factor and each of its interactions.

---

Results

With the data collection obtained previously, the choice of a statistical mathematical technique that allows to infer reliably on the behavior of a WISP network will continue. For this purpose the analysis of variance (ANOVA) will be taken as a tool. In the present investigation, more than one particular factor is studied and the influence that one may have with each other, taking into account this type of conditions it is necessary to perform a factorial design [14]. In general, factorial designs are the most efficient for this type of experiments. By factorial design it is understood that in each test or complete replica of the experiment, all possible combinations of the factor levels are investigated. The effect of a factor is defined as the change in response produced by a change in factor level.

Design matrix product of sampling campaigns

Tables V and VI show the design matrix for Throughput and download delay respectively, according to the results obtained during the development of the experiment.

Statistical Model and Analysis of Variance for Throughput

This model is represented by a factorial design of three (3) ABC factors that models the behavior of a response or output variable Y that can be expressed by: [3]

A = operating frequency
B = distance
C = Number CPE
N = Number of replicas of the experiment Yijkl = Throughput for operating frequency i at a distance between CPE and AP j, with a number of stations(k), for replication l.
μ= General mean without interaction effects
Ai= effect of the i - th level of factor A
Bj= effect of the i - th level of factor B
Ck= effect of the i - th level of factor C
(AB)ij= Effect of the interaction between the factors Frequency and distance
(AC)ik= Effect of the interaction between the factors frequency and amount of CPE
(BC)jk= Effect of the interaction between the factors distance and amount of CPE
(ABC)ijk= Effect of the interaction of the three factors
εijkl= Random error

The factorial study of three factors (A, B, C) allows to investigate the effects: A, B, C, AB, AC, BC and ABC; where the level of breakdown or detail with which they can be studied depends on the number of levels used in each factor. For the particular case, there are seven interesting effects without considering breakdown and with them seven null hypotheses (H_o) can be raised and each one paired with an alternative hypothesis (HA). The ANOVA to test each of these hypotheses is illustrated in table XI.

For cases in which all the factors of the experiment are fixed, it is possible to easily formulate and test hypotheses about the main effects and interactions. In the case of fixed effect models, the tests to test the hypotheses about each main effect and the interactions can be constructed by dividing the corresponding CM of the effect or the interaction by the CME. The degrees of freedom for each main effect are the levels of the factor minus one and the number of degrees of freedom for an interaction is the product of the number of degrees of freedom associated with the individual components of it: [15]

Experimentation procedure:
Dependent variable: THROUGHPUT
Factors:
(A) Operating frequency
(B) Distance
(C) Amount of CPE

Number of experimental samples: 90
Next, the respective variance analysis calculations for Troughput will be carried out, extracting the data reported in Table V. The mathematical equation to determine the correlation factor (FC) is:

The following expression will be used to calculate the sum of squares (SC):

The SCE is determined by the following expression:

The following table is illustrated in analysis of variance for Throuhgput:

For the effect whose p-value is less than the value specified for α, it is declared statistically significant or said to be active. That is, those values where p-value <0.05.

The ANOVA table breaks down the variability of THROUGHPUT in contributions due to several factors. Since the sum of Type III squares (default) has been chosen, the contribution of each factor has been measured by eliminating the effects of the rest of the factors. The p-values verify the statistical importance of each of the factors. Given that according to the analysis of variance there are three effects with p-values lower than 0.05 (A, C and AC), these factors have a statistically significant effect on THROUGHPUT for a 95.0% confidence for which the H_o hypothesis is rejected.

On the other hand, the effects that do not influence the model statistically (B, AB, BC and ABC) could be considered eliminating them by sending their effects on the model at the end of the error (ε). It is very important to keep in mind that the only statistically significant interaction between the factors was that corresponding to the AC interaction, which corresponds to the interaction between the frequency factors and number of CPEs or stations in the network, which is consistent with the mathematical expressions to calculate the value of the Throughput presented above.

Verification of the Assumptions of the Model for Throughput

The validity of the results obtained in any analysis of variance is subject to the assumptions of the model being fulfilled. These assumptions are: normality, constant variance (equal variance of treatments) and independence. That is, the answer (Y) should be distributed normally, with the same variance in each treatment and the measurements must be independent. These assumptions about Y are translated into assumptions about the term error (e) in the model [16].

The assumptions can be checked analytically (mathematically) or graphically, the latter, the most used for its relative simplicity. The graphic form is not entirely reliable, but enough for real situations to show evidence to accept or deny the assumption. For the denial a marked visual effect that supports the decision taken is required.

Some points that could leave the normal behavior of an assumption should not be considered as extreme conditions that violate the assumption in question. These cases can be interpreted as atypical measurements, since they can significantly affect the resulting analyzes.

Normality Assumption

A graphic procedure to verify compliance with the assumption of waste normality consists in plotting the waste on paper or in the normal probability graph that is included in almost all statistical packages.

This graph of type X-Y has the scales in such a way that, if the residues follow a normal distribution, when plotting them they tend to be aligned in a straight line; therefore, if they clearly do not align, it is concluded that the normality assumption is not correct [17].

It should be emphasized that the adjustment of the points to a line does not have to be perfect, since the analysis of variance resists small and moderate deviations from the normality assumption.

In figure 5, corresponding to the probability graph, it is observed that some residuals do not fit properly to the plotted line; which does not prevent compliance with the assumption of normality since these data can be considered as atypical without being sensitive in consideration.

Constant Variance Assumption

One way to verify the assumption of constant variance (or that the treatments have the same variance) is to plot the estimated values against the residues (Ŷij vs εij). Generally, Ŷij goes on the X axis (horizontal) and the residuals on the vertical axis. If the points on the residual plot against the estimates are distributed randomly in a horizontal band, without any clear and blunt pattern, then it is a sign that it complies with the assumption that the treatments have equal variance. On the contrary, if they are distributed with some clear and blunt pattern, such as a bugle or funnel, then it is a sign that the assumption of constant variance is not being fulfilled. In particular, the narrow part of the funnel indicates that at these levels estimated for the response variable, less variability is expected, so it should be analyzed whether these values help maximize or minimize the desired result [16].

In figure 6, it is observed that the assumption of constant variance in which the residual values are plotted vs. estimated values is met, having as reference the criteria outlined above where it indicates a behavior without a clear pattern in the plotted data.

Independence Assumption

The assumption of independence in waste can be verified if the order in which data was collected against the corresponding waste is plotted. In this way, if graphing on the horizontal axis the time (run order) and on the vertical axis the residuals, a clearly defined non-random trend or pattern is detected, this is evidence that there is a correlation between the errors and, therefore, the assumption of independence is not fulfilled. If the behavior of the points is random within a horizontal band, the assumption is being fulfilled [14].

In the figure 7, you can see that the residual values are randomly distributed over the horizontal strip. This indicates that the assumption of independence is fulfilled taking into account the defined criteria.

---

Conclusions

An experimental design is a sequence of stages or activities that must be performed to successfully achieve the objectives pursued. In view of the above and for optimal development of this article, a mixed factorial design was selected, with three quantitative factors (frequency, distance and number of CPE stations), which satisfactorily responded to each of the needs in the process of investigation. However, it is important to consider that the results obtained may be subject to modifications and subsequent adjustments due to the degree of sensitivity of the instruments used to perform the measurement and the conditions existing in the network at the time of carrying out the experiment. Additionally, the analysis of variance (ANOVA) allowed the breakdown of Throughput variability into each of the contributions by factor, under a confidence level of 95.0%. In the case of Throughput, the analysis of variance identified three statistically significant effects. The only statistically significant interaction between the factors was the AC interaction, which corresponds to the interaction between the frequency and CPE factors existing in the network, which is very consistent with the mathematical expressions for the calculation of the throughput.

---

References

[1] F. Juan Carlos Vesga, H. Martha Fabiola Contreras, and W. Harold Esneider Perez, “Optimization of the spectral efficiency in WLAN networks in the 2.4GHz band under the use of allocation models,” Indian J. Sci. Technol., vol. 11, no. 22, pp. 1–13, Jun. 2018.

[2] M. Soleymani, B. Maham, and F. Ashtiani, “Analysis of the downlink saturation throughput of an asymmetric IEEE 802.11n-based WLAN,” in 2016 IEEE International Conference on Communications (ICC), 2016, pp. 1–6.

[3] F. J. Carlos Vesga, H. Martha Fabiola Contreras, and B. Jose Antonio Vesga, “Design of empirical propagation models supported in the Log-Normal Shadowing model for the 2.4GHz and 5GHz bands under Indoor environments,” Indian J. Sci. Technol., vol. 11, no. 22, pp. 1–18, Jun. 2018.

[4] N. S. Ravindranath, I. Singh, A. Prasad, and V. S. Rao, “Performance Evaluation of IEEE 802.11ac and 802.11n using NS3,” Indian J. Sci. Technol., vol. 9, no. 26, Jul. 2016.

[5] N. S. Ravindranath, I. Singh, A. Prasad, and V. S. Rao, Indian journal of science and technology IndJST, vol. 9, no. 26. 2016.

[6] F. Tramarin, S. Vitturi, M. Luvisotto, and A. Zanella, “On the Use of IEEE 802.11n for Industrial Communications,” IEEE Trans. Ind. Informatics, vol. 12, no. 5, pp. 1877–1886, Oct. 2016.

[7] J. Trachewsky et al., “A 2×2 MIMO baseband for high-throughput wireless local-area networking (802.11n),” in 2007 IEEE Hot Chips 19 Symposium (HCS), 2007, pp. 1–14.

[8] J. C. Ferreira, G. Granados-Acuña, and J. A. Vesga-Barrera, “Evaluación del rendimiento de una red LAN sobre power line communications para la transmisión de VOIP,” ITECKNE, vol. 13, no. 1, pp. 83–95, Apr. 2016.

[9] S. Promwong and P. Panthap, “Experimental evaluation of complex form Friis’ transmission formula with indoor/outdoor for ultra wideband inpulse radio,” in 2008 International Conference on Computer and Communication Engineering, 2008, pp. 1037–1041.

[10] C. L. Barrett, M. Drozda, A. Marathe, and M. V. Marathe, “Analyzing interaction between network protocols, topology and traffic in wireless radio networks,” in 2003 IEEE Wireless Communications and Networking, 2003. WCNC 2003., vol. 3, pp. 1760–1766.

[11] S. K. Mohanty and R. K. Giri, “The analysis of Broadband Communication over indoor Powerline channel,” Int. Conf. Commun. Signal Process., pp. 1293–1299, 2014.

[12] W. A. Syafei, “Implementation of K-Best method for MIMO decoder in WLAN 802.11n,” in 2015 2nd International Conference on Information Technology, Computer, and Electrical Engineering (ICITACEE), 2015, pp. 417–421.

[13] D. Gong and Y. Yang, “Link-Layer Multicast in Large-Scale 802.11n Wireless LANs with Smart Antennas,” IEEE Trans. Comput., vol. 65, no. 7, pp. 2118–2133, Jul. 2016.

[14] L. A. Herrera S, “Análisis de la Varianza de un Grupo de Experimentos en Parcelas Subdivididas,” Rev. la Fac. Ciencias Vet., vol. 52, no. 1, pp. 59–72, 2011.

[15] W. Moreno, Aplicaciones al diseño y análisis de experimentos. Bucaramanga, 2002.

[16] G. B. P. Del Cañizo López JF, López Martín D, Lledó García E, “Diseño de modelos experimentales en investigación quirúrgica,” Actas Urol Esp, vol. 32, no. 1, pp. 27–40, 2008.

[17] A. Diaz Cadavid, Diseno estadistico de experimentos. Editorial Universidad de Antioquia, 2009.