On the Internet, end hosts and network nodes interdependently work to smoothly transfer traffic. Observed traffic dynamics are the result of those interactions among those entities. To manage Internet traffic to provide satisfactory quality services, such dynamics need to be well understood to predict traffic patterns. In particular, some nodes have a function that sends back-pressure signals to backward nodes to reduce their sending rate and mitigate congestion. Transmission Control Protocol (TCP) congestion control in end-hosts also mitigates traffic deviation to eliminate temporary congestion by reducing the TCP sending rate. How these congestion controls mitigate congestion has been extensively investigated. However, these controls only throttle their sending rate but do not reduce traffic volume. Such congestion control fails if congestion is persistent, e.g., for hours, because unsent traffic demand will infinitely accumulate. However, on the actual Internet, even with persistent congestion, such accumulation does not seem to occur. During congestion, users and/or applications tend to reduce their traffic demand in reaction to quality of service (QoS) degradation to avoid negative service experience. We previously estimated that 2% packet loss results in 23% traffic reduction because of this upper-layer reaction [1]. We view this reduction as an upper-layer congestion-avoidance mechanism and construct a closed-loop model of this mechanism, which we call the Upper-Layer Closed-Loop (ULCL) model. We also show that by using ULCL, we can predict the degree of QoS degradation and traffic reduction as an equilibrium of the feedback loop. We applied our model to traffic and packet-loss ratio time series data gathered in an actual network and demonstrate that it effectively estimates actual traffic and packet-loss ratio.

In this paper, we propose a CTRIL (Common Trend and Regression with Independent Loss) model to infer latent traffic demand in overloaded links as well as how much it is reduced due to QoS (Quality of Service) degradation. To appropriately provision link bandwidth for such overloaded links, we need to infer how much traffic will increase without QoS degradation. Because original latent traffic demand cannot be observed, we propose a method that compares the other traffic time series of an underloaded link, and by assuming that the latent traffic demands in both overloaded and underloaded are common, and actualized traffic demand in the overloaded link is decreased from common pattern due to the effect of QoS degradation. To realize the method, we developed a CTRIL model on the basis of a state-space model where observed traffic is generated from a latent trend but is decreased by the QoS degradation. By applying the CTRIL model to actual HTTP (Hypertext transfer protocol) traffic and QoS time series data, we reveal that 1% packet loss decreases traffic demand by 12.3%, and the estimated latent traffic demand is larger than the observed one by 23.0%.