Wireless Local Area Networks (WLANs) are widely deployed for internet access. Multiple interfering Access Points (APs) lead to a significant increase in collisions, that reduces throughput and affects media traffic. Thus, interference mitigation among different APs becomes a crucial issue in Multi-Cell WLAN systems. One solution to this issue is to assign a different frequency channel to each AP so as to prevent neighboring cells from operating on the same channel. However, most of the existing WLANs today operate in the unlicensed 2.4GHz Industrial, Scientific and Medical (ISM) band, which suffers from lack of the available channels. Therefore, effective channel assignment to minimize the interference in Multi-Cell WLANs is necessary. In this article, we formulate the channel assignment problem as a mixed integer linear programming (MILP) problem that minimizes the worst case total interference power. The main advantage of this algorithm is that it provides a global solution and at the same time guarantees a non-overlapping channel assignment. We also propose a Lagrangian relaxation approach to transform the MILP into a low complexity linear program which can be solved efficiently in real time, even for large sized networks. Simulation results reveal that both the MILP algorithm and the Lagrangian relaxation approach provide a total interference reduction below the default setting of having all APs assigned the same channel. In addition, simulation results on cumulative density function (CDF) of the SINR at the user level prove the validity of the proposed algorithms.