Femtocell has been considered as a key promising technology to improve the capacity of a cellular system. However, the femtocells deployed inside a macrocell coverage are potentially suffered from excessive interference. This paper proposes a novel radio resource optimization in closed access femtocell networks based on bat algorithm. Bat algorithm is inspired by the behavior of bats in their echolocation process. While the original bat algorithm is designed to solve the complex optimization problem in continuous search space, the proposed modified bat algorithm extends the search optimization in a discrete search space which is suitable for radio resource allocation problem. The simulation results verify the convergence of the proposed optimization scheme to the global optimal solution and reveal that the proposed scheme based on modified bat algorithm facilitates the improvement of the femtocell network capacity.