Two new algorithms for performing modular exponentiation are suggested. Nonstandard number systems are used. The first algorithm is based on the representing the exponent as a sum of generalized Fibonacci numbers. This representation is known as Zeckendorf representation. When precomputing is allowed the resulting algorithm is more efficient than the classical binary algorithm, but requires more memory. The second algorithm is based on a new number system, which is called hybrid binary-ternary number system (HBTNS). The properties of the HBTNS are investigated. With or without precomputing the resulting algorithm for modular exponentiation is superior to the classical binary algorithm. A conjecture is made that if more bases are used asymptotically optimal algorithm can be obtained. Comparisons are made and directions for future research are given.