The VoIP-based Internet Phonesystem is now seen as one of the killer applications in the high speed and broadband internet environment. Given the wide-spread use of the Internet Phone, it is necessary to provide security services for guaranteeing users' privacy. However, providing security service in Internet Phone has the possibility of incurring additional overheads such as call setup delay time. In this paper, we present a one-way key agreement model based on VoIP in order to reduce call setup time as well as protecting user privacy. The proposed approach decreases the delay time of the call setup in comparison with the previous models because our model enables the key generation in caller side without waiting the response from the receiver.