Cognitive radio has been developed recently as a promising solution to tackle the spectrum related issues such as spectrum scarcity and spectrum underutilization. Cognitive spectrum assignment is necessary for allocating spectrum bands to secondary users in order to avoid conflicts among secondary users and maximize the total network performance under a given set of conditions. In most spectrum assignment schemes, throughput is considered as the main criterion for spectrum selection or spectrum assignment. In this paper, we propose a distortion-aware channel allocation scheme for multiple secondary users who compete for primary channels to transmit multimedia data. In the proposed scheme, idle spectrum bands are assigned to the multimedia secondary users that attain the highest video distortion reduction. The scheme is expected to mitigate the selfish behaviors of users in competing channels. The performance effectiveness of our proposed channel allocation scheme is demonstrated through simulation by comparing with a benchmark of two reference spectrum assignment schemes.