This paper introduces a network-aware video delivery framework where the quality-of-service (QoS) interaction between prioritized packet video and relative differentiated service (DiffServ) network is taken into account. With this framework, we propose a dynamic class mapping (DCM) scheme to allow video applications to cope with service degradation and class-based resource constraint in a time-varying network environment. In the proposed scheme, an explicit congestion notification (ECN)-based feedback mechanism is utilized to notify the status of network classes and the received service quality assessment to the end-host applications urgently. Based on the feedback information, DCM agent at ingress point can dynamically re-map each packet onto a network class in order to satisfy the desired QoS requirement. Simulation results verify the enhanced QoS performance of the streaming video application by comparing the static class-mapping and the class re-mapping based on loss-driven feedback.