With the development of silicon-based Nano-photonics, Optical Network on Chip (ONoC) is, due to its high bandwidth and low latency, becoming an important choice for future multi-core networks. As a key ONoC technology, the arbitration scheme should provide differential arbitration service with high throughput and low latency for various types and priorities of traffic in CMPs. In this work, we propose a fast hierarchical arbitration scheme based on multi-level priority QoS. First, given multi-priority data buffer queue, arbiters provide differential transmissions with fair service for all nodes and guarantee the max-transmit-delay and min-communication-bandwidth for all queues. Second, arbiter adopts the transmit bound resource reservation scheme to reserve time slots for all nodes fairly, thereby achieving a throughput of 100%. Third, we propose fast arbitration with a layout of fast optical arbitration channels (FOACs) to reduce the arbitration period, thereby reducing packet transmitting delay. Simulation results show that with our hierarchical arbitration scheme, all nodes are allocated almost equal service access probability under various traffic patterns; thus, the min-communication-bandwidth and max-transmit-delay is guaranteed to be 5% and 80 cycles, respectively, under the overload demands. This scheme improves throughput by 17% compared to FeatherWeight under a self-similar traffic pattern and decreases arbitration delay by 15% compare to 2-pass arbitration, incurring a total power overhead of 5%.