We study photonic packet switches to support asynchronously arriving variable-length packets. A scheduler for contention resolution is operated in electrical domain even when data street of the buffer is provided in optical domain. In this scheme, the scheduler may be a bottleneck. To compensate the gap of high-speed optical transmission and slow-speed electronic processing, we propose a multi-stage fiber delay line (FDL) buffer architecture that forms a tree structure in which each node has a block of FDLs and a scheduler. This is especially useful for output-buffer switches in which scheduling complexity is proportional to the number of ports of the packet switch. Through a newly-developed approximate analytical method, we show the optimum unit length of the fiber delay lines to decrease packet loss probability. We also show the sufficient number of FDLs in the two-stage buffer.