In the library, recognizing the activity of the reader can better uncover the reading habit of the reader and make book management more convenient. In this study, we present the design and implementation of a reading activity recognition approach based on passive RFID tags. By collecting and analyzing the phase profiling distribution feature, our approach can trace the reader's trajectory, recognize which book is picked up, and detect the book misplacement. We give a detailed analysis of the factors that can affect phase profiling in theory and combine these factors with relevant activities. The proposed approach recognizes the activities based on the amplitude of the variation of phase profiling, so that the activities can be inferred in real time through the phase monitoring of tags. We then implement our approach with off-the-shelf RFID equipment, and the experiments show that our approach can achieve high accuracy and efficiency in activity recognition in a real-world situation. We conclude our work and further discuss the necessity of a personalized book recommendation system in future libraries.