With the acceptance of social sharing, public bike sharing services have become popular worldwide. One of the most important tasks in operating a bike sharing system is managing the bike supply at each station to avoid either running out of bicycles or docks to park them. This requires the system operator to redistribute bicycles from overcrowded stations to under-supplied ones. Trip demand prediction plays a crucial role in improving redistribution strategies. Predicting trip demand is a highly challenging problem because it is influenced by multiple levels of factors, both environmental and individual, e.g., weather and user characteristics. Although several existing studies successfully address either of them in isolation, no framework exists that can consider all factors simultaneously. This paper starts by analyzing trip data from real-world bike-sharing systems. The analysis reveals the interplay of the multiple levels of the factors. Based on the analysis results, we develop a novel form of the point process; it jointly incorporates multiple levels of factors to predict trip demand, i.e., predicting the pick-up and drop-off levels in the future and when over-demand is likely to occur. Our extensive experiments on real-world bike sharing systems demonstrate the superiority of our trip demand prediction method over five existing methods.