Wireless communication security has been increasingly important nowadays. Directional modulation (DM) is seen as a promising wireless physical layer security technology. Traditional DM is a transmit-side technology that projects digitally modulated information signals in the desired directions (or at the desired locations) while simultaneously distorting the constellation formats of the same signals in other directions (or at all other locations). However, these directly exposed digitally modulated information signals are easily intercepted by eavesdroppers along the desired directions (or around the desired locations). A new DM scheme for secure point-to-multipoint communication based on the spread spectrum assisted orthogonal frequency diverse array (short for SS-OFDA-M-DM) is proposed in this paper. It can achieve point-to-multipoint secure communication for multiple cooperative receivers at different locations. In the proposed SS-OFDA-M-DM scheme, only cooperative users that use specific DM receivers with right spread spectrum parameters can retrieve right symbols. Eavesdroppers without knowledge of spread spectrum parameters cannot intercept useful signals directly at the desired locations. Moreover, they cannot receive normal symbols at other locations either even if the right spread spectrum parameters are known. Numerical simulation results verify the validity of our proposed scheme.