Zero-effort bilateral authentication was introduced recently to use a trusted wristwear to continuously authenticate a smartphone user. A user is allowed to use the smartphone if both wristwear and smartphone are determined to be held by the same person by comparing the wristwear's motion with the smartphone's input or motion, depending on the grip — which hand holds the smartphone and which hand provides the input. Unfortunately, the scheme has several shortcomings. First, it may work improperly when the user is walking since the gait can conceal the wrist's motions of making touches. Second, it continuously compares the motions of the two devices, which incurs a heavy communication burden. Third, the acceleration-based grip inference, which assumes that the smartphone is horizontal with the ground is inapplicable in practice. To address these shortcomings, we propose WearAuth, wristwear-assisted user authentication for smartphones in this paper. WearAuth applies wavelet-based multi-resolution analysis to extract the desired touch-specific movements regardless of whether the user is stationary or moving; uses discrete Fourier transform-based approximate correlation to reduce the communication overhead; and takes a new approach to directly compute the relative device orientation without using acceleration to infer the grip more precisely. In two experiments with 50 subjects, WearAuth produced false negative rates of 3.6% or less and false positive rates of 1.69% or less. We conclude that WearAuth operates properly under various usage cases and is robust to sophisticated attacks.