In this paper, we propose a quality-level selection method for adaptive video streaming with scalable video coding (SVC). The proposed method works on the client with the dynamic adaptive streaming over HTTP (DASH) with SVC. The proposed method consists of two components: introducing segment group and a buffer-aware layer selection algorithm. In general, quality of experience (QoE) performance degrades due to stalling (playback buffer underflow), low playback quality, frequent quality-level switching, and extreme-down quality switching. The proposed algorithm focuses on reducing the frequent quality-level switching, and extreme-down quality switching without increasing stalling and degrading playback quality. In the proposed method, a SVC-DASH client selects a layer every G segments, called a segment group to prevent frequent quality-level switching. In addition, the proposed method selects the quality of a layer based on a playback buffer in a layer selection algorithm for preventing extreme-down switching. We implement the proposed method on a real SVC-DASH system and evaluate its performance by subjective evaluations of multiple users. As a result, we confirm that the proposed algorithm can obtain better mean opinion score (MOS) value than a conventional SVC-DASH, and confirm that the proposed algorithm is effective to improve QoE performance in SVC-DASH.