One of the challenging issues in affective computing is to give a machine the ability to recognize the affective states with intensity of a person. Few studies are directed toward this goal by categorizing affective behavior of the person into a set of discrete categories. But still two problems exist: gesture is not yet a concern as a channel of affective communication in interactive technology, and existing systems only model discrete categories but not affective dimensions, e.g., intensity. Modeling the intensity of emotion has been well addressed in synthetic autonomous agent and virtual environment literature, but there is an evident lack of attention in other important research areas such as affective computing, machine vision, and robotic. In this work, we propose an affective gesture recognition system that can recognize the emotion of a child and the intensity of the emotion states in a scenario of game playing. We used levels of cognitive and non-cognitive appraisal factors to estimate intensity of emotion. System has an intelligent agent (called Mix) that takes these factors into consideration and adapt the game state to create a more positive interactive environment for the child.