You've gotta time the pressing of square. Press it just as it reaches about head height, then let go when you want the player to take the volley. When you let go of square is when they will attempt the shot (unless there is no volley animation available). It's also a good idea to use R2 to walk slighly backwards away from the balls trajectory so you can get a good swing at it. Running onto the volleys is the best way.