Audio clarity in video conferencing is a very important component of this form of communication. Audio is (in most views) more important than video.
During an important meeting, if the video all of a sudden decided to stop working, you would be highly upset (especially if you paid money to have video conferencing). At least you would still have the audio to work with. Therefore, instead of a video conference, you would be having a teleconference.
However, if you were having a video conference and the audio was disabled for some reason, the conference would be ruined.
You would only be able to look at your partner but not communicate (unless you know sign language).
However, if your audio is intact and dependable you would not have a problem.
If you have a conference with a large group then you have another problem. That audio would be centralized to the location that your conference is coming from (for ex. Television, computers, et.). Usually in a large conference, the participants will only be able to hear as loud as the television or computer will go. Therefore it would be challenging for those in the back of the room to hear (unless you have a telepresence surround system).
If you have poor or choppy audio clarity in video conferencing most likely you have a poor quality codec (compressor / de-compressor). This is when audio goes from a process of transferring audio from analog to digital so we can receive the message. See How Does Video Conferencing Work.
For these reasons, it is important to put as much importance on the audio component of a conferencing system as you give the video component. They are complementary components that in most implementations are equally important.