I had an interesting discussion with my co-worker Amy the other day about a Web site that our company created. We just got out of a strategy meeting discussing the next steps for the site and we were having a friendly debate about what was more important in understanding what needs to be fixed on your site; the data coming from your Web analytics or the results of a usability study. They are both important tools to consider, however I do think there are practical applications for both.
Web analytics can tell you a lot about what people are doing on your site, where are they going, what are they clicking on, etc. I've certainly talked a lot about what you can gain from Web analytics. Amy's argument though was that you don't know why someone decided to leave your site, it could be the interface, could be the message, could be the products, you just don't know. This is true but you can look at what are your biggest problem areas are, make changes and measure the results. Then you will know what worked. The excellent point that Amy brought up is that it can take a long time to make those small modifications, by the time you get that figured out your opportunity is probably lost.
The cases when I believe a usability study is useful is launching a brand new site. How do you know what your customers want to see by measuring after it is launched? You don't. The other main instance this is advisable is if you want to take your site in an entirely new direction or branch into different areas. Finding out from customers directly is the best way to know before you shoot yourself in the foot.
At the end of the day, both usability studies and Web analytics are tools that you should know are available. Costs and time to do a usability study are one factor to consider, but your Web analytics still need to be reviewed by someone. I think both are useful, but in the end I would trust data of all of my users over what information I'm given from a handful of customers I bring in and watch.
This does bring up an important feature that should be on every Web site, and that is a prominent way for your customers to leave comments or send you feedback. Customers are as willing as ever to help companies succeed, it is a strange phenomenon but you might as well use it. People will tell you what needs to be improved if you ask them.