grout or caulk around shower base

I had my travertine shower installed about 4 years ago and unfortunately the installer did not do a good job of grouting around the shower floor. I contacted him about the problem so he came back and used unsanded caulk in a matching color in the corners and around the base of the shower. Over the years the corners had accumulated some mildew even though I dry it after each use. I finally found the appropriate caulk and started removing the old caulk but found that in some places there was no grout at all around the base. My question is: Should I try to re-grout around the base and let it dry thoroughly before trying to caulk as added protection?