Vitamin D should be a staple supplement even living in sunny California. It seems to work too good for me though and can set off some autoimmune reactions if I take too much. My doc thinks that's not possible. I think he's full of shit.
At any rate, you should also be taking probiotics of some sort. The current theory is that the gut controls much of our immune system so it's worth a shot to take a daily probiotic. The first thing some people will tell you is that probiotics don't survive stomach acid into the intestines so they are a waste of time and money. But some brands are coated and formulated to make it to your intestines relatively intact (think enteric coated aspirin type of thing).
Also, there's the theory that even if the probiotic "bugs" don't make it to your intestines alive the simple fact that you're adding bacteria to you stomach and gut produces an immune response. Truthfully I don't know if either theory is correct but I'm giving it a try and so far so good. I've been using these tiny Enzymatic Therapy Pearls and they seem alright. I "feel" better taking them but admittedly that could simply be placebo effect.
You can also try upping your cultured food intake like with yogurt and real fermented pickles, etc. (Real kimchi is very good for you but it's an acquired taste for some people.) Definitely avoid sugar though!
My friend eats Activia yogurt like a mad man but fails to realize it's loaded with sugar which neutralizes any good effects he's getting from it. Just the other day he was complaining about "coming down with something". I tried telling him that eating sugar-laden yogurt and eating leftover Halloween candy was bad for his immune system and wasn't doing him any favors. Neither is adding a huge shot of hazelnut syrup to his coffee each morning. But it's like talking to a brick wall with this guy. He's got no self discipline when it comes to food.
At any rate, avoid sugar (which I know you are), take vitamin D and take some kind of probiotic.