Update
As of April 11, 2025, the models from Google AI Studio now output coherent LaTeX formatting without specific guided prompts. This post now serves as a historical reference.
Solution
Paste this in your system prompt (or after your user prompt):
Enclose LaTeX in $ and don't use double/triple ticks.
Source
https://www.reddit.com/r/Bard/comments/1h2ndnu/comment/m1q5gea/