What Happened
JFrog researchers found that Vanna.AI, a Python library that turns natural-language questions into SQL, also asks an LLM to generate Plotly visualization code and then runs it with Python's exec(). By crafting a question that smuggles a prompt-injection payload through the SQL step, an attacker could steer the LLM into emitting arbitrary Python that executed on the host. The proof of concept listed files on the server, but any command could run with the privileges of the hosting process.
Impact
Applications exposing Vanna's visualization flow to end users faced full remote code execution on the backend (CVSS 8.1). Vanna published hardening guidance urging developers to run generated code only inside a sandboxed environment.
How to Prevent This
- Never pass LLM-generated code directly to exec() or similar dynamic evaluation on a trusted host
- Execute any model-produced code inside a locked-down sandbox with no filesystem or network access
- Do not rely on system-prompt guardrails alone to prevent injection into downstream code generation
- Validate and constrain generated SQL and plotting code against a strict allowlist of operations
- Run the hosting process with least privilege so a breakout yields minimal access