Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models

1City University of Hong Kong, 2Monash University

Abstract

Scalable Vector Graphics (SVG) has become the de facto standard for vector graphics in digital design, offering resolution independence and precise control over individual elements. Despite their advantages, creating high-quality SVG content remains challenging, as it demands technical expertise with professional editing software and a considerable time investment to craft complex shapes. Recent text-to-SVG generation methods aim to make vector graphics creation more accessible, but they still encounter limitations in shape regularity, generalization ability, and expressiveness. To address these challenges, we introduce Chat2SVG, a hybrid framework that combines the strengths of Large Language Models (LLMs) and image diffusion models for text-to-SVG generation. Our approach first uses an LLM to generate semantically meaningful SVG templates from basic geometric primitives. Guided by image diffusion models, a dual-stage optimization pipeline refines paths in latent space and adjusts point coordinates to enhance geometric complexity. Extensive experiments show that Chat2SVG outperforms existing methods in visual fidelity, path regularity, and semantic alignment. Additionally, our system enables intuitive editing through natural language instructions, making professional vector graphics creation accessible to all users.


How does it work?


Given a text prompt, our system first leverages an LLM to generate an SVG template composed of basic geometric primitives. The rendered template is enhanced through SDEdit with ControlNet to add visual details while preserving the overall composition, yielding a target image. The SVG then undergoes a dual-stage optimization process to match the target image.
(1) Primitives are converted to latent embeddings through latent inversion and optimized along with their visual attributes (i.e., filling colors ci, stroke properties si, and transformation matrices Ti).
(2) Point-level optimization is performed to refine the geometric details of SVG paths.

Text-Guided SVG Generation


SVG examples generated by our Chat2SVG. We highlight some shapes to demonstrate semantic clarity and path quality.

Text-Guided SVG Editing


We perform two rounds of refinement on each SVG template and show the optimized output. This figure shows editing types including deletion, modification, and addition.