Chart generation

[zli999]

Such a nice work!

I have a question about the code generation for charts.
During the generation, is there any codes generated by GPT-4o having the visual issues, e.g., the overlap of annotations or legends?
Have you used any methods to deal with these synthetic data?

[hewei2001]

Hello! 😊

Thank you so much for your kind words about our work and for raising such a thoughtful question!

Regarding the code generation for charts, you’re absolutely right—currently, even advanced language models can sometimes produce code with visual issues like overlapping annotations or legends. Here’s how we approach handling these challenges:

1. Model Selection: We’ve observed that more powerful models (such as GPT-4o or Claude 3.5) generally have a lower error rate compared to weaker ones (like Llama 3.1-70B). If feasible, using a stronger model may help reduce these visual issues.

2. Filtering with MLLMs: Utilizing MLLMs with real visual capabilities can be an effective way to filter out incorrect charts. You can refer to Section 3.3 in our paper for more details on this method.

3. Prompting Engineering: For common error patterns, we also leverage prompt engineering to guide the models. For example, in 3D charts, we prompt the model to adjust the rotation of text on axes to reduce overlaps. See this file.

I hope these methods provide some insight and are helpful for your needs! And of course, feel free to explore and expand on these techniques to synthesize even higher-quality data. Good luck! 😇