Updated prompt to extract text and format it in Markdown, including a…

[dzemeuksis]

Updated prompt to extract text and format it in Markdown, including additional visual details, instead of only describing the image.

The markitdown module is designed to extract text from various documents and save it in Markdown format, as stated in its purpose. This change aligns the default behavior of image processing with the overall goal of the module.

Previously, providing an image resulted in a plain text description, which likely did not meet user expectations. Users are more likely to expect extracted text and formatting when supplying an image, making this change a better fit for the module's intended functionality.

[dzemeuksis]

@microsoft-github-policy-service agree

[PetrAPConsulting]

Hi @dzemeuksis,
I have similar suggestion. I am using this prompt (varified wit GPT 4o) which cover the most pictures embedded in the documents. I do not want to open separate pull request and yours is the most suitable to propose it.

"""Convert this image into a structured markdown representation that preserves its data and relationships. Follow these conversion guidelines based on content type:

For Tables:
Create a proper markdown table with headers and data rows. For example:

Column1	Column2
Data1	Data2

For Mathematical Formulas:
Use LaTeX notation within markdown delimiters. For example:
$$ y = mx + b $$

For Charts and Graphs:

1. Extract the actual data points and represent them in a markdown table
2. Include axis labels, units, and scale information
3. Describe the relationship pattern (linear, exponential, etc.) as a markdown header

For Flowcharts and Diagrams:
Convert to mermaid markdown syntax when possible:

graph LR
    A-->B
    B-->C

Loading

For Process Flows:
Create a numbered list with clear step progression and any branching conditions.

For Technical Diagrams:

1. Create a hierarchical structure using markdown headers
2. List components and their relationships
3. Preserve any measurements or specifications in tables

Additional Guidelines:

• Maintain numerical precision exactly as shown
• Preserve all labels and annotations as markdown text
• Include metadata as key-value pairs at the top
• Use markdown quotes for any explanatory text
• Structure the output to prioritize machine readability
• Preserve relationships between data elements using markdown hierarchy"""

Let me know if you see it as reasonable to push it together.

Petr

[dzemeuksis]

@PetrAPConsulting , that sounds great! What do you think about combining our proposals into something like this:

Analyze the image and extract all visible text in the original language. 
Reproduce the extracted text in a structured Markdown format, preserving any formatting such as headings, bullet points, and highlights. 
Ensure the output accurately reflects the structure and style of the original document.

Follow these additional guidelines based on the content type:

- **Tables:** Create a proper markdown table with headers and data rows.
- **Mathematical Formulas:** Use LaTeX notation within markdown delimiters, e.g., `$$ y = mx + b $$`.
- **Charts and Graphs:**
  - Extract data points into a markdown table.
  - Include axis labels, units, and scale information.
  - Describe patterns (e.g., linear, exponential) under markdown headers.
- **Flowcharts and Diagrams:**
  - Use mermaid markdown syntax where possible.
  - For process flows, create a numbered list with clear step progression.
  - For technical diagrams, list components and their relationships in a structured way, preserving measurements/specifications in tables.

For any visual elements that cannot be represented directly in Markdown, describe them in plain text under a section titled "Visual Notes."

Maintain numerical precision exactly as shown, preserve all labels and annotations as markdown text, and structure the output for both human and machine readability. Output only the converted Markdown text without any additional commentary or explanations.

[PetrAPConsulting]

@dzemeuksis
Hi,

I suppose you proposed prompt is fine but if you would agree I would extend it in some content types and keept example of mermaid. But again, it's up to you.

Analyze the image and extract all visible text in the original language. Reproduce the extracted text in a structured Markdown format, preserving any formatting such as headings, bullet points, and highlights.
Ensure the output accurately reflects the structure and style of the original document.

Follow these additional guidelines based on the content type:

Tables:

• Create exact markdown representation of the table using markdown syntax (|column1|column2|)
• Create a separator row (|---|---|) after the header
• Transcribe all values exactly as they appear in the table

Mathematical Formulas:

• Use LaTeX notation within markdown delimiters, e.g., $$ y = mx + b $$

Charts and Graphs:

• Identify the graph type (bar, line, pie, etc.)
• Extract data points into a markdown table
• Include axis labels, units, and scale information
• Describe patterns (e.g., linear, exponential) under markdown headers
• Record maximums, minimums, and important values

Flowcharts and Diagrams:

• Use mermaid markdown syntax where possible:

  graph LR
      A-->B
      B-->C

Loading

• For process flows, create a numbered list with clear step progression and any branching conditions
• For technical diagrams, list components and their relationships in a structured way, preserving measurements/specifications in tables

For any visual elements that cannot be represented directly in Markdown, describe them in plain text under a section titled "Visual Notes."

Maintain numerical precision exactly as shown, preserve all labels and annotations as markdown text, and structure the output for both human and machine readability. Output only the converted Markdown text without any additional commentary or explanations.

[PetrAPConsulting]

only one comment, to keep price of conversion reasonable pictures should not be bigger than ~1000x1000 px