ggml error (not enough space in the context's memory pool) with SDXL when large prompt + large negative prompt

[olivbrau]

I'm trying SDXL (+ fixedVAE) on CPU backend.
I encounter an error, but I'm not totally sure what triggers it.
With a medium size prompt (~35 words), I succeeded in generating image 1024x1024 without any problem

But when I tried a larger prompt + a large negative prompt, I get this error during final decoding phase :

ggml_new_object: not enough space in the context's memory pool (needed 23568528, available 23068672)
D:\a\stable-diffusion.cpp\stable-diffusion.cpp\ggml\src\ggml.c:1576: GGML_ASSERT(obj_new) failed

I don't understand the message because I've a lot of available RAM during this processing (and the amount of RAM in question here is very small...)
The process ends with code -1073740791

Prompt :

((European girl)). (((Late evening summer))), ((((asleep sprawled-out on towel in neighboring suburban garden)))), ((high drone aerial view)), (((40-degrees angle))), ((fence)), (((No Nude))), ((shadowy places)), (((long legs))), ((teeshirt)), (((jeans))), ((wide hips)), ((((short messy dark side-swept pixie-cut hair with severe undercut)))), ((medium breasts)), ((barefoot)), (((sweaty dark skin))), ((spine)), ((logical lighting)), ((logical shadows)), (accurate limbs), ((intricate detail)), (small towel), (headphones and cell phone), book, ((600mm lens)),

Negative prompt :

(worst quality), text, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, bad feet, extra fingers, mutated hands, bad proportions, extra limbs, disfigured, bad anatomy, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, mutated hands, fused fingers, long neck

When I try with only the prompt and no negative prompt, it works, everything is OK.
When I try with only the prompt and a small negative prompt (eg. only "bad anatomy"), it works too.

Here is a full log that shows the error (only 2 iterations here, to get the error quickly...) :

D:\Users\braultoli\Desktop\sd-master-9578fdc-bin-win-avx2-x64\inference_tool_CPU_AVX2_2024_11_30>sd -m "..\StableDiffusion XL 1.0 F16\sd_xl_base_1.0.safetensors" --vae "..\StableDiffusion XL 1.0 F16\sdxl_vae.safetensors" --sampling-method euler --steps 2 --cfg-scale 7.0 -H 1024 -W 1024 -s 42 -t 20 -p "((European girl)). (((Late evening summer))), ((((asleep sprawled-out on towel in neighboring suburban garden)))), ((high drone aerial view)), (((40-degrees angle))), ((fence)), (((No Nude))), ((shadowy places)), (((long legs))), ((teeshirt)), (((jeans))), ((wide hips)), ((((short messy dark side-swept pixie-cut hair with severe undercut)))), ((medium breasts)), ((barefoot)), (((sweaty dark skin))), ((spine)), ((logical lighting)), ((logical shadows)), (accurate limbs), ((intricate detail)), (small towel), (headphones and cell phone), book, ((600mm lens))," -n "(worst quality), text, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, bad feet, extra fingers, mutated hands, bad proportions, extra limbs, disfigured, bad anatomy, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, mutated hands, fused fingers, long neck" -v
Option:
n_threads: 20
mode: txt2img
model_path: ..\StableDiffusion XL 1.0 F16\sd_xl_base_1.0.safetensors
wtype: unspecified
clip_l_path:
clip_g_path:
t5xxl_path:
diffusion_model_path:
vae_path: ..\StableDiffusion XL 1.0 F16\sdxl_vae.safetensors
taesd_path:
esrgan_path:
controlnet_path:
embeddings_path:
stacked_id_embeddings_path:
input_id_images_path:
style ratio: 20.00
normalize input image : false
output_path: output.png
init_img:
control_image:
clip on cpu: false
controlnet cpu: false
vae decoder on cpu:false
diffusion flash attention:false
strength(control): 0.90
prompt: ((European girl)). (((Late evening summer))), ((((asleep sprawled-out on towel in neighboring suburban garden)))), ((high drone aerial view)), (((40-degrees angle))), ((fence)), (((No Nude))), ((shadowy places)), (((long legs))), ((teeshirt)), (((jeans))), ((wide hips)), ((((short messy dark side-swept pixie-cut hair with severe undercut)))), ((medium breasts)), ((barefoot)), (((sweaty dark skin))), ((spine)), ((logical lighting)), ((logical shadows)), (accurate limbs), ((intricate detail)), (small towel), (headphones and cell phone), book, ((600mm lens)),
negative_prompt: (worst quality), text, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, bad feet, extra fingers, mutated hands, bad proportions, extra limbs, disfigured, bad anatomy, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, mutated hands, fused fingers, long neck
min_cfg: 1.00
cfg_scale: 7.00
slg_scale: 0.00
guidance: 3.50
clip_skip: -1
width: 1024
height: 1024
sample_method: euler
schedule: default
sample_steps: 2
strength(img2img): 0.75
rng: cuda
seed: 42
batch_count: 1
vae_tiling: false
upscale_repeats: 1
System Info:
SSE3 = 1
AVX = 1
AVX2 = 1
AVX512 = 0
AVX512_VBMI = 0
AVX512_VNNI = 0
FMA = 1
NEON = 0
ARM_FMA = 0
F16C = 1
FP16_VA = 0
WASM_SIMD = 0
VSX = 0
[DEBUG] stable-diffusion.cpp:182 - Using CPU backend
[INFO ] stable-diffusion.cpp:191 - loading model from '..\StableDiffusion XL 1.0 F16\sd_xl_base_1.0.safetensors'
[INFO ] model.cpp:888 - load ..\StableDiffusion XL 1.0 F16\sd_xl_base_1.0.safetensors using safetensors format
[DEBUG] model.cpp:959 - init from '..\StableDiffusion XL 1.0 F16\sd_xl_base_1.0.safetensors'
[INFO ] stable-diffusion.cpp:226 - loading vae from '..\StableDiffusion XL 1.0 F16\sdxl_vae.safetensors'
[INFO ] model.cpp:888 - load ..\StableDiffusion XL 1.0 F16\sdxl_vae.safetensors using safetensors format
[DEBUG] model.cpp:959 - init from '..\StableDiffusion XL 1.0 F16\sdxl_vae.safetensors'
[INFO ] stable-diffusion.cpp:238 - Version: SDXL
[INFO ] stable-diffusion.cpp:271 - Weight type: f16
[INFO ] stable-diffusion.cpp:272 - Conditioner weight type: f16
[INFO ] stable-diffusion.cpp:273 - Diffusion model weight type: f16
[INFO ] stable-diffusion.cpp:274 - VAE weight type: f32
[DEBUG] stable-diffusion.cpp:276 - ggml tensor size = 400 bytes
[DEBUG] clip.hpp:171 - vocab size: 49408
[DEBUG] clip.hpp:182 - trigger word img already in vocab
[DEBUG] ggml_extend.hpp:1075 - clip params backend buffer size = 469.44 MB(RAM) (196 tensors)
[DEBUG] ggml_extend.hpp:1075 - clip params backend buffer size = 2649.92 MB(RAM) (517 tensors)
[DEBUG] ggml_extend.hpp:1075 - unet params backend buffer size = 4900.07 MB(RAM) (1680 tensors)
[DEBUG] ggml_extend.hpp:1075 - vae params backend buffer size = 94.47 MB(RAM) (140 tensors)
[DEBUG] stable-diffusion.cpp:413 - loading weights
[DEBUG] model.cpp:1645 - loading tensors from ..\StableDiffusion XL 1.0 F16\sd_xl_base_1.0.safetensors
[DEBUG] model.cpp:1645 - loading tensors from ..\StableDiffusion XL 1.0 F16\sdxl_vae.safetensors
[INFO ] stable-diffusion.cpp:512 - total params memory size = 8113.89MB (VRAM 0.00MB, RAM 8113.89MB): clip 3119.36MB(RAM), unet 4900.07MB(RAM), vae 94.47MB(RAM), controlnet 0.00MB(VRAM), pmid 0.00MB(RAM)
[INFO ] stable-diffusion.cpp:516 - loading model from '..\StableDiffusion XL 1.0 F16\sd_xl_base_1.0.safetensors' completed, taking 17.31s
[INFO ] stable-diffusion.cpp:546 - running in eps-prediction mode
[DEBUG] stable-diffusion.cpp:590 - finished loaded file
[DEBUG] stable-diffusion.cpp:1464 - txt2img 1024x1024
[DEBUG] stable-diffusion.cpp:1194 - prompt after extract and remove lora: "((European girl)). (((Late evening summer))), ((((asleep sprawled-out on towel in neighboring suburban garden)))), ((high drone aerial view)), (((40-degrees angle))), ((fence)), (((No Nude))), ((shadowy places)), (((long legs))), ((teeshirt)), (((jeans))), ((wide hips)), ((((short messy dark side-swept pixie-cut hair with severe undercut)))), ((medium breasts)), ((barefoot)), (((sweaty dark skin))), ((spine)), ((logical lighting)), ((logical shadows)), (accurate limbs), ((intricate detail)), (small towel), (headphones and cell phone), book, ((600mm lens)),"
[INFO ] stable-diffusion.cpp:673 - Attempting to apply 0 LoRAs
[INFO ] stable-diffusion.cpp:1199 - apply_loras completed, taking 0.00s
[DEBUG] conditioner.hpp:329 - parse '((European girl)). (((Late evening summer))), ((((asleep sprawled-out on towel in neighboring suburban garden)))), ((high drone aerial view)), (((40-degrees angle))), ((fence)), (((No Nude))), ((shadowy places)), (((long legs))), ((teeshirt)), (((jeans))), ((wide hips)), ((((short messy dark side-swept pixie-cut hair with severe undercut)))), ((medium breasts)), ((barefoot)), (((sweaty dark skin))), ((spine)), ((logical lighting)), ((logical shadows)), (accurate limbs), ((intricate detail)), (small towel), (headphones and cell phone), book, ((600mm lens)),' to [['European girl', 1.21], ['. ', 1], ['Late evening summer', 1.331], [', ', 1], ['asleep sprawled-out on towel in neighboring suburban garden', 1.4641], [', ', 1], ['high drone aerial view', 1.21], [', ', 1], ['40-degrees angle', 1.331], [', ', 1], ['fence', 1.21], [', ', 1], ['No Nude', 1.331], [', ', 1], ['shadowy places', 1.21], [', ', 1], ['long legs', 1.331], [', ', 1], ['teeshirt', 1.21], [', ', 1], ['jeans', 1.331],
[DEBUG] clip.hpp:311 - token length: 154
[DEBUG] ggml_extend.hpp:1026 - clip compute buffer size: 1.40 MB(RAM)
[DEBUG] ggml_extend.hpp:1026 - clip compute buffer size: 2.33 MB(RAM)
[DEBUG] ggml_extend.hpp:1026 - clip compute buffer size: 2.33 MB(RAM)
[DEBUG] conditioner.hpp:457 - computing condition graph completed, taking 1330 ms
[DEBUG] ggml_extend.hpp:1026 - clip compute buffer size: 1.40 MB(RAM)
[DEBUG] ggml_extend.hpp:1026 - clip compute buffer size: 2.33 MB(RAM)
[DEBUG] conditioner.hpp:457 - computing condition graph completed, taking 1993 ms
[DEBUG] conditioner.hpp:329 - parse '(worst quality), text, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, bad feet, extra fingers, mutated hands, bad proportions, extra limbs, disfigured, bad anatomy, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, mutated hands, fused fingers, long neck' to [['worst quality', 1.1], [', text, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, bad feet, extra fingers, mutated hands, bad proportions, extra limbs, disfigured, bad anatomy, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, mutated hands, fused fingers, long neck', 1], ]
[DEBUG] clip.hpp:311 - token length: 154
[DEBUG] ggml_extend.hpp:1026 - clip compute buffer size: 1.40 MB(RAM)
[DEBUG] ggml_extend.hpp:1026 - clip compute buffer size: 2.33 MB(RAM)
[DEBUG] ggml_extend.hpp:1026 - clip compute buffer size: 2.33 MB(RAM)
[DEBUG] conditioner.hpp:457 - computing condition graph completed, taking 1334 ms
[DEBUG] ggml_extend.hpp:1026 - clip compute buffer size: 1.40 MB(RAM)
[DEBUG] ggml_extend.hpp:1026 - clip compute buffer size: 2.33 MB(RAM)
[DEBUG] conditioner.hpp:457 - computing condition graph completed, taking 2013 ms
[INFO ] stable-diffusion.cpp:1332 - get_learned_condition completed, taking 4119 ms
[INFO ] stable-diffusion.cpp:1355 - sampling using Euler method
[INFO ] stable-diffusion.cpp:1359 - generating image: 1/1 - seed 42
[DEBUG] ggml_extend.hpp:1026 - unet compute buffer size: 879.25 MB(RAM)
|==================================================| 2/2 - 88.88s/it
[INFO ] stable-diffusion.cpp:1395 - sampling completed, taking 180.08s
[INFO ] stable-diffusion.cpp:1403 - generating 1 latent images completed, taking 180.75s
[INFO ] stable-diffusion.cpp:1406 - decoding 1 latents
ggml_new_object: not enough space in the context's memory pool (needed 23566736, available 23068672)
D:\a\stable-diffusion.cpp\stable-diffusion.cpp\ggml\src\ggml.c:1576: GGML_ASSERT(obj_new) failed

[lostdisc]

I tested the same prompt and can confirm the error happens to me too, FWIW. Trying VAE tiling did not help; it fails in the same way before ever getting to the tiling part.

[olivbrau]

I've tried with a slightly larger negative prompt, and it worked.
So I think there is a kind of limit in the nb token before the bug arise.