Actions are Empty Strings: Blocksworld+ToT with Llama3.1

[sumedhpendurkar]

I am trying to run ``examples/ToT/blocksworld/test_tot_v1_dfs.sh'' on step 2, step 4, step 6. However, I noticed that Llama 8B returns empty strings as actions.

example: ['unstack the yellow block from on top of the red block', 'unstack the blue block from on top of the yellow block', '', 'pick up the blue block']

Upon inspection I see that prompt that is being passed is `` the block'' and thats it. I believe `generate' function the input is being trimmed resulting in this issue. I also observed that increasing max_depth is causing more empty strings as actions.

Is there any easy fix to this? Maybe some hyperparameter that can be tuned/code fix?

[sumedhpendurkar]

This can be reproduced by running step 4 with the default parameters provided, or running step 2 with increased depth.

[Ber666]

Hi! Did you use the instruct model or base model?

[sumedhpendurkar]

I used the base model, should I be using the instruct model for the prompts?

[Ber666]

Sorry for late reply.. No, the prompt was written for base models. We will look into the problem.

[Ber666]

Similar to this issue, the problem seems to be from the model class. We updated the code to support huggingface model (PR) and the code works well in our test.