Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

inference_params problem & mamba2 error: TypeError: causal_conv1d_fwd(): incompatible function arguments. #688

Open
JianKai0716 opened this issue Feb 8, 2025 · 0 comments

Comments

@JianKai0716
Copy link

JianKai0716 commented Feb 8, 2025

First, I encountered the following errors during inference by replacing inference_params=None with other settings.

from mamba_ssm.utils.generation import InferenceParams
m = Mamba(dim_model, layer_idx=7).to(device=device)
inference_all = InferenceParams(max_seqlen=16, max_batch_size=3)
y_with_inference = m(x_all, inference_params=inference_all)

TypeError: InferenceParams.init() got an unexpected keyword argument 'max_seqlen'

Secondly, the following error showed up when I called the Mamba2 from mamba_ssm

from mamba_ssm import Mamba2
self.mamba_1 = Mamba2(d_model=7, d_state=16, d_conv=3, expand=2, headdim=7).to("cuda")

TypeError: causal_conv1d_fwd(): incompatible function arguments. The following argument types are supported: 1. (arg0: at::Tensor, arg1: at::Tensor, arg2: Optional[at::Tensor], arg3: Optional[at::Tensor], arg4: bool) -> at::Tensor
Invoked with: tensor([[[-26.1522, -15.0262, -14.7450],
[ 2.7521, -1.2708, -4.2284],
[-19.3438, -14.0407, -6.7787],
...,

I would appreciate it if someone could help me resolve the confusion above.

@JianKai0716 JianKai0716 changed the title inference problem & mamba2 error inference_params problem & mamba2 error: TypeError: causal_conv1d_fwd(): incompatible function arguments. Feb 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant