vllm error

[hj611]

in ruing grounding/uground_qwen2vl.py：
/python3.10/site-packages/vllm/model_executor/layers/rotary_embedding.py", line 1003, in get_rope
[rank0]: raise ValueError(f"Unknown RoPE scaling type {scaling_type}")
[rank0]: ValueError: Unknown RoPE scaling type default
same error occurred when run “vllm serve osunlp/UGround-V1-7B --api-key token-abc123 --dtype float16”
Can you provide a possible solution? Thank you！

[boyugou]

Hi! Here's how I intialized the Python environment (copied from Qwen2-VL's official repo):

#inference
pip install git+https://github.com/huggingface/transformers@21fac7abba2a37fae86106f87fcf9974fd1e3830 accelerate

# It's highly recommanded to use `[decord]` feature for faster video loading.
pip install qwen-vl-utils[decord]

#deployment
pip install git+https://github.com/huggingface/transformers@21fac7abba2a37fae86106f87fcf9974fd1e3830
pip install accelerate
pip install qwen-vl-utils
pip install 'vllm==0.6.1' 

Could you have a try whether these can solve your issue?

[boyugou]

Also feel free to reach out to us if you encounter any issues.

[maxwelljones14]

I also had some trouble trying to get the grounding code working. In particular, I ran into this bug with error: assert "factor" in rope_scaling

I ended up creating an environment with a python version of 3.12:

conda create -n ugroundv1 python=3.12 -y

setting up a python only build as suggested in the vllm docs:

git clone https://github.com/vllm-project/vllm.git
cd vllm
VLLM_USE_PRECOMPILED=1 pip install --editable .

then only after that pip installing transformers, accelerate, and qwen-vl-utils[decord]

since I have vllm in my filesystem, I replaced from vllm import LLM, SamplingParams with from vllm.vllm import LLM, SamplingParams

I also had to manually set a new max_model_len to 5000 when calling LLM(.) as my machine (A5000 24 Gb) was too small for the full input size!

[hj611]

嗨！下面是我初始化 Python 环境的方法（从Qwen2-VL 的官方仓库复制而来）：

#inference
pip install git+https://github.com/huggingface/transformers@21fac7abba2a37fae86106f87fcf9974fd1e3830 accelerate

# It's highly recommanded to use `[decord]` feature for faster video loading.
pip install qwen-vl-utils[decord]

#deployment
pip install git+https://github.com/huggingface/transformers@21fac7abba2a37fae86106f87fcf9974fd1e3830
pip install accelerate
pip install qwen-vl-utils
pip install 'vllm==0.6.1' 

您可以尝试一下这些是否能解决您的问题？

您好，感谢您的回复，问题已经解决！