Skip to content

Commit

Permalink
llama : skip loading unused tensors (#12004)
Browse files Browse the repository at this point in the history
* llama : assign unknown/unused tensors to host buffer type

ggml-ci

* llama : skip unused tensors

ggml-ci
  • Loading branch information
ggerganov authored Feb 21, 2025
1 parent 586d5fe commit 51f311e
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions src/llama-model.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1424,6 +1424,14 @@ bool llama_model::load_tensors(llama_model_loader & ml) {
throw std::runtime_error(format("missing tensor info mapping for %s", tn.str().c_str()));
}

// skip unused tensors
if (info.op == GGML_OP_NONE) {
LLAMA_LOG_WARN("model has unused tensor %s -- ignoring\n", tn.str().c_str());
ml.n_created++;

return nullptr;
}

// tensors with "bias" suffix are always used with GGML_OP_ADD
ggml_op op;
bool bias = tn.suffix != nullptr && strcmp(tn.suffix, "bias") == 0;
Expand Down

0 comments on commit 51f311e

Please sign in to comment.