support chat generator as input of TextGenerationPipeline

[lhoestq]

Pipelines generally accept a generator input like pipe(input for input in inputs). In this case the output is an iterator that yields results one by one as they arrive from the pipeline.

TextGenerationPipeline does accept a generator of strings pipe(text for text in texts). Hoewever it doesn't accept a generator of chats pipe(chat for chat in chats) at the moment. This PR fixes this issue and adds support for pipe(chat for chat in chats).

In particular this PR enables this use case with a nice tqdm bar :)

>>> import pandas as pd
>>> from tqdm import tqdm
>>> df = pd.DataFrame({"text": ["Nvidia announces new RTX graphics cards"] * 10})
>>> prompt = "What is the main topic of this text ? REPLY IN LESS THAN 3 WORDS. Text: '{}'"
>>> df["main_topic"] = [
...     y[0]["generated_text"][1]["content"]
...     for y in pipe(([{"role": "user", "content": prompt.format(x)}] for x in tqdm(df["text"])), do_sample=False)
... ]
100%|███████████████████████████████████████████████████████████████████| 10/10 [00:05<00:00,  1.96it/s]
>>> df.head()
                                      text                    main_topic
0  Nvidia announces new RTX graphics cards  New Graphics Cards Announced
1  Nvidia announces new RTX graphics cards  New Graphics Cards Announced
2  Nvidia announces new RTX graphics cards  New Graphics Cards Announced
3  Nvidia announces new RTX graphics cards  New Graphics Cards Announced
4  Nvidia announces new RTX graphics cards  New Graphics Cards Announced

currently it raises

File ~/hf/transformers/src/transformers/pipelines/text_generation.py:310, in TextGenerationPipeline.preprocess(self, prompt_text, prefix, handle_long_generation, add_special_tokens, truncation, padding, max_length, continue_final_message, **generate_kwargs)
    301     inputs = self.tokenizer.apply_chat_template(
    302         prompt_text.messages,
    303         add_generation_prompt=not continue_final_message,
   (...)
    307         **tokenizer_kwargs,
    308     )
    309 else:
--> 310     inputs = self.tokenizer(prefix + prompt_text, return_tensors=self.framework, **tokenizer_kwargs)
    312 inputs["prompt_text"] = prompt_text
    314 if handle_long_generation == "hole":

TypeError: can only concatenate str (not "list") to str

[Rocketknight1]

Yes, this looks good! Can you add a small test (or modify an existing test) in tests/pipelines/test_pipelines_text_generation.py to cover this case?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ruidazeng]

This is awesome actually!