Multiple Services & Bentos in the same project

[danieltudosiu]

Hi everyone,

I am tackling a decently complicated semi-sequential (some steps can be ran in parallel, but the whole logic is sequential) pipeline and I was looking at BentoML for the serving of it. The overall gist is that we are using about a dozen models that need to work togehter. Some of them have peculiar requirements thus needing to isolate their environment from the other ones.

Could you please offer some guidance about how to best structure it as to take full advantage of the BentoML capabilities?

Currently we are aware of the Custom build context but not sure if it is everything we ought to be aware of. The project structure we are making is similar to the following for the whole project:

- dependencies
   - concrete_type_1_model_x_repo
   - concrete_type_1_model_y_repo
- src
    - services
        - abstract type 1
            - asbract_type_1_service.py
            - concrete type 1 model x
                - bentofile.yml
                - model_x_service.py
            - concrete type 1 model y
            - ...
        - abstract type 2
        - abstract type 3
        - ... 
- .gitignore
- .python-version
- .pyproject.toml
- README.md

Did we understand correctly that we ought to point the bentofile.yml in concrete type 1 model x folder to the required files for service found model_x_service.py such as the concrete_type_1_model_x_repo?

Cheers!

[frostming]

Did we understand correctly that we ought to point the bentofile.yml in concrete type 1 model x folder to the required files for service found model_x_service.py such as the concrete_type_1_model_x_repo?

bentofile.yml is usually located at the build context, which means the bento is built from this directory. So in bentofile.yaml of concret type 1 model x you need to point to model_x_service

[danieltudosiu]

Thanks a lot for the clarification <3