Manually update model repository index

[vilkkiE]

Is your feature request related to a problem? Please describe.
As I understand, when using --model-control-mode=explicit Triton server updates the model repository index only when first starting the server. If you add models with new names to the repository after the server has been started, the server doesn't recognize that they exist and trying to load them fails.

Describe the solution you'd like
I'd like there to be an API call that allows you to update the repository index in case there are changes

Describe alternatives you've considered
I guess control mode poll regularly updates the index, but it also tries to load all models in the repository, which isn't ideal.

[nnshah1]

I believe this is similar to: #7066

Is that accurate?

[nnshah1]

Here is a potential workaround - though not finalized behavior:

https://github.com/triton-inference-server/core/pull/340/files

[vilkkiE]

@nnshah1 Thank you for the response. I don't think that other issue is quite the same as what I'm describing. I'm not trying to upload the model in the load_model call, but rather I'm adding models to the model repository separately (not with triton client) and then I try to load them with triton client like this:

triton_client.load_model(
    model_name=model_name,
    config=json_obj
)

But this results in an error InferenceServerException: [StatusCode.INTERNAL] failed to load 'model_name', failed to poll from model repository

[nnshah1]

got it - I think I understand the steps here are:

1. start triton with explicit model control
2. add a model to the repository (say on disk)
3. try to load the model via the model load

--> error that it is unknown name?

Can you confirm what version of Triton you are using / etc.?

[vilkkiE]

got it - I think I understand the steps here are:

start triton with explicit model control
add a model to the repository (say on disk)
try to load the model via the model load
--> error that it is unknown name?

Yes that is correct

Can you confirm what version of Triton you are using / etc.?

I am using tritonserver:23.02-py3 image

[teddius]

@nnshah1 is there a way to configure triton to refresh the model index every 3 seconds?

[nnshah1]

@nnshah1 is there a way to configure triton to refresh the model index every 3 seconds?

there is model control mode poll - which polls for changes - but are you wanting to "poll but not load" as we are discussing here?