Add ability to extract encoder embedding.

[colinator]

Adds the ability to get the encoder output. No other top-level methods expose ggml_tensors though - I'm not sure that's cool. It seemed the quickest way. What I'm doing, eventually, is this:

ggml_tensor * tensor = whisper_get_encoder_embedding(ctx);
std::vector<float> tensor_data(ggml_nelements(tensor));
ggml_backend_tensor_get(tensor, tensor_data.data(), 0, ggml_nbytes(tensor))

... so maybe it should expose a method that returns or populates a vector, instead of returning a ggml_tensor? What do you think?

[bobqianic]

Style

[ggerganov]

so maybe it should expose a method that returns or populates a vector, instead of returning a ggml_tensor?

Yes, populating a buffer with the data would be more portable compared to returning ggml_tensor:

WHISPER_API void whisper_get_encoder_embedding(float * buffer);

[bobqianic]

so maybe it should expose a method that returns or populates a vector, instead of returning a ggml_tensor?

Yes, populating a buffer with the data would be more portable compared to returning ggml_tensor:

WHISPER_API void whisper_get_encoder_embedding(float * buffer);

This leads to the question: how can user determine the size of the buffer? In this scenario, we end up with only a float pointer, which points to the buffer we've just filled in whisper_get_encoder_embedding

[ggerganov]

The buffer size can be determined using the model parameters:

whisper.cpp/whisper.h

Lines 347 to 358 in 2623640

	WHISPER_API int whisper_model_n_vocab (struct whisper_context * ctx);
	WHISPER_API int whisper_model_n_audio_ctx (struct whisper_context * ctx);
	WHISPER_API int whisper_model_n_audio_state(struct whisper_context * ctx);
	WHISPER_API int whisper_model_n_audio_head (struct whisper_context * ctx);
	WHISPER_API int whisper_model_n_audio_layer(struct whisper_context * ctx);
	WHISPER_API int whisper_model_n_text_ctx (struct whisper_context * ctx);
	WHISPER_API int whisper_model_n_text_state (struct whisper_context * ctx);
	WHISPER_API int whisper_model_n_text_head (struct whisper_context * ctx);
	WHISPER_API int whisper_model_n_text_layer (struct whisper_context * ctx);
	WHISPER_API int whisper_model_n_mels (struct whisper_context * ctx);
	WHISPER_API int whisper_model_ftype (struct whisper_context * ctx);
	WHISPER_API int whisper_model_type (struct whisper_context * ctx);