Issues with Capitalization and Punctuation in Transcribed Audios

[RomanLeo2003]

After transcribing several audio files using medium model, I have noticed that the transcriptions lack capitalization and punctuation. For example:

Transcribed text with punctuation and capitalization: "Produces, for example, a Renault headlight. They say, yes, yes, we produce it."

Transcribed text without punctuation and capitalization: "produces for example a renault headlight they say yes yes we produce it"

I suspect that this issue might be due to some accumulated cache in the model (or something similar). This problem seems to occur with certain types of content, but I am not sure. BTW, sometimes the problem fixes itself after a few minutes of audio. Therefore, my questions are:

Why does this happen?
How can I fix it?

I use this configuration of parameters:

vad_parameters = {
    'threshold': 0.5,
    'min_speech_duration_ms': 400,
    'max_speech_duration_s': float("inf"),
    'min_silence_duration_ms': 400,
    'window_size_samples': 1024,
    'speech_pad_ms': 750
}

hallucination_silence_threshold = 0.8
model_size = "medium"
compute_type = "float16"
beam_size = 8