This has nothing to do with long or short sounds, but with the fact that the ExoPlayer stops in the background (idle / Doze mode) on some devices (especially Samsung). This problem has been discussed extensively in the forum.
In the meantime I have found a solution for this. You need a foreground service for this.
PS: Maybe one of the moderators can split this and merge it with the other topic.