Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve default live quality increase settings to work with live streams playing at ~3 segments live offset. #9784

Closed
cdongieux opened this issue Dec 14, 2021 · 8 comments
Assignees

Comments

@cdongieux
Copy link

I'm reproducing an issue with all versions of ExoPlayer from 2.13.0 to 2.16.1 where ExoPlayer does not switch to a higher resolution on a Live DASH stream with AVC video tracks.

ExoPlayer starts playing the 896x504 track then never chooses 1280x720 or 1920x1080 even if there is enough network bandwidth.

Here is the list of available video tracks:

MediaCodecVideoRenderer [
     Group:0, adaptive_supported=YES [
       [X] Track:0, id=1209, mimeType=video/avc, bitrate=400000, codecs=avc1.64000d, drm=[widevine,cenc], res=384x216, fps=25.0, supported=YES
       [X] Track:1, id=1211, mimeType=video/avc, bitrate=800000, codecs=avc1.64001e, drm=[widevine,cenc], res=640x360, fps=25.0, supported=YES
       [X] Track:2, id=1212, mimeType=video/avc, bitrate=1600000, codecs=avc1.64001f, drm=[widevine,cenc], res=896x504, fps=25.0, supported=YES
       [X] Track:3, id=1213, mimeType=video/avc, bitrate=3000000, codecs=avc1.64001f, drm=[widevine,cenc], res=1280x720, fps=25.0, supported=YES
       [X] Track:4, id=1214, mimeType=video/avc, bitrate=4800000, codecs=avc1.640028, drm=[widevine,cenc], res=1920x1080, fps=25.0, supported=YES
     ]
]

As stream URL and DRM info are sensitive, I'll provide you by email a sample code hosted on a private GitHub repository to help you reproduce the issue. Please let me know if you need anything else.

@tonihei
Copy link
Collaborator

tonihei commented Dec 14, 2021

The short answer is that there is not enough buffered data to safely switch up to a higher quality.

The manifest is configured with a suggested presentation delay of 9.6 seconds, which is exactly 3 segments (as one segment duration is 3.2 seconds). So at the point where the player needs to select a newly available segment, it has at most 6.4 seconds (=2 segments) of media buffered. The actual value I observed with my network was between 5 and 6 seconds.

The AdaptiveTrackSelection default configuration has two relevant values for switching up:

  • minDurationForQualityIncreaseMs (set to 10 seconds by default)
  • bufferedFractionToLiveEdgeForQualityIncrease (set to 0.75 by default) => effective threshold: 0.75 * 9.6 seconds = 7.2 seconds, or slightly less, depending on actual timing

The buffered duration needs to be more than one of these two values for the player to switch up to a higher quality. This never happens (as it is between 5 and 6 seconds), and so the player stays at the lower quality although the bandwidth measurements support switching up. If you change the setting of either value it'll work (for example bufferedFractionToLiveEdgeForQualityIncrease to 0.5 or minDurationForQualityIncreaseMs to 5 seconds).


The issue has come up multiple times now for live streams that have a reasonable configuration of playing about 3 segments away from the live edge (e.g. #9643). A live offset of 3 segments is also recommended by default for HLS. So we should ensure our default configuration allows such streams to increase the quality under good conditions.

There are two options to implement this:

  • We could change the default of bufferedFractionToLiveEdgeForQualityIncrease to 0.5. This would ensure that live playbacks with a configured live offset of 3 segments get a chance to switch up (because the actual buffered fraction will be between 0.5 and 0.66 under good network conditions).
  • We can change the definition of availableDurationUs in AdaptiveTrackSelection.updateSelectedTrack to only include the time that was actually available for buffering before calling this method. So in the example above, this would be 6.4 seconds instead of 9.6 seconds. This would ensure the default parameters work as intended again (0.75 *6.4 seconds = 4.8 seconds < 5-6 seconds of buffered duration).

@tonihei tonihei changed the title No adaptive track selection on Live DASH stream with AVC Improve default live quality increase settings to work with live streams playing at ~3 segments live offset. Dec 14, 2021
@cdongieux
Copy link
Author

Thanks a lot for your detailed answer!

icbaker pushed a commit that referenced this issue Dec 20, 2021
We check the fraction of the available duration we have already
buffered for live streams to see if we can increase the quality.
This fraction compares against the overall available media duration
at the time of the track selection, which by definition can't include
one of the availabe chunks (as this is the one we want to load next).

That means, for example, that for a reasonable live offset of 3 segments
we can at most reach a fraction of 0.66, which is less than our default
threshold of 0.75, meaning we can never switch up.

By subtracting one chunk duration from the available duration, we make
this comparison fair again and allow all live streams (regardless of
live offset) to reach up to 100% buffered data (which is above our
default value of 75%), so that they can increase the quality.

Issue: #9784
PiperOrigin-RevId: 416791033
tonihei added a commit to androidx/media that referenced this issue Jan 5, 2022
We check the fraction of the available duration we have already
buffered for live streams to see if we can increase the quality.
This fraction compares against the overall available media duration
at the time of the track selection, which by definition can't include
one of the availabe chunks (as this is the one we want to load next).

That means, for example, that for a reasonable live offset of 3 segments
we can at most reach a fraction of 0.66, which is less than our default
threshold of 0.75, meaning we can never switch up.

By subtracting one chunk duration from the available duration, we make
this comparison fair again and allow all live streams (regardless of
live offset) to reach up to 100% buffered data (which is above our
default value of 75%), so that they can increase the quality.

Issue: google/ExoPlayer#9784
PiperOrigin-RevId: 416791033
@tonihei
Copy link
Collaborator

tonihei commented Jan 31, 2022

Implemented by the commit above.

@tonihei tonihei closed this as completed Jan 31, 2022
@luckygoyal-bitmovin
Copy link

Hi @tonihei , I encountered same issue and tested the above fix commit. It improves the behaviour but I can still reproduce this for some cases where I have 4 seconds segments and 4 seconds as MPD update period. Below is my observation. Each MPD update contains 4 segments with a new segment added and oldest segment removed every MPD update(4 seconds).

  • The available duration is constantly > 12 seconds, so minDurationForQualityIncrease is returned as 10 seconds.
  • Buffered duration goes above 10 seconds threshold only when all segments in MPD are downloaded.
  • At this point there are no more segments to download. So even if download rate is much higher, and a higher bitrate rendition is selected to downloaded next but the switch up does not occur.
  • By the time next MPD update occurs(after approx. 3-4 seconds), one more segment is added and oldest one removed. The available duration remains > 10 seconds but buffered duration reduces < 10 seconds as approx. 3-4 seconds have elapsed since last download and the buffer level has come done to around 8 seconds. As buffer level(approx. 8 seconds) is < minDurationForQualityIncrease (10 seconds), so no switch up occurs. The newly added segment is downloaded for older quality.
  • Now buffer duration > 10 seconds after segment download but there is no next segment to download yet. The above sequence repeats with next manifest update e.i when segment becomes available bufferDuration < 10 seconds and when bufferDuration > 10 seconds, next segment is not available yet for download.

@jamesdavidholding
Copy link

jamesdavidholding commented Feb 8, 2022

Hi @tonihei with the above insight provided against this issue still being present for 4 Second segments with a 4 second update period can I ask would you be able to reopen this ticket to handle this permutation also?

Can I also ask in the interim of this issue not being merged and the above 4sec segment issue observed what would recommend for segment update and segment length to not encounter this issue? Do you see an issue with 2Sec segments with a 4second update for example.

@tonihei
Copy link
Collaborator

tonihei commented Feb 10, 2022

@luckygoyal-bitmovin

What you describe above is exactly the problem we solved I believe.

By the time next MPD update occurs(after approx. 3-4 seconds), one more segment is added and oldest one removed. The available duration remains > 10 seconds but buffered duration reduces < 10 seconds as approx. 3-4 seconds have elapsed since last download and the buffer level has come done to around 8 seconds.

In this scenario, we have bufferedDuration=8 sec, availableDuration= 11 sec, chunkDuration = 4 sec, so the calculated threshold will be minDurationForQualityIncrease = (11 - 4) * 0.75 = 5.25 sec and because bufferedDuration > minDurationForQualityIncrease, the player will be allowed to switch up.

If this doesn't happen, then the measured bandwidth is potentially not high enough for the up-switch? If that's not what you are seeing, please provide a concrete example for values used during the computation or reproduction steps with a specific stream.

@jamesdavidholding

can I ask would you be able to reopen this ticket to handle this permutation also
Do you see an issue with 2Sec segments with a 4second update for example.

The new logic works independent of the actual chunk duration, so there is no need to solve it for any particular permutation. But it generally assumes that the MPD update period is the same as the chunk duration to work nicely, so it wouldn't work as well with 2 sec segment and a 4 sec update period.

@luckygoyal-bitmovin
Copy link

thanks @tonihei for your reply.

In this scenario, we have bufferedDuration=8 sec, availableDuration= 11 sec, chunkDuration = 4 sec, so the calculated threshold will be minDurationForQualityIncrease = (11 - 4) * 0.75 = 5.25 sec and because bufferedDuration > minDurationForQualityIncrease, the player will be allowed to switch up.

Above does not happen as availableDuration = 11 seconds is greater than default minDurationForQualityIncreaseUs which is 10 seconds. Due to this isAvailableDurationTooShort is calculated as false and the method minDurationForQualityIncreaseUs returns default value(10 seconds) as as the minimum required duration which is greater than buffered duration(8 seconds). The first check for isAvailableDurationTooShort does not account chunkDurationUs and compares the full available duration with default minimum required duration value of 10 seconds.

private long minDurationForQualityIncreaseUs(long availableDurationUs, long chunkDurationUs) {
    boolean isAvailableDurationTooShort =
        availableDurationUs != C.TIME_UNSET
            && availableDurationUs <= minDurationForQualityIncreaseUs;
    if (!isAvailableDurationTooShort) {
      return minDurationForQualityIncreaseUs;
    }
    ..........

@tonihei
Copy link
Collaborator

tonihei commented Feb 10, 2022

I see, thanks for pointing this out! This edge case only applies if availableDurationUs > configured configured minDurationForQualityIncreaseUs, but the adjusted minDurationForQualityIncreaseUs < configured minDurationForQualityIncreaseUs. Will update the condition to always choose the minimum of adjusted and configured minDurationForQualityIncreaseUs.

@tonihei tonihei reopened this Feb 10, 2022
icbaker pushed a commit to androidx/media that referenced this issue Feb 17, 2022
We have two ways to choose the minDurationForQualityIncreaseMs value in
AdaptiveTrackSelection: use the configured value for non-live or when
enough buffered data is available, or use a fraction of the available
duration to allow switching when playing close to the live edge.

The decision point when to use which value isn't quite consistent because
we compare against availableDurationUs before making the adjustments. This
means there is range of values where no up-switching is possible despite
perfect buffering. Fix this by choosing the minimum of both values.

Issue: google/ExoPlayer#9784

#minor-release

PiperOrigin-RevId: 428474332
icbaker pushed a commit that referenced this issue Feb 17, 2022
We have two ways to choose the minDurationForQualityIncreaseMs value in
AdaptiveTrackSelection: use the configured value for non-live or when
enough buffered data is available, or use a fraction of the available
duration to allow switching when playing close to the live edge.

The decision point when to use which value isn't quite consistent because
we compare against availableDurationUs before making the adjustments. This
means there is range of values where no up-switching is possible despite
perfect buffering. Fix this by choosing the minimum of both values.

Issue: #9784

#minor-release

PiperOrigin-RevId: 428474332
icbaker pushed a commit that referenced this issue Feb 23, 2022
We have two ways to choose the minDurationForQualityIncreaseMs value in
AdaptiveTrackSelection: use the configured value for non-live or when
enough buffered data is available, or use a fraction of the available
duration to allow switching when playing close to the live edge.

The decision point when to use which value isn't quite consistent because
we compare against availableDurationUs before making the adjustments. This
means there is range of values where no up-switching is possible despite
perfect buffering. Fix this by choosing the minimum of both values.

Issue: #9784

#minor-release

PiperOrigin-RevId: 428474332
@icbaker icbaker closed this as completed Feb 23, 2022
icbaker pushed a commit to androidx/media that referenced this issue Feb 25, 2022
We have two ways to choose the minDurationForQualityIncreaseMs value in
AdaptiveTrackSelection: use the configured value for non-live or when
enough buffered data is available, or use a fraction of the available
duration to allow switching when playing close to the live edge.

The decision point when to use which value isn't quite consistent because
we compare against availableDurationUs before making the adjustments. This
means there is range of values where no up-switching is possible despite
perfect buffering. Fix this by choosing the minimum of both values.

Issue: google/ExoPlayer#9784

#minor-release

PiperOrigin-RevId: 428474332
@google google locked and limited conversation to collaborators Apr 25, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants