Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seek at a chunk level when seeking to chunk start positions #1031

Merged
merged 2 commits into from
Jan 31, 2024

Conversation

garethfenn
Copy link

This avoids issues that can arise due to the slight discrepancies between chunk start times (obtained from the manifest of segment index) and the timestamps of the samples contained within those chunks.

This PR adds the equivalent logic of google/ExoPlayer@a4114f5, for HLS TS streams. I have copied the description from that commit.

It is apparent when SeekParameters are used, added for HLS in google/ExoPlayer#9536, that the video sample chosen is not the first keyframe in the chunk, rather the last keyframe of the preceding chunk.

It is ultimately down to using "findSampleBefore()":

private int findSampleBefore(int relativeStartIndex, int length, long timeUs, boolean keyframe) {

It can be reproduced using the Exoplayer Demo "HLS Apple multivariant playlist advanced (TS)" stream. The first video keyframe PTS in the chunks are offset back by 16.666ms, hence not snapped to. The previous keyframe is used,
some 1983.334ms out of sync with the chunk PTS and that of the audio PTS.

As far as I can tell, it is only visually apparent for tunneled playback where HW AV sync is used, resulting in a jarring acceleration of video decode to catch up with audio.

@garethfenn garethfenn marked this pull request as ready for review January 26, 2024 16:17
@tonihei
Copy link
Collaborator

tonihei commented Jan 26, 2024

Thanks for the contribution! The general idea sounds great, we probably need to do some testing to verify it works as expected, but otherwise I think we can merge this change.

Can I ask you to do two changes though?

  • The PR needs to target the main branch (not release)
  • SampleQueue.seekTo(index) starts supplying samples from this specific index without checking whether it is a keyframe. This works for DASH because chunks are by spec required to start with a keyframe. HLS chunks have no such requirement though unless they are specifically marked as independent. So we should check whether mediaPlaylist.hasIndependentSegments is true and only then take the go-to-first-sample-of-chunk shortcut.

@garethfenn
Copy link
Author

Thanks @tonihei. Apologies, I'll correct the target branch on Monday.

Regarding HLS independent chunks, I'd first like to highlight that there is already a check here:

The result is SeekParameters are effectively disabled for content that doesn't declare the independent segments HLS tag. getAdjustedSeekPositionUS() does not adjust the seek position to the start of the chunk, and so this added code is not used. That being said, let me know if a secondary check is still required.

@garethfenn garethfenn changed the base branch from release to main January 29, 2024 09:03
@tonihei
Copy link
Collaborator

tonihei commented Jan 29, 2024

The result is SeekParameters are effectively disabled for content that doesn't declare the independent segments HLS tag. getAdjustedSeekPositionUS() does not adjust the seek position to the start of the chunk, and so this added code is not used. That being said, let me know if a secondary check is still required.

Someone could still manually seek to exactly the start of a chunk even without adjustments via seek parameters. Given chunk durations are often 'round' numbers, this isn't too unlikely actually.

This avoids issues that can arise due to the slight discrepancies between
chunk start times (obtained from the manifest of segment index) and
the timestamps of the samples contained within those chunks.
@garethfenn
Copy link
Author

garethfenn commented Jan 30, 2024

Someone could still manually seek to exactly the start of a chunk even without adjustments via seek parameters. Given chunk durations are often 'round' numbers, this isn't too unlikely actually.

Sure. I've added a conditional on the chunkSource having independent segments. This required adding a getter to HlsChunkSource. Let me know if this was a reasonable thing to do.

@tonihei
Copy link
Collaborator

tonihei commented Jan 30, 2024

That should work, thanks! I'm going to test this a bit to see what happens.

@tonihei
Copy link
Collaborator

tonihei commented Jan 30, 2024

Also added a test case for this. Thanks again, we'll merge this change soon!

@tonihei tonihei self-assigned this Jan 30, 2024
@copybara-service copybara-service bot merged commit 45bd5c6 into androidx:main Jan 31, 2024
1 check passed
SheenaChhabra pushed a commit that referenced this pull request Feb 9, 2024
PiperOrigin-RevId: 603008793
(cherry picked from commit 45bd5c6)
@androidx androidx locked and limited conversation to collaborators Apr 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants