Skip to content

feat: Add Twelve Labs Marengo Integration #282

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

wbrennan899
Copy link

@wbrennan899 wbrennan899 commented Jun 25, 2025

Description

This PR adds TwelveLabs model provider integration to the Strands SDK, enabling both semantic video search and video understanding capabilities. The implementation provides two specialized model providers:

  • TwelveLabsSearchModel: Semantic video search using the Marengo model
  • TwelveLabsPegasusModel: Video understanding and Q&A using the Pegasus model

Key Features

TwelveLabsSearchModel:

  • Search TwelveLabs video indexes using natural language queries
  • Search video and audio content with configurable modalities (visual, audio)
  • Support for both clip-level and video-grouped results

TwelveLabsPegasusModel:

  • Upload and index videos with automatic deduplication
  • Chat with existing videos in your TwelveLabs index for Q&A

Related Issues

Documentation PR

Type of Change

New feature

Testing

How have you tested the change?

  • I ran hatch run prepare - All linting and formatting checks pass
  • Unit tests: 26/26 tests pass for the TwelveLabs model implementation
  • Integration with existing SDK patterns verified
  • Error handling tested for all TwelveLabs API error codes
  • Type checking with mypy passes
  • Code formatting with ruff passes

The integration tests failed due to missing API credentials for AWS/OpenAI services, which is expected and unrelated to the TwelveLabs implementation.

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@awsarron awsarron self-assigned this Jun 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants