For some reason you can't yield from in an async generator. Consequently, we have to convert to individual yields.

Can we have a helper to do this - presumedly we'll need to collect events a lot? (maybe not though)

Done. I defined this helper in conftest.py and named it alist. I renamed my existing alist helper to agenerator.

Is it possible to call stream_async here or unify it so that they're both just plumbing to the same place?

And with #289 we'll now removing the callback_handler override, so I think this gets simpler overall. Maybe coordinate with @Unshure to see what else simplifies down

Done.

💭 I wonder if the Async Iterator should be emitting something like this anyway (allowing us to unify the __call__ method even more)

See revision.

where does items come from here? is this python magic?

This fixture is returning a function and so items is just the argument you pass to the call. For example:

def test_me(alist):
   await alist([1, 2, 3])

[1, 2, 3] is items. Note though, this has been renamed to agenerator in my revision.

Is it worth making this a union and allowing a generator or an async generator? That way we don't have to fake it for providers that aren't fully async - the SDK would handle that instead?

It would also simplify testing/mocking I think?

As an added benefit, it would more easily let us identify what gaps we have with providers (those that aren't async would be more obvious)

Downside is that it's more work in the SDK, but IMHO I think it might be a valid tradeoff. The SDK accepts both async/non-async and transforms it appropriately. But it emits only async (and wrappers for non-async use cases)

Note that there is overhead in running asyncio.run. It creates an event loop on every invocation. However, this overhead should be negligible compared to the underlying LLM calls.

Additional call out, you cannot call asyncio.run in an async context and so this could technically be a breaking change for some customers. As an example:

async def func_a():
    return 1

def func_b():
    asyncio.run(func_a())
    
asyncio.run(func_b())

The above will result in the following exception:

RuntimeError: asyncio.run() cannot be called from a running event loop

As a fix, we just ask our async callers to use stream_async instead of __call__. Added a note about this for #253.

I wanted to avoid having to define a helper function to invoke stream_async. Unfortunately, asyncio.run cannot call async generators. There is no equivalent method for this either in asyncio. Consequently, I define acall to return (instead of yield) the agent result.