pgml chat with history + additional functionality #1047
Conversation
santiatpml
commented
Oct 3, 2023
- Chat history in a separate collection
- Keeps track of users, conversations, and interfaces
- Prompt engineering on system and base prompts to follow specific instructions
- Options for language, personality, programming language etc.
pgml-apps/pgml-chat/pyproject.toml
Outdated
| @@ -11,14 +11,21 @@ packages = [{include = "pgml_chat"}] | |||
| python = ">=3.8,<4.0" | |||
| openai = "^0.27.8" | |||
| rich = "^13.4.2" | |||
| pgml = "^0.9.0" | |||
| pgml = {version = "0.9.4", source = "testpypi"} | |||
There was a problem hiding this comment.
should we wait for a release?
There was a problem hiding this comment.
I'm planning to push pgml-chat to pypi after pgml 0.9.4 is released to pypi. In the meantime, want to update master.
pgml-apps/pgml-chat/.gitignore
Outdated
| #.idea/ | ||
|
|
||
| pgml_chat/pgml_playground.py |
There was a problem hiding this comment.
Now I'm curious what's in here.
| messages = [] | ||
| messages.append({"role": "system", "content": system_prompt}) | ||
|
|
||
| chat_history_messages = await chat_collection.get_documents( { |
There was a problem hiding this comment.
An implementation more inline with the thesis would combine the first call with this call in a CTE, and follow up with the next call all in 1.
| async def generate_response( | ||
| messages, openai_api_key, temperature=0.7, max_tokens=256, top_p=0.9 | ||
| ): | ||
| openai.api_key = openai_api_key | ||
| log.debug("Generating response from OpenAI API: " + str(messages)) | ||
| response = openai.ChatCompletion.create( | ||
| model="gpt-3.5-turbo", | ||
| # model="gpt-3.5-turbo-16k", | ||
| model="gpt-4", |
There was a problem hiding this comment.
OPENAI_API_KEY is an environment variable. User pays for it.
| WHERE metadata @> '{\"interface\" : \"cli\"}'::JSONB | ||
| AND metadata @> '{\"role\" : \"user\"}'::JSONB | ||
| OR metadata @> '{\"role\" : \"assistant\"}'::JSONB | ||
| ORDER BY metadata->>'timestamp' DESC LIMIT %d""" % ( |
There was a problem hiding this comment.
Why is the timestamp in the metadata, instead of a field on the record?
There was a problem hiding this comment.
We have a field on the record. I insert user message and the response together and each of them will have different timestamps. I could also order by the field on the record.
| @@ -0,0 +1,114 @@ | |||
| from pgml import Collection, Builtins, Pipeline | |||
There was a problem hiding this comment.
I see what's in the playground now 👀