feat: hybrid pydantic support for both v1 and v2 #1652

samsja · 2023-06-15T09:27:55Z

Context

this PR allow support for pydantic v2 and v1 at the same time

Note: it is still very early progress

Signed-off-by: samsja <[email protected]>

docarray/typing/abstract_type.py

Signed-off-by: samsja <[email protected]>

github-actions · 2023-06-22T07:05:22Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

docarray/array/doc_vec/io.py

docarray/base_doc/mixins/io.py

docarray/documents/audio.py

docarray/documents/image.py

docarray/documents/mesh/vertices_and_faces.py

JohannesMessner

Okay first pass done, I have few questions and will probably make a more thorough pass tomorrow!

JohannesMessner · 2023-09-06T13:16:05Z

docarray/utils/_internal/pydantic.py

+    from pydantic.validators import bytes_validator
+
+else:
+    from pydantic.v1.validators import bytes_validator


Am I understanding correctly that we are falling back to the bytes validator of v1 even if pydantic v2 is used? What is the reason behind this?

JohannesMessner · 2023-09-06T13:17:10Z

docarray/utils/create_dynamic_doc_class.py

+    if is_pydantic_v2:
+        raise NotImplementedError(
+            'This method is not supported in Pydantic 2.0. Please use Pydantic 1.8.2 or lower.'
+        )
+


Why is this?

the method we used has been removed from pydantic v2, will have to find a fix later

docarray/base_doc/any_doc.py

JohannesMessner · 2023-09-06T13:20:27Z

docarray/base_doc/doc.py

@@ -36,6 +45,13 @@

    from docarray.array.doc_vec.column_storage import ColumnStorageView

+if is_pydantic_v2:
+
+    IncEx: typing_extensions.TypeAlias = (


What does IncEx stand for?

this is the named used in pydantic function signature. Not even sure what this means

JohannesMessner · 2023-09-06T13:21:37Z

docarray/base_doc/doc.py

-        json_encoders = {AbstractTensor: lambda x: x}
+    if is_pydantic_v2:
+
+        class Config:


Should we still have this Config inner class, it is not recommended anymore in v2, right?
Also, where did the orjson stuff and the extra protobuf setting go?

good question,

basically now in pydantic v2 this way is "deprecated". But at the same time, it is still working. Therefore if we keep it as a class it means we don't have to repeat the config for pydantic v2. Might be easier to maintain ...

Also, where did the orjson stuff and the extra protobuf setting go?

This is handle differently with pydantic v2 and therefore not needed

hmm, using deprecated stuff is not great. Does it raise a warning?

it does indeed raise a warning ...

docarray/base_doc/doc.py

github-actions · 2023-09-07T10:00:13Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-09-07T10:02:33Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-09-07T11:29:53Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

Co-authored-by: Johannes Messner <[email protected]> Signed-off-by: samsja <[email protected]>

github-actions · 2023-09-07T11:31:33Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-09-07T11:38:08Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-09-07T11:38:51Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-09-07T12:34:20Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

JohannesMessner

the only thing that feels icky to me is using the deprecated Config inner class

github-actions · 2023-09-08T07:34:46Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-09-08T08:05:51Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

samsja · 2023-09-08T08:06:00Z

I will go with @JohannesMessner proposal and switch to ConfigDict instead of class

github-actions · 2023-09-08T08:17:41Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-09-08T08:39:13Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-09-08T08:42:07Z

📝 Docs are deployed on https://ft-feat-full-pydantic-v2-support--jina-docs.netlify.app 🎉

feat: init commit on adding v2 support

e26c6c5

Signed-off-by: samsja <[email protected]>

github-actions bot added size/s area/core area/typing labels Jun 15, 2023

feat: make some progress

30a8c17

Signed-off-by: samsja <[email protected]>

github-actions bot added size/m and removed size/s labels Jun 15, 2023

samsja changed the title ~~feat: init commit on adding v2 support~~ feat: hybrid pydantic support for both v1 and v2 Jun 15, 2023

fix: fix test update

ee347b0

Signed-off-by: samsja <[email protected]>

JoanFM reviewed Jun 15, 2023

View reviewed changes

docarray/typing/abstract_type.py Outdated Show resolved Hide resolved

fix: fix refactoring validation

64216c7

Signed-off-by: samsja <[email protected]>

github-actions bot added size/l component/array and removed size/m labels Jun 15, 2023

samsja added 9 commits June 15, 2023 14:46

fix: fix ndarray and doclist

8989d82

Signed-off-by: samsja <[email protected]>

fix: move to var

e2082d9

Signed-off-by: samsja <[email protected]>

fix: fix some stuff

c20f49b

Signed-off-by: samsja <[email protected]>

fix: fix some stuff on v1

b55005f

Signed-off-by: samsja <[email protected]>

feat: pass half of the test

1d7097c

Signed-off-by: samsja <[email protected]>

fix: add schema to doc vec

addf361

Signed-off-by: samsja <[email protected]>

feat: fix anyurl

168163b

Signed-off-by: samsja <[email protected]>

fix: remove useles try catch

a7d30ed

Signed-off-by: samsja <[email protected]>

refactor: use _docarray_fields everywhere

979edc7

Signed-off-by: samsja <[email protected]>

github-actions bot added the area/testing label Jun 19, 2023

samsja added 3 commits June 19, 2023 16:32

fix: fix is required

0d1e194

Signed-off-by: samsja <[email protected]>

fix: fix validation of any url

f370888

Signed-off-by: samsja <[email protected]>

fix: make dict and json pydantic v1 only for now

dd0f96a

Signed-off-by: samsja <[email protected]>

github-actions bot added size/xl and removed size/l labels Jun 22, 2023

JoanFM requested changes Sep 6, 2023

View reviewed changes

JohannesMessner reviewed Sep 6, 2023

View reviewed changes

fix: docstring validate

c259b09

fix: docstring validate

02b2b61

samsja and others added 2 commits September 7, 2023 13:30

fix: put back cast

cbf7a87

feat: apply johannes suggestion

5e2378e

Co-authored-by: Johannes Messner <[email protected]> Signed-off-by: samsja <[email protected]>

samsja added 2 commits September 7, 2023 13:36

feat: add comment

19e444b

feat: add comment

a16018a

samsja requested a review from JohannesMessner September 7, 2023 12:34

fix: skip docstrng tet for pydantic v2 for now

863e0b8

JohannesMessner approved these changes Sep 7, 2023

View reviewed changes

JoanFM approved these changes Sep 7, 2023

View reviewed changes

fix: skip docstrng tet for pydantic v2 for now

d7a7a49

samsja force-pushed the feat-full-pydantic-v2-support branch from 12a1dc6 to d7a7a49 Compare September 8, 2023 08:38

samsja merged commit 715252a into main Sep 10, 2023

samsja deleted the feat-full-pydantic-v2-support branch September 10, 2023 21:06

JoanFM mentioned this pull request Oct 2, 2023

Release Notes #1817

Closed

feat: hybrid pydantic support for both v1 and v2 #1652

feat: hybrid pydantic support for both v1 and v2 #1652

Uh oh!

Conversation

samsja commented Jun 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Uh oh!

Uh oh!

github-actions bot commented Jun 22, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JohannesMessner left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Sep 7, 2023

Uh oh!

github-actions bot commented Sep 7, 2023

Uh oh!

github-actions bot commented Sep 7, 2023

Uh oh!

github-actions bot commented Sep 7, 2023

Uh oh!

github-actions bot commented Sep 7, 2023

Uh oh!

github-actions bot commented Sep 7, 2023

Uh oh!

github-actions bot commented Sep 7, 2023

Uh oh!

JohannesMessner left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Sep 8, 2023

Uh oh!

github-actions bot commented Sep 8, 2023

Uh oh!

samsja commented Sep 8, 2023

Uh oh!

github-actions bot commented Sep 8, 2023

Uh oh!

github-actions bot commented Sep 8, 2023

Uh oh!

github-actions bot commented Sep 8, 2023

Uh oh!

Uh oh!

samsja commented Jun 15, 2023 •

edited

Loading