Skip to content

gh-95245: Document Py_TPFLAGS_PREHEADER #135861

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

gh-95245: Document Py_TPFLAGS_PREHEADER #135861

wants to merge 6 commits into from
+15 −0

Conversation

sergey-miryanov
Copy link
Contributor

@sergey-miryanov sergey-miryanov commented Jun 23, 2025 •
edited by encukou
Loading

I see following usages of Py_TPFLAGS_PREHEADER:

  1. Calculate size of preheader (but preheader also contains GC info)
  2. Check if types are support managed dict or managed weakref

I checked all linked PR from #95245 and can't find any explanation. I also checked https://github.com/python/cpython/blob/main/Objects/object_layout.md.

@encukou Please take a look.

@bedevere-app bedevere-app bot mentioned this pull request Jun 23, 2025
Open
38 tasks
ZeroIntensity
ZeroIntensity reviewed Jun 23, 2025
View reviewed changes
Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should include documentation on the type object page too.

@encukou encukou changed the title gh-135755: Document Py_TPFLAGS_PREHEADER gh-95245: Document Py_TPFLAGS_PREHEADER Jun 24, 2025
@bedevere-app bedevere-app bot mentioned this pull request Jun 24, 2025
Open
@sergey-miryanov
Copy link
Contributor Author

@ZeroIntensity Thanks for review! Fixed (I hope :)

ZeroIntensity
ZeroIntensity reviewed Jun 24, 2025
View reviewed changes
Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that if this is internal-only, it would be better to completely drop it from public headers rather than document it as such.

@ZeroIntensity ZeroIntensity added skip news docs Documentation in the Doc dir labels Jun 24, 2025
@github-project-automation github-project-automation bot moved this to Todo in Docs PRs Jun 24, 2025
@ZeroIntensity
Copy link
Member

I think that if this is internal-only, it would be better to completely drop it from public headers rather than document it as such.

(Don't do this yet, by the way. Let's come to a consensus in the issue.)

@sergey-miryanov
Copy link
Contributor Author

As no comments from @iritkatriel or @markshannon I propose to make it internal as @ZeroIntensity suggested and rename it to _Py_TPFLAGS_PREHEADER.

@encukou WDYT?

@encukou
Copy link
Member

encukou commented Jul 7, 2025

Per PEP-387, removing (or renaming) it probablyneeds a deprecation cycle. Or is there evidence that no existing code uses it?

@markshannon
Copy link
Member

Since the tp_flags flags must be public, it seemed sensible to put Py_TPFLAGS_PREHEADER in the same header.
I have no objection to moving it.

It might be better to rename it to Py_TPFLAGS_VM_MANAGED_FIELDS to focus on the semantics, not the layout, and keep it public?

@sergey-miryanov
Copy link
Contributor Author

sergey-miryanov commented Jul 7, 2025 •
edited
Loading

Per PEP-387, removing (or renaming) it probablyneeds a deprecation cycle. Or is there evidence that no existing code uses it?

https://grep.app/search?q=Py_TPFLAGS_PREHEADER shows close to nothing of usage. Also, I'm downloading top-pypi packages to checks usage (as described here https://hugovk.dev/blog/2022/how-to-search-5000-python-projects/), will report as it finished.

@sergey-miryanov
Copy link
Contributor Author

It might be better to rename it to Py_TPFLAGS_VM_MANAGED_FIELDS to focus on the semantics, not the layout, and keep it public?

@markshannon WDYT about wordings in this PR? Can you suggest some improvements?

@sergey-miryanov
Copy link
Contributor Author

sergey-miryanov commented Jul 7, 2025 •
edited
Loading

will report as it finished.

Searched over 13k+ packages:

➜ .\python.exe ../misc/cpython/search_pypi_top.py -q ..\top-pypi\output\  "Py_TPFLAGS_PREHEADER"
..\top-pypi\output\recordclass-0.23.1.tar.gz: recordclass-0.23.1/lib/recordclass/_dataobject.c: tp->tp_flags &= ~Py_TPFLAGS_PREHEADER;

Time: 0:01:10.586519
Found 1 matching lines in 1 projects

➜ ( Get-ChildItem ..\top-pypi\output\ | Measure-Object ).Count;
13961

@encukou
Copy link
Member

encukou commented Jul 7, 2025

It might be better to rename it to Py_TPFLAGS_VM_MANAGED_FIELDS to focus on the semantics, not the layout, and keep it public?

If it's public, it's best not to rename it.

encukou
encukou reviewed Jul 7, 2025
View reviewed changes
ZeroIntensity
ZeroIntensity reviewed Jul 7, 2025
View reviewed changes
Doc/c-api/typeobj.rst
Currently, this macro is equivalent to
:c:expr:`Py_TPFLAGS_MANAGED_DICT | Py_TPFLAGS_MANAGED_WEAKREF`.

This macro is intended for internal usage only; prefer using individual
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still not a big fan of documenting this as internal-only. If it's in the public headers and documentation, it should be at least usable. How about we say that its value is unstable, so we can change it later?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about this?
This macro is intended for internal usage only and may be changed without any notes

Is it OK or you prefer to completely remove the internal usage part?

@ZeroIntensity
Copy link
Member

If it's public, it's best not to rename it.

I think it's probably safe to rename it, at least for 3.14. Based on a code search, we're the only ones who use it (all the non-CPython mentions are people copying our headers).

@sergey-miryanov
Copy link
Contributor Author

sergey-miryanov commented Jul 7, 2025 •
edited
Loading

If it's public, it's best not to rename it.

I think it's probably safe to rename it, at least for 3.14. Based on a code search, we're the only ones who use it (all the non-CPython mentions are people copying our headers).

Only recordclass uses it - #135861 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@encukou encukou encukou left review comments

@ZeroIntensity ZeroIntensity ZeroIntensity left review comments

Assignees
No one assigned
Labels
awaiting review docs Documentation in the Doc dir skip news
Projects
Docs PRs
Status: Todo
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@sergey-miryanov @ZeroIntensity @encukou @markshannon