Skip to content

py/misc: Fix fallback implementation of mp_popcount. #17498

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 25, 2025

Conversation

dpgeorge
Copy link
Member

Summary

The fallback mp_popcount() was broken, missing a shift.

Testing

Tested using gcc 7.3.1 which does not have the popcount built-in and uses this fallback version. Without the fix, mpy-cross -march=rv32imc -X emit=native produces mpy files with corrupt RISC-V machine code. With the fix, mpy-cross output is correct.

Note: gcc 7.3.1 can be tested using docker image larsks/esp-open-sdk.

This bug was found as part of Octoprobe integration: octoprobe/testbed_micropython#27

@dpgeorge dpgeorge added the py-core Relates to py/ directory in source label Jun 14, 2025
@dpgeorge
Copy link
Member Author

@agatti FYI. Please can you double check this popcount implementation and my fix here, thanks!

Copy link

codecov bot commented Jun 14, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.57%. Comparing base (e57aa7e) to head (6fee099).
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #17498   +/-   ##
=======================================
  Coverage   98.57%   98.57%           
=======================================
  Files         169      169           
  Lines       21968    21968           
=======================================
  Hits        21654    21654           
  Misses        314      314           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

Copy link
Contributor

@jepler jepler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I plugged this into cbmc and it is equivalent to a naive for-loop popcnt implementation for all 32-bit unsigned integers. The original formulation encounters a verification error.

@hmaerki
Copy link
Contributor

hmaerki commented Jun 14, 2025

I run the octoprobe tests against this PR: https://reports.octoprobe.org/github_selfhosted_testrun_76/octoprobe_summary_report.html

These are the results: tests_mpy-cross_native.txt

Now all md5 equal: The error has gone!

@agatti
Copy link
Contributor

agatti commented Jun 15, 2025

I don't have access to any of my boards/dev machines until the end of the month to see what's going on there (along with my copy of "Hacker's Delight" to double check things against), but if this PR is reported by multiple sources to fix the issue then please go ahead!

I took that code from an old project of mine running on a ColdFire board that's been working for a few years now, so now I wonder whether this works due to code consuming that value that either does the shift itself or masks it out and fills the other bits with something else - no idea until I take a look.

Tested using gcc 7.3.1 which does not have the popcount built-in and uses
this fallback version.  Without the fix, mpy-cross produces mpy files with
corrupt RISC-V machine code.  With the fix, mpy-cross output is correct.

Signed-off-by: Damien George <[email protected]>
@dpgeorge dpgeorge force-pushed the py-misc-fix-mp-popcount-impl branch from 57d2fe8 to 6fee099 Compare June 25, 2025 01:37
@dpgeorge dpgeorge merged commit 6fee099 into micropython:master Jun 25, 2025
65 checks passed
@dpgeorge dpgeorge deleted the py-misc-fix-mp-popcount-impl branch June 25, 2025 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
py-core Relates to py/ directory in source
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants