Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent dump results of jq #2367

Closed
shao-hua-li opened this issue Nov 7, 2021 · 5 comments · Fixed by #2661
Closed

Inconsistent dump results of jq #2367

shao-hua-li opened this issue Nov 7, 2021 · 5 comments · Fixed by #2661
Labels
Milestone

Comments

@shao-hua-li
Copy link

Hi there,

I found that for large floating numbers, when compiling with gcc and clang, jq would emit different outputs.

For example, for echo '{"a": 1E9999999999}' | jq .,
gcc compiled jq would emit:

{
  "a": ((double)1.79769313486231570814527423731704357e+308L)
}

while clang compiled jq would emit:

{
  "a": 1.7976931348623157e+308
}

I think the cause is probably different interpretation of DBL_MAX / DBL_MIN in gcc and clang. But to ensure the correctness, robustness, and consistency of jq, it's important to keep programs' behaviour consistent across main-stream compilers.

@itchyny itchyny added the bug label Jun 3, 2023
@itchyny itchyny added this to the 1.7 release milestone Jun 25, 2023
@wader
Copy link
Member

wader commented Jun 28, 2023

Reproduced with debian 12 and gcc:

root@6eeb8d61a151:/Users/wader/src/jq# gcc --version
gcc (Debian 12.2.0-14) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

root@6eeb8d61a151:/Users/wader/src/jq# echo '{"a": 1E999999999}' | ./jq
{
  "a": 1E+999999999
}
root@6eeb8d61a151:/Users/wader/src/jq# echo '{"a": 1E9999999999}' | ./jq
{
  "a": ((double)1.79769313486231570814527423731704357e+308L)
}

@wader
Copy link
Member

wader commented Jun 28, 2023

Seems to boil down to this difference:

root@6eeb8d61a151:/Users/wader/src/jq# echo -e '#include <float.h>\n#define STR(x) #x\n#define XSTR(x) STR(x)\nXSTR(DBL_MAX) DBL_MAX __DBL_MAX__' | clang -E -
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 361 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2
# 1 "/usr/lib/llvm-14/lib/clang/14.0.6/include/float.h" 1 3
# 2 "<stdin>" 2


"1.7976931348623157e+308" 1.7976931348623157e+308 1.7976931348623157e+308
root@6eeb8d61a151:/Users/wader/src/jq# echo -e '#include <float.h>\n#define STR(x) #x\n#define XSTR(x) STR(x)\nXSTR(DBL_MAX) DBL_MAX __DBL_MAX__' | cpp
# 0 "<stdin>"
# 0 "<built-in>"
# 0 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 0 "<command-line>" 2
# 1 "<stdin>"
# 1 "/usr/lib/gcc/x86_64-linux-gnu/12/include/float.h" 1 3 4
# 2 "<stdin>" 2


"((double)1.79769313486231570814527423731704357e+308L)" ((double)1.79769313486231570814527423731704357e+308L) ((double)1.79769313486231570814527423731704357e+308L)

DBL_MAX_STR is XSTR(DBL_MAX) and is used here https://github.com/jqlang/jq/blob/master/src/jv.c#L665
__DBL_MAX__ i think is defined by the compiler and is different.

@wader
Copy link
Member

wader commented Jun 28, 2023

I wonder what the backwards compatibiltiy is about at https://github.com/jqlang/jq/blob/master/src/jv.c#L660-L667 to preserve how infinite/0 is represented etc? was added as part of #1752 and there are some tests about being compatible but not sure they made me wiser, maybe @leonid-s-usov remembers?

Otherwise i guess we could just skip the decNumberIsInfinite check and it will return plit->literal_data?

Other alternatives could be to runtime string format the constants at startup or maybe define our own min/max constants somehow?

@itchyny
Copy link
Contributor

itchyny commented Jun 28, 2023

Removing these statements outputs Infinity (not JSON!), not the input literal 1E9999999999.

@wader
Copy link
Member

wader commented Jun 28, 2023

That's not good. So that is the case when the decNumber code thinks it's too large even

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants