You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Floating point errors can cause incorrect code to generate. This depends on the language, but as an example we noticed that encoding latitude 1, longitude 1 produces an X for the 11th digit.
The 10 digit code latitude and longitude ranges are the same - 1.0 to 1.000125. In the 11 digit sub-grid, the characters are assigned as:
Longitude 1.0 to 1.0003125
Longitude 1.0003125 to 1.000625
Longitude 1.000625 to 1.0009375
Longitude 1.0009375 to 1.000125
Latitude 1.0001 to 1.000125
R
V
W
X
Latitude 1.000075 to 1.0001
J
M
P
Q
Latitude 1.000050 to 1.000075
C
F
G
H
Latitude 1.000025 to 1.000050
6
7
8
9
Latitude 1.0 to 1.000025
2
3
4
5
So for latitude 1.0, longitude 1.0 we want to get 2. But we are getting X.
This seems to be a problem in the encodeGrid functions where we try to work out which column the longitude is in by getting the remainder of dividing the longitude by the grid size (0.000125).
You can check this in Javascript by doing:
1.0 % 0.000125
It should be zero, but in my browser I'm seeing 0.00012499999999997919 which is nonsense.
Instead of directly comparing values in that function, we may need to use "close enough" comparisons, where if numbers are within 1E-10 of each other that's good enough.
Another strategy is instead of doing the remainder of 181.5 % 0.000125, we ditch the degrees and use 0.5 % 0.000125.
The solution in each case is probably language dependent.
The text was updated successfully, but these errors were encountered:
Floating point errors can cause incorrect code to generate. This depends on the language, but as an example we noticed that encoding latitude 1, longitude 1 produces an X for the 11th digit.
The 10 digit code latitude and longitude ranges are the same - 1.0 to 1.000125. In the 11 digit sub-grid, the characters are assigned as:
So for latitude 1.0, longitude 1.0 we want to get 2. But we are getting X.
This seems to be a problem in the encodeGrid functions where we try to work out which column the longitude is in by getting the remainder of dividing the longitude by the grid size (0.000125).
You can check this in Javascript by doing:
1.0 % 0.000125
It should be zero, but in my browser I'm seeing 0.00012499999999997919 which is nonsense.
Instead of directly comparing values in that function, we may need to use "close enough" comparisons, where if numbers are within 1E-10 of each other that's good enough.
Another strategy is instead of doing the remainder of 181.5 % 0.000125, we ditch the degrees and use 0.5 % 0.000125.
The solution in each case is probably language dependent.
The text was updated successfully, but these errors were encountered: