This fixes character handling on platforms with 16-bit wchar_t (notably, Windows), which was broken (in different ways) on both CPython and PyPy.
Fixes #552
This allows surrogates anywhere in the input, compatible with the json module from the standard library.
This also refactors two interfaces:
- The `PyUnicode` to `char*` conversion is moved into its own function, separated from the `JSONTypeContext` handling, so it can be reused for other things in the future (e.g. indentation and separators) which don't have a type context.
- Converting the `char*` output to a Python string with surrogates intact requires the string length for `PyUnicode_Decode` & Co. While `strlen` could be used, the length is already known inside the encoder, so the encoder function now also takes an extra `size_t` pointer argument to return that and no longer NUL-terminates the string. This also permits output that contains NUL bytes (even though that would be invalid JSON), e.g. if an object's `__json__` method return value were to contain them.
Fixes #156
Fixes #447
Fixes #537
Supersedes #284
Fix segfaults on musl libc when ultrajson runs in a thread. On musl libc
the default thread stack size is only 80k so allocating a 128k buffer on
stack will guarantee a crash. There seems not to be any evident
performance benefit using big buffer on stack either so we just reduce
the default.
fixes #254
with this ujson matches the builtin json behavior for NaN and Inf.
if a user wants to retain the old behavior they can pass allow_nan=False
to ensure strict json compatibility.
To fix issues with floating-point precision we've made use of Google's
double-conversion lib to handle conversions of doubles to and from strings.
In addition to fixing our precision problems this will improve double
encoding by 4-5x. Decoding is however slightly slower according to the
benchmarks - but accurate at least.
This change removes the double_precision encoding option and the
precise_float decoding option.
Previously a None dict item key would be outputted in JSON as "None".
To better align with the standard json module this was changed to output
"null". There's no proper representation of null object keys in JSON so
this is implementation specific but it seems more natural to follow
suit when it can be done without a significant performance hit.
Added and used branch prediction macros (LIKELY/UNLIKELY) as well.
