mirror/ ultrajson
1
0
Fork 0
mirror of https://github.com/ultrajson/ultrajson.git synced 2024-05-29 02:26:09 +02:00
Code Issues
750 Commits 3 Branches 27 Tags 8.6 MiB
Commit Graph

110 Commits

Author SHA1 Message Date
JustAnotherArchivist 66060a0fab Add and fix debug memory checks for all buffer appends on encoding 
In DEBUG mode, this ensures that all buffer appends are safe.
It also refactors direct `memcpy` calls into a helper `Buffer_memcpy` function that ensures correct buffer pointer movement and has a similar safety check.
 2022-04-24 13:58:00 -07:00
JustAnotherArchivist f4d2c87ab6 Refactor buffer reservations to ensure sufficient space on all additions 
* Removed the reservations in Buffer_EscapeStringUnvalidated and Buffer_EscapeStringValidated as those are not needed and may hide other bugs.
* Debug check in Buffer_EscapeStringValidated was triggering incorrectly.
* The reservation on JT_RAW was much larger than necessary; the value is copied directly, so the factor six is not needed, and this may hide other bugs.
* Explicit accurate reservations everywhere else.
 2022-04-05 21:04:39 +01:00
Brénainn Woodsend 5875168c41 Fix some more seg-faults on encoding. 2022-04-05 21:04:39 +01:00
Brénainn Woodsend 1a39406b3a Remove the hidden JSON_NO_EXTRA_WHITESPACE compile knob. 
Unsetting it can lead to seg-faults. I don't think it's worth having to fix and
then test this undocumented permutation.
 2022-04-05 21:04:39 +01:00
Brénainn Woodsend 61dd6f19e8 Fix unchecked buffer overflows (CVE-2021-45958). 
Add a few extra memory reserve calls to account for the extra space that
indentation needs.

These kinds of memory issues are hard to spot because the buffer is resized in
powers of 2 meaning that a miscalculation would only show any symptoms if the
required buffer size is estimated to be just below a 2 power but is actually
just above. Add a debug mode which replaces the 2 power scheme with reserving
only the memory explicitly requested and adds some overflow checks.
 2022-04-05 21:04:39 +01:00
joncrall 13aa30e152
Fix nan bug in pandas port 2022-04-04 13:56:19 -04:00
joncrall f090103b31
NaN and Inf in loads - Port of Pandas #30295 2022-04-04 13:56:11 -04:00
JustAnotherArchivist f9aa23b5e6 Remove dead code that used to handle the separate int type in Python 2 2022-02-20 10:59:11 +00:00
JustAnotherArchivist 7f269a4818 Clean up iterators, type contexts, and recursion level on errors 2022-02-16 08:17:47 +00:00
JustAnotherArchivist 4bd21e2483 Fix exceptions on encoding list or dict elements and non-overflow errors on int handling getting silenced 
Fixes #273
 2022-02-16 08:17:47 +00:00
Dr. Nick e00caaebd5 dconv no longer uses global instances of StringToDoubleConverter/DoubleToStringConverter 2021-08-03 10:17:10 -04:00
Hugo van Kemenade f2d79b89c4 Remove unused variable 2021-04-07 10:23:26 +03:00
Filip Salomonsson 7a8a614017
Fix typos in error message 2021-02-03 17:32:42 +01:00
David W.H. Swenson 6013e71381
Merge remote-tracking branch 'upstream/master' into fix_large_floats 2020-11-15 19:56:19 +01:00
Hugo van Kemenade 13e2ac7eea
Merge pull request #443 from dwhswenson/match_python_exponents 2020-11-15 19:25:37 +02:00
David W.H. Swenson 954a9a0a00
cleanup 2020-11-11 16:54:57 +01:00
David W.H. Swenson a48f8b22f1
Set same bounds as std lib for negative exponent 2020-11-11 16:41:34 +01:00
David W.H. Swenson b773bf05dc
Fix errors on reading long decimal floats 2020-11-11 14:51:27 +01:00
David W.H. Swenson af699c3cd0
Match Python json output for exponents 2020-11-11 14:41:51 +01:00
Sam Sneddon f4029cc6ef Fix #429: Make empty dict/list indented serialization match stdlib json 
Previously, we'd output a couple of new lines between the start and end
of the object, whereas the stdlib doesn't bother with whitespace if
they're empty.

In my testing, the only difference in indented serialization now is
float representation.
 2020-11-10 10:57:15 +00:00
Hugo 4ae63bee5c Lint end-of-file-fixer 2020-05-12 09:36:47 +03:00
Hugo 5f1e8479fa Lint trailing-whitespace 2020-05-12 09:21:45 +03:00
Eric Le Lay e0c113e6a2 Merge branch 'master' into 264-reject_bytes 2020-05-08 17:34:35 +02:00
Hugo van Kemenade f953a0978a
Update comment 2020-05-04 09:23:39 +03:00
Hugo van Kemenade d9ca1c9b5b
Merge branch 'master' into add_nan_support 2020-03-27 21:41:33 +02:00
Hugo 61453ad7fd Fix typo 2020-03-08 00:17:27 +02:00
Hugo c810a5b8a6 Also define LIKELY/UNLIKELY for _WIN32 2020-03-08 00:17:27 +02:00
Hugo 75695ba61e Indent ifdefs 2020-03-08 00:17:27 +02:00
Natanael Copa 0f52df8f9b Reduce default buffer on stack size 
Fix segfaults on musl libc when ultrajson runs in a thread. On musl libc
the default thread stack size is only 80k so allocating a 128k buffer on
stack will guarantee a crash. There seems not to be any evident
performance benefit using big buffer on stack either so we just reduce
the default.

fixes #254
 2020-03-02 23:45:56 +02:00
Hugo f0b428ea37 Merge branch 'master' into 50-object-trailing-comma 2020-03-01 23:54:09 +02:00
Eric Le Lay b69b37f6d0
fix typo in doc (2) 
Co-Authored-By: Hugo van Kemenade <hugovk@users.noreply.github.com>
 2020-03-01 15:53:43 +01:00
Hugo van Kemenade 631850788d
Merge branch 'master' into add_nan_support 2020-02-25 22:34:37 +02:00
Hugo van Kemenade 1588690257
Merge branch 'master' into 264-reject_bytes 2020-02-25 22:28:14 +02:00
Hugo d53480c332 http -> https 2020-02-18 21:57:13 +02:00
Mark Guzman fe0e88d345
adding an allow_nan keyword argument to dumps defaulted to True 
with this ujson matches the builtin json behavior for NaN and Inf.
if a user wants to retain the old behavior they can pass allow_nan=False
to ensure strict json compatibility.
 2019-02-20 09:50:05 -05:00
Tim Dawborn 948807cfe5
Correct JSON decoding implementation to die on trailing commas in Objects. Fixes esnme/ultrajson#50. 2018-12-10 18:01:40 +11:00
Eric Le Lay ad280fd99e new reject_bytes option to raise on bytes 
raise TypeError when encountering bytes in ujson.dumps() to prevent
unexpected Unicode exceptions in production.
Fixes #264
 2017-06-11 11:58:10 +02:00
markpiekarz 629128de5e Update ultrajsondec.c 
Explicitly include stdint.h to define SIZE_MAX.
 2017-03-29 17:02:31 +11:00
Joakim Hamren eb7d894f22 Integrated google's double-conversion lib 
To fix issues with floating-point precision we've made use of Google's
double-conversion lib to handle conversions of doubles to and from strings.

In addition to fixing our precision problems this will improve double
encoding by 4-5x. Decoding is however slightly slower according to the
benchmarks - but accurate at least.

This change removes the double_precision encoding option and the
precise_float decoding option.
 2017-02-14 12:20:04 +01:00
Joakim Hamren ac4637fbc4 Following std json handling of None dict key 
Previously a None dict item key would be outputted in JSON as "None".
To better align with the standard json module this was changed to output
"null". There's no proper representation of null object keys in JSON so
this is implementation specific but it seems more natural to follow
suit when it can be done without a significant performance hit.

Added and used branch prediction macros (LIKELY/UNLIKELY) as well.
 2017-02-04 16:36:14 +01:00
Joakim Hamren 870ee48fe1 Fixes for sort_keys bug and a typo. 
- Fixed segfault when using sort_keys=True on dict with unorderable keys (GH247)

    - Fixed refcount becoming negative when using sort_keys=True (GH243)

    - Fixed compile error when defining JSON_NO_EXTRA_WHITESPACE
      caused by a wrongly named variable. (GH245)
 2017-02-04 01:07:52 +01:00
Joakim Hamrén ab6b6f88cd Merge pull request #217 from orivej/indent-dict-values 
Indent dict values
 2016-10-10 22:25:22 +02:00
Joakim Hamrén d25e024f48 Merge pull request #238 from WGH-/static-functions 
added "static" to C functions, where possible
 2016-10-10 22:06:20 +02:00
Joakim Hamren 6a0d07a7df Indentation fixes and other style fixes 2016-10-10 21:50:57 +02:00
WGH 6cf6c7ff25 added "static" to C functions, where possible 
1. It reduces clutter in symbol table.
2. It fixes issues with C99 inline semantics for functions
   marked as inline (#237, #180, #222), which manifests
   when compiled with GCC>=5.
 2016-08-27 17:34:22 +03:00
Orivej Desh 11b0d2db25 Indent dict values 2016-04-03 16:30:43 +00:00
Joakim Hamren 01cb24910e Merge branch 'correct_unit_tests_and_benchmarks' of https://github.com/timdawborn/ultrajson into timdawborn-correct_unit_tests_and_benchmarks 
Conflicts:
	tests/tests.py
 2016-01-18 06:20:14 +01:00
Joakim Hamrén 86bb8ffd10 Merge pull request #157 from wlanslovenija/raw-json 
If an object has a __json__ method, use it when encoding
 2016-01-15 05:06:19 +01:00
Joakim Hamrén 06fe4c68f0 Merge pull request #186 from shashurup/master 
escape_forward_slashes now only ignores forward slashes, not everythi…
 2016-01-15 03:50:42 +01:00
Tim Dawborn 04428508d6 * Refactored the unit testing and benchmarking code to work natively under 
both Python 2 and Python 3, removing the need for 2to3 to be run. This
  indirectly fixes #177.
* Corrected a duplicate-named method in the unit testing code. Corrected the
  now-exposed broken logic that wasn't being tested. This was
  highlighted in #186 but the author did not appear to realise that the
  method name was being masked in the unit test class.
* Corrected the test case skipping logic to use unittest.skipIf instead
  of just returning from the method upon an ImportError so that the skip
  can actually be registered as a skipped test instead of a passed test.
* Updated the tests to additionally run on Python 3.5 on Travis. This
  covers #195.
* Merged the two benchmarking files into one, and modularised the code.
  Also added native RST output so the benchmarking results can be placed
  directly into the README file.
 2015-12-29 14:23:23 +11:00