Support JSON5 features

Author: lengjingzju

Makefile

@@ -7,7 +7,7 @@
 PACKAGE_NAME    = ljson
 
 staticlib       = libljson.a
-sharedlib       = libljson.so 2 0 0
+sharedlib       = libljson.so 2 1 0
 testedbin       = ljson
 testednum       = jnum_test
 

README.md

@@ -2,7 +2,7 @@
 
 [中文版](./README_zh-cn.md)
 
-LJSON is a C implemented JSON library that is much faster than cJSON and substantially faster than RapidJSON, it is currently the fastest general-purpose JSON library.
+LJSON is a C implemented JSON library that is much faster than cJSON and substantially faster than RapidJSON, it is currently the fastest general-purpose JSON library and supports the vast majority of JSON5 features.
 LJSON supports JSON parsing, printing and editing, provides DOM and SAX APIs, and I/O supports string and file, it fully supports the test cases of nativejson-benchmark.
 By default, LJSON uses the personally developed ldouble algorithm to print double to string. Compared with the standard library, it may only be the 15th decimal place difference. It is currently the fastest double to string algorithm; users can also choose the personally optimized grisu2 algorithm or dragonbox algorithm.
 
@@ -12,6 +12,7 @@ By default, LJSON uses the personally developed ldouble algorithm to print doubl
 * Lighter: Provide a variety of methods to save memory, such as pool memory, file parsing while reading, file writing while printing, and SAX APIs. It can make memory usage a constant
 * Stronger: Support DOM and SAX-style APIs, provide APIs for JSON in classic mode and memory pool mode, support string and file as input and output, is extended to support long long integer and hexadecimal number
 * More friendly: C language implementation, does not depend on any other library, does not contain platform-related code, only one header file and source file, and the interface corresponding to cJSON. the code logic is clearer than any other JSON libraries
+* JSON5: Supports the vast majority of JSON5 features, such as hexadecimal digits, comments, array and object tail element comma, but does not support strings without double quotes
 
 ## Compile and Run
 
@@ -51,13 +52,19 @@ make O=<output path> CROSS_COMPILE=<tool prefix> && make O=<output path> DESTDIR
 
 * Set the value of the variable `JSON_ERROR_PRINT_ENABLE` in `json.c` to `1` and then re-compile
 
-### Error detection
+### Parse Config
 
-* Set the value of the variable `JSON_STRICT_PARSE_MODE` in `json.c` to `0` / `1` / `2` and then re-compile
-    * 0: Turn off not common error detection, such as trailing characters left after parsing
-    * 1: Detect more errors and allow key to be empty string
-    * 2: In addition to error detection enabled by 1, some non-standard features are also turned off, such as hexadecimal numbers, the first json object is not an array or object
-        * It 100% matches the test cases of  [nativejson-benchmark](https://github.com/miloyip/nativejson-benchmark) when set to 2
+* `#define JSON_PARSE_SKIP_COMMENT         1` : Whether to allow C-like single-line comments and multi-line comments(JSON5 feature)
+* `#define JSON_PARSE_LAST_COMMA           1` : Whether to allow comma in last element of array or object(JSON5 feature)
+* `#define JSON_PARSE_EMPTY_KEY            0` : Whether to allow empty key
+* `#define JSON_PARSE_SPECIAL_CHAR         1` : Whether to allow special characters such as newline in the string(JSON5 feature)
+* `#define JSON_PARSE_HEX_NUM              1` : Whether to allow HEX number(JSON5 feature)
+* `#define JSON_PARSE_SPECIAL_NUM          1` :Whether to allow special number such as starting with '.', '+', '0', for example: `+99` `.1234` `10.` `001`(JSON5 feature)
+* `#define JSON_PARSE_SPECIAL_DOUBLE       1` : Whether to allow special double such as `NaN`, `Infinity`, `-Infinity`(JSON5 feature)
+* `#define JSON_PARSE_SINGLE_VALUE         1` : Whether to allow json starting with non-array and non-object
+* `#define JSON_PARSE_FINISHED_CHAR        0` : Whether to allow characters other than spaces after finishing parsing
+
+Note: It 100% matches the test cases of [nativejson-benchmark](https://github.com/miloyip/nativejson-benchmark) when only `JSON_PARSE_EMPTY_KEY` is set to 1, all others are set to 0.
 
 ## Speed Test
 

README_zh-cn.md

@@ -2,7 +2,7 @@
 
 [English Edition](./README.md)
 
-LJSON 是一个远远快于 cJSON、大幅度快于 RapidJSON 的 C 实现的 JSON 库，他是目前最快的通用 JSON 库。
+LJSON 是一个远远快于 cJSON、大幅度快于 RapidJSON 的 C 实现的 JSON 库，他是目前最快的通用 JSON 库，也支持JSON5的大多数特性。
 LJSON 支持 JSON 的解析、打印、编辑，提供 DOM 和 SAX 接口，I/O 支持字符串和文件，且完全支持 nativejson-benchmark 的测试用例。
 LJSON 默认使用个人开发的 ldouble 算法打印浮点数，和标准库对比可能只有第15位小数的区别，是目前最快的浮点数转字符串算法；也可选择个人优化过的 grisu2 算法或 dragonbox 算法。
 
@@ -12,6 +12,7 @@ LJSON 默认使用个人开发的 ldouble 算法打印浮点数，和标准库
 * 更省：提供多种省内存的手段，例如内存池、文件边读边解析、边打印边写文件、SAX方式的接口，可做到内存占用是个常数
 * 更强：支持DOM和SAX风格的API，提供普通模式和内存池模式JSON的接口，支持字符串和文件作为输入输出(可扩展支持其它流)，扩展支持长长整形和十六进制数字
 * 更友好：C语言实现，不依赖任何库，不含平台相关代码，只有一个头文件和库文件，和cJSON对应一致的接口，代码逻辑比任何JSON库都更清晰
+* JSON5：支持绝大多数JSON5特性，例如十六进制数字、注释、数组和对象的尾元素逗号等，不支持无双引号的字符串
 
 ## 编译运行
 
@@ -51,13 +52,19 @@ make O=<编译输出目录> CROSS_COMPILE=<交叉编译器前缀> && make O=<编
 
 * 设置 json.c 中的变量 `JSON_ERROR_PRINT_ENABLE` 的值为 `1` 后重新编译
 
-### 错误检测
+### 解析配置
 
-* 设置 json.c 中的变量 `JSON_STRICT_PARSE_MODE` 的值为 `0` / `1` / `2` 后重新编译
-    * 0: 关闭不是常见的错误检测，例如解析完成后还剩尾后字符
-    * 1: 检测更多的错误，且允许 key 为空字符串
-    * 2: 除去 1 开启的错误检测之外，还关闭某些不是标准的特性，例如十六进制数字，第一个json对象不是array或object
-        * 设置为2时 100% 符合 [nativejson-benchmark](https://github.com/miloyip/nativejson-benchmark) 的测试用例
+* `#define JSON_PARSE_SKIP_COMMENT         1` : 是否允许类似C语言的单行注释和多行注释(JSON5特性)
+* `#define JSON_PARSE_LAST_COMMA           1` : 是否允许JSON_ARRAY或JSON_OBJECT的最后一个元素的末尾有逗号(JSON5特性)
+* `#define JSON_PARSE_EMPTY_KEY            0` : 是否允许键为空字符串
+* `#define JSON_PARSE_SPECIAL_CHAR         1` : 是否允许字符串中有特殊的字符，例如换行符(JSON5特性)
+* `#define JSON_PARSE_HEX_NUM              1` : 是否允许十六进制的解析(JSON5特性)
+* `#define JSON_PARSE_SPECIAL_NUM          1` : 是否允许特殊的数字，例如前导0，加号，无整数的浮点数等，例如 `+99` `.1234` `10.` `001` 等(JSON5特性)
+* `#define JSON_PARSE_SPECIAL_DOUBLE       1` : 是否允许特殊的double值 `NaN` `Infinity` `-Infinity`(JSON5特性)
+* `#define JSON_PARSE_SINGLE_VALUE         1` : 是否允许不是JSON_ARRAY或JSON_OBJECT开头的JSON值
+* `#define JSON_PARSE_FINISHED_CHAR        0` : 是否解析完成后忽略检查字符串尾部的合法性
+
+注：如果需要100%符合 [nativejson-benchmark](https://github.com/miloyip/nativejson-benchmark)，需要将 `JSON_PARSE_EMPTY_KEY` 置为1，其它值全部置为0。
 
 ## 性能测试
 

jnum.c

@@ -1312,11 +1312,11 @@ int jnum_dtoa(double num, char *buffer)
     switch (exponent) {
     case DP_EXPONENT_MAX:
         if (significand) {
-            memcpy(buffer, "nan", 4);
+            memcpy(buffer, "NaN", 4);
             return 3;
         } else {
-            memcpy(s, "inf", 4);
-            return signbit + 3;
+            memcpy(s, "Infinity", 9);
+            return signbit + 8;
         }
         break;
 
@@ -1360,14 +1360,14 @@ int jnum_dtoa(double num, char *buffer)
     return s - buffer;
 }
 
-int jnum_parse_hex(const char *str, jnum_type_t *type, jnum_value_t *value)
+static int jnum_parse_hex(const char *str, jnum_type_t *type, jnum_value_t *value)
 {
     const char *s = str;
     char c;
     uint64_t m = 0;
 
     s += 2;
-    while (1) {
+    while ((s - str) < 18) {
         switch ((c = *s)) {
         case '0': case '1': case '2': case '3': case '4':
         case '5': case '6': case '7': case '8': case '9':
@@ -1398,7 +1398,7 @@ int jnum_parse_hex(const char *str, jnum_type_t *type, jnum_value_t *value)
     return s - str;
 }
 
-static int jnum_parse_num_unit(const char *str, jnum_type_t *type, jnum_value_t *value)
+static int jnum_parse_num(const char *str, jnum_type_t *type, jnum_value_t *value)
 {
     static const double div10_lut[20] = {
         1    , 1e-1 , 1e-2 , 1e-3 , 1e-4 , 1e-5 , 1e-6 , 1e-7 , 1e-8 , 1e-9 ,
@@ -1420,6 +1420,17 @@ static int jnum_parse_num_unit(const char *str, jnum_type_t *type, jnum_value_t
     default: break;
     }
 
+    switch (*s) {
+    case '0': case '1': case '2': case '3': case '4':
+    case '5': case '6': case '7': case '8': case '9':
+    case '.':
+        break;
+    default:
+        *type = JNUM_NULL;
+        value->vint = 0;
+        return 0;
+    }
+
     while (*s == '0')
         ++s;
 
@@ -1430,9 +1441,6 @@ static int jnum_parse_num_unit(const char *str, jnum_type_t *type, jnum_value_t
 
     if (k < 20) {
         switch (*s) {
-        case 'e': case 'E':
-            d = m;
-            goto next4;
         case '.':
             d = m;
             goto next2;
@@ -1456,8 +1464,6 @@ static int jnum_parse_num_unit(const char *str, jnum_type_t *type, jnum_value_t
             case '0': case '1': case '2': case '3': case '4':
             case '5': case '6': case '7': case '8': case '9':
                 break;
-            case 'e': case 'E':
-                goto next4;
             case '.':
                 goto next2;
             default:
@@ -1509,37 +1515,70 @@ static int jnum_parse_num_unit(const char *str, jnum_type_t *type, jnum_value_t
             ++s;
     }
 
-    switch (*s) {
-    case 'e': case 'E':
-        goto next4;
-    default:
-        break;
-    }
-
 next3:
     *type = JNUM_DOUBLE;
     value->vdbl = sign == 1 ? d : -d;
     return s - str;
-
-next4:
-    *type = JNUM_BOOL; /* Only used to mark exponential */
-    value->vdbl = sign == 1 ? d : -d;
-    return s - str;
 }
 
-int jnum_parse_num(const char *str, jnum_type_t *type, jnum_value_t *value)
+int jnum_parse(const char *str, jnum_type_t *type, jnum_value_t *value)
 {
-    int len = jnum_parse_num_unit(str, type, value);
-    if (*type == JNUM_BOOL) {
-        jnum_value_t e;
+    const char *s = str;
+    int len = 0, len2 = 0;
+    jnum_type_t t;
+    jnum_value_t v;
+
+    while (1) {
+        switch (*s) {
+        case '\b': case '\f': case '\n': case '\r': case '\t': case '\v': case ' ':
+            ++s;
+            break;
+        default:
+            goto next;
+        }
+    }
+
+next:
+    len = s - str;
+    if (*s == '0' && (*(s + 1) == 'x' || *(s + 1) == 'X')) {
+        len2 = jnum_parse_hex(s, type, value);
+        if (len2 == 2) {
+            *type = JNUM_INT;
+            value->vint = 0;
+            return len + 1;
+        }
+        return len + len2;
+    }
+
+    len += jnum_parse_num(s, type, value);
+    if (*type == JNUM_NULL)
+        return 0;
+
+    switch (*(str + len)) {
+    case 'e': case 'E':
+        len2 = jnum_parse_num(str + len + 1, &t, &v);
+        if (t == JNUM_NULL)
+            return len;
 
-        len += 1 + jnum_parse_num_unit(str + len + 1, type, &e);
         switch (*type) {
-        case JNUM_INT: value->vdbl *= pow(10, e.vint); break;
-        case JNUM_LINT: value->vdbl *= pow(10, e.vlint); break;
-        default: value->vdbl *= pow(10, e.vdbl); break;
+        case JNUM_INT: value->vdbl = value->vint; break;
+        case JNUM_LINT: value->vdbl = value->vlint; break;
+        default: break;
         }
+
         *type = JNUM_DOUBLE;
+        len += len2 + 1;
+
+        switch (t) {
+        case JNUM_INT: value->vdbl *= pow(10, v.vint); break;
+        case JNUM_LINT: value->vdbl *= pow(10, v.vlint); break;
+        case JNUM_DOUBLE: value->vdbl *= pow(10, v.vdbl); break;
+        default: break;
+        }
+        break;
+
+    default:
+        break;
     }
 
     return len;
@@ -1551,9 +1590,6 @@ rtype fname(const char *str)                            \
     jnum_type_t type;                                   \
     jnum_value_t value;                                 \
     rtype val = 0;                                      \
-    unsigned char c = *(unsigned char *)str;            \
-                                                        \
-    while (c <= ' ' && c) c = *(unsigned char *)++str;  \
     jnum_parse(str, &type, &value);                     \
     switch (type) {                                     \
     case JNUM_BOOL:   val = (rtype)value.vbool;break;   \
@@ -1562,7 +1598,7 @@ rtype fname(const char *str)                            \
     case JNUM_LINT:   val = (rtype)value.vlint;break;   \
     case JNUM_LHEX:   val = (rtype)value.vlhex;break;   \
     case JNUM_DOUBLE: val = (rtype)value.vdbl; break;   \
-    default: break;                                     \
+    default:          val = 0;                 break;   \
     }                                                   \
     return val;                                         \
 }

jnum.h

@@ -13,6 +13,7 @@ extern "C" {
 #endif
 
 typedef enum {
+    JNUM_NULL,
     JNUM_BOOL,
     JNUM_INT,
     JNUM_HEX,
@@ -41,18 +42,7 @@ int64_t jnum_atol(const char *str);
 uint32_t jnum_atoh(const char *str);
 uint64_t jnum_atolh(const char *str);
 double jnum_atod(const char *str);
-
-int jnum_parse_hex(const char *str, jnum_type_t *type, jnum_value_t *value);
-int jnum_parse_num(const char *str, jnum_type_t *type, jnum_value_t *value);
-
-static inline int jnum_parse(const char *str, jnum_type_t *type, jnum_value_t *value)
-{
-    const char *s = str;
-
-    if (*s == '0' && (*(s+1) == 'x' || *(s+1) == 'X'))
-        return jnum_parse_hex(str, type, value);
-    return jnum_parse_num(str, type, value);
-}
+int jnum_parse(const char *str, jnum_type_t *type, jnum_value_t *value);
 
 #ifdef __cplusplus
 }